Skip to main content

OpenSearch

Nine's On-Demand OpenSearch removes the operational burden of running and maintaining search and analytics infrastructure. The service scales with your needs and is well suited for application search, log analysis, monitoring, and observability workloads.

Tiers​

OpenSearch is available in two tiers. The tier is selected via the Cluster Type configuration option during creation:

Single-Node

A single node cluster consists of one instance. This is suitable for development, testing, or workloads that do not require high availability.

Multi-Node​

A multi node cluster consists of three instances. It provides additional fault tolerance, higher availability, improved performance, and is recommended for production workloads.

Getting Started​

To get started, use nctl or Cockpit, our web interface:

  1. Create a New Service directly in Cockpit to get started. Don't forget to allow your IP to connect to the instance.
  2. Retrieve the credentials from the overview page. Note the FQDN, username and password.
  3. Connect to the newly created instance: 
    curl -XGET "https://${FQDN}/_cluster/health" -sku "${USER}:${PASSWORD}"

The resource can be managed via Cockpit, our command-line tool nctl or by directly using the API.

Billing is automated based on the resources used. Nine's automated management ensures the smooth operation of the service.

Updates and Maintenance Windows​

Security and software updates are generally performed during the NKE maintenance windows.

Therefore, short service interruptions may occur during this maintenance window without prior notice.

Pricing​

note

We recommend nine-search-s or larger sizes for production workloads. If high availability is required, a multi node cluster should be used.

XS
nine-search-xs		S
nine-search-s		M
nine-search-m		L
nine-search-l		XL
nine-search-xl
Virtual CPU (VCPU)22448
RAM2 GB4 GB8 GB16 GB32 GB
Storage space10 GB20 GB60 GB120 GB200 GB
Monthly fees (single)
Monthly fees (multi)n/a

Additional storage space per 10 GB:

Each instance includes a base amount of storage space. If usage grows beyond that, storage space expands automatically and the additional storage is billed accordingly. See Automatic Storage Space Expansion for details, including that storage space cannot be reduced afterwards.

Configuration Options​

On-Demand Services come with a sensible base configuration that is suitable for most setups. They also provide the ability to change certain configuration options. Adjusting these configuration options can result in a restart of the instance.

The instance can be customized with the following configuration options:

Name​

The name of the instance can be freely chosen, but must be unique. Once created, the name cannot be changed.

Location​

Depending on available resources, instances can be created in these locations:

API / Cockpit NameNameLocation
nine-es34NTT Zürich 1Rümlang

The location cannot be changed after the resource was created.

Version​

OpenSearch is available in version 2 and 3. We recommend using version 3.

Allowed IP Addresses​

Defines the IPv4 addresses and address ranges that are allowed to establish connections to the service. Access from our Kubernetes products (NKE and GKE) and from Deploio is enabled by default.

The access restriction can be adjusted at any time. Adjustments are made non-disruptively moments after the form is submitted.

Cluster Type​

The type of OpenSearch cluster to deploy. See Tiers for an overview of the available options.

Refer to the official documentation to learn more about configuring number_of_shards and number_of_replicas and how to determine optimal values for your use case.

Machine Type​

The amount of memory available to OpenSearch. The Java heap size scales with the configured memory.

Machine types can be changed after creation, which will restart the instances and can lead to them being unavailable for a few minutes.

Bucket Users​

A list of bucket users that will gain read access to the default snapshot repository containing the OpenSearch snapshots.

Connection​

tip

As per Allowed IP Addresses, you may need to allow additional IPs to connect to your instance.

The instances are accessible via https on the default port 443. All requests require basic auth. Check the Getting Started section for details on how to retrieve the credentials.

TLS​

note

The CA certificate will not match the hostname of the service. You may need to disable hostname verification explicitly.

The on-demand services only accept connections secured by TLS. Depending on the client or library used, it may be necessary to explicitly enable TLS.

The TLS certificate is self-signed. In addition to enabling TLS transport encryption, you may need to provide the certificate authority used for validation.

You can retrieve the CA either through Cockpit or nctl:

  1. Open the On-Demand Services overview and select the respective service.
  2. The CA is available under the resource's Certificate tab.

Deploio Environment Variables​

When this service is referenced from a Deploio application, Nine injects connection credentials as environment variables. <NAME> is the target service resource name uppercased with non-alphanumeric characters replaced by _.

VariableDescription
NINE_OS_<NAME>_FQDNHostname. Uses private networking DNS when private networking is configured; otherwise the public hostname.
NINE_OS_<NAME>_USERUsername.
NINE_OS_<NAME>_PASSWORDPassword.
NINE_OS_<NAME>_CA_CERTCA certificate. Only injected when a CA certificate is present.

Health Checks​

Nine ensures the availability of all necessary infrastructure components for running OpenSearch around the clock.

Users are responsible for investigating and optimizing OpenSearch index health, as it is primarily determined by their clients' usage patterns.

You can check cluster and index health by using the /_cluster/health and /_cluster/health/<index> endpoints.

  • red health status: The cluster has some unassigned primary shards, which means that some operations such as searches and indexing may fail.
  • yellow health status: The cluster has no unassigned primary shards but some unassigned replica shards. This increases your risk of data loss and can degrade cluster performance.
  • green health status: All primary shards and their replicas are successfully allocated to available nodes, ensuring full redundancy and complete data availability for searches and indexing.

The official OpenSearch documentation dives deeper into cluster health status.

A common cause of a yellow or red status in OpenSearch clusters is that nodes do not share a consistent view of the cluster. Heavy client-driven indexing or search traffic can cause sudden memory pressure and disrupt shard synchronisation and allocation. Changes made through the OpenSearch management API can also affect shard replicas, routing, or index templates and prevent shards from being assigned correctly.

Why Is My Cluster Initialized and Stuck With a yellow Status?​

Single-node clusters always remain with a yellow status because there are no other nodes to which OpenSearch can assign a replica.

Snapshots​

A snapshot repository named default is automatically configured and registered. This enables you to use the OpenSearch Snapshots API to manage snapshots. Usage is charged at the object storage rates.

Replace ${FQDN}, ${USER}, ${PASSWORD}, and ${SNAPSHOT_NAME} with the appropriate values in the commands below as described in the Getting Started section.

Create​

To create a snapshot in the default repository, use the following command:

curl -XPUT "https://${FQDN}/_snapshot/default/${SNAPSHOT_NAME}" -sku "${USER}:${PASSWORD}" -H 'Content-Type: application/json' -d '{
  "indices": "*",
  "ignore_unavailable": true,
  "include_global_state": false,
  "partial": false
}'

For more details on snapshot options, refer to the OpenSearch Create Snapshot documentation.

Restore​

To restore a snapshot from the default repository, use the following command.

If an index with the same name already exists in the cluster, you must close, delete, or rename it before restoring. You can use the rename_pattern and rename_replacement parameters in the request body to rename indices during restoration.

curl -XPOST "https://${FQDN}/_snapshot/default/${SNAPSHOT_NAME}/_restore" -sku "${USER}:${PASSWORD}" -H 'Content-Type: application/json' -d '{
  "indices": "*",
  "ignore_unavailable": true,
  "include_global_state": false
}'

For more details on restore options, refer to the OpenSearch Restore Snapshot documentation.

Access Snapshots Bucket​

Create a bucket user and reference it in Bucket Users to gain read access to the corresponding bucket:

  1. Create a New Bucket User directly in Cockpit. Ensure the location is the same as the OpenSearch instance.
  2. Retrieve the credentials from the overview page. Note the access and secret key.
  3. Add the users by navigating to and editing an existing OpenSearch instance.
  4. Note the snapshots bucket from the overview page.
  5. Use an Object Storage client to connect to the bucket.

Automatic Storage Space Expansion​

Stored data persists across instance restarts. To provide a stable environment, the available storage space is monitored at 5-minute intervals. If our monitoring detects that the available storage space falls below a threshold, the storage quota is expanded automatically. The additional storage space is billed automatically.

Storage space cannot be decreased even if the index size has been reduced. The only way to reduce storage usage is to create a new OpenSearch instance.