You're viewing version 2.11 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.

Data streams

Introduced 2.6

In OpenSearch Dashboards, the Index Management application allows you to view and manage data streams as shown in the following image.

Viewing a data stream

To view a data stream and its health status, choose Data streams under Index management as shown in the following image.

The following are the three data stream health statuses:

• Green: All primary and replica shards are assigned.
• Yellow: At least one replica shard is not assigned.
• Red: At least one primary shard is not assigned.

Creating a data stream

To create a data stream, perform the following steps:

1. Under Index Management, choose Data streams.

2. Choose Create data stream.

3. Enter a name for the data stream under Data stream name.

4. Ensure that you have a matching index template. This will be populated under Matching index template, as shown in the following image.

   

5. The Inherited settings from template and Index alias sections are read-only, and display the backing indexes that are contained in the data stream.

6. The number of primary shards, number of replicas, and the refresh interval are inherited from the template, as shown in the following image.

   

7. Choose Create data stream.

Deleting a data stream

To delete a data stream, perform the following steps:

1. Under Index Management, choose Data streams.

2. Select the data stream that you want to delete.

3. Choose Actions, and then choose Delete.

Rolling over a data stream

To perform a rollover operation on a data stream, perform the following steps:

1. Under Index Management, choose Data streams.

2. Choose Actions, and then choose Roll over, as shown in the following image.

   

3. Under Configure source, select the source data stream on which you want to perform the rollover operation.

4. Choose Roll over, as shown in the following image.

   

Force merging data streams

To perform a force merge operation on two or more indexes, perform the following steps:

1. Under Index Management, choose Data streams.

2. Select the data streams on which you want to perform the force merge operation.

3. Choose Actions, and then choose Force merge.

4. Under Configure source index, specify the data streams you want to force merge.

5. Optionally, under Advanced settings you can to choose to Flush indices or Only expunge delete and then specify the Max number of segments to merge to as shown in the following image.

   

Refreshing a data stream

Refreshing a data stream makes new updates to the index visible to search operations.

The refresh operation can be applied only to open indexes associated with the specified data streams.

To refresh a data stream, select the data stream from the Data streams list under Index Management. Then select Refresh from the Actions dropdown list.

Flushing a data stream

The flush operation performs a Lucene commit, writing segments to disk and starting a new translog.

The flush operation can be applied only to open indexes associated with the specified data streams.

To flush a data stream, select the data stream from the Data streams list under Index Management. Then select Flush from the Actions dropdown list.

Clearing a data stream cache

The clear cache operation can be applied only to open indexes associated with the specified data streams.

To clear a data stream cache, select the index from the Indices list under Index Management. Then select Clear cache from the Actions dropdown list.