Link Search Menu Expand Document Documentation Menu
Documentation

Ingest your data into OpenSearch

There are several ways to ingest data into OpenSearch:

  • Ingest individual documents. For more information, see Indexing documents.
  • Index multiple documents in bulk. For more information, see Bulk indexing.
  • Use Data Prepper—an OpenSearch server-side data collector that can enrich data for downstream analysis and visualization. For more information, see Data Prepper.
  • Use other ingestion tools. For more information, see OpenSearch tools.

Bulk indexing

To index documents in bulk, you can use the Bulk API. For example, if you want to index several documents into the students index, send the following request:

POST _bulk
{ "create": { "_index": "students", "_id": "2" } }
{ "name": "Jonathan Powers", "gpa": 3.85, "grad_year": 2025 }
{ "create": { "_index": "students", "_id": "3" } }
{ "name": "Jane Doe", "gpa": 3.52, "grad_year": 2024 }

Experiment with sample data

OpenSearch provides a fictitious e-commerce dataset that you can use to experiment with REST API requests and OpenSearch Dashboards visualizations. You can create an index and define field mappings by downloading the corresponding dataset and mapping files.

Create a sample index

Use the following steps to create a sample index and define field mappings for the document fields:

  1. Download ecommerce-field_mappings.json. This file defines a mapping for the sample data you will use.

    To use cURL, send the following request:

     curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/2.17/assets/examples/ecommerce-field_mappings.json

    To use wget, send the following request:

     wget https://raw.githubusercontent.com/opensearch-project/documentation-website/2.17/assets/examples/ecommerce-field_mappings.json

  2. Download ecommerce.ndjson. This file contains the index data formatted so that it can be ingested by the Bulk API:

    To use cURL, send the following request:

     curl -O https://raw.githubusercontent.com/opensearch-project/documentation-website/2.17/assets/examples/ecommerce.ndjson

    To use wget, send the following request:

     wget https://raw.githubusercontent.com/opensearch-project/documentation-website/2.17/assets/examples/ecommerce.ndjson

  3. Define the field mappings provided in the mapping file: 
     curl -H "Content-Type: application/json" -X PUT "https://localhost:9200/ecommerce" -ku admin:<custom-admin-password> --data-binary "@ecommerce-field_mappings.json"

  4. Upload the documents using the Bulk API:

     curl -H "Content-Type: application/x-ndjson" -X PUT "https://localhost:9200/ecommerce/_bulk" -ku admin:<custom-admin-password> --data-binary "@ecommerce.ndjson"

Query the data

Query the data using the Search API. The following query searches for documents in which customer_first_name is Sonya:

GET ecommerce/_search
{
  "query": {
    "match": {
      "customer_first_name": "Sonya"
    }
  }
}

Visualize the data

To learn how to use OpenSearch Dashboards to visualize the data, see the OpenSearch Dashboards quickstart guide.

Further reading

Next steps

350 characters left

Have a question? .

Want to contribute? or .