You're viewing version 2.17 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.
Foreach processor
The foreach
processor is used to iterate over a list of values in an input document and apply a transformation to each value. This can be useful for tasks like processing all the elements in an array consistently, such as converting all elements in a string to lowercase or uppercase.
The following is the syntax for the foreach
processor:
{
"foreach": {
"field": "<field_name>",
"processor": {
"<processor_type>": {
"<processor_config>": "<processor_value>"
}
}
}
}
Configuration parameters
The following table lists the required and optional parameters for the foreach
processor.
Parameter | Required/Optional | Description |
---|---|---|
field | Required | The array field to iterate over. |
processor | Required | The processor to execute against each field. |
ignore_missing | Optional | If true and the specified field does not exist or is null, then the processor will quietly exit without modifying the document. |
description | Optional | A brief description of the processor. |
if | Optional | A condition for running the processor. |
ignore_failure | Optional | Specifies whether the processor continues execution even if it encounters an error. If set to true , then failures are ignored. Default is false . |
on_failure | Optional | A list of processors to run if the processor fails. |
tag | Optional | An identifier tag for the processor. Useful for debugging in order to distinguish between processors of the same type. |
Using the processor
Follow these steps to use the processor in a pipeline.
Step 1: Create a pipeline
The following query creates a pipeline named test-foreach
that uses the foreach
processor to iterate over each element in the protocols
field:
PUT _ingest/pipeline/test-foreach
{
"description": "Lowercase all the elements in an array",
"processors": [
{
"foreach": {
"field": "protocols",
"processor": {
"lowercase": {
"field": "_ingest._value"
}
}
}
Step 2 (Optional): Test the pipeline
It is recommended that you test your pipeline before you ingest documents.
To test the pipeline, run the following query:
POST _ingest/pipeline/test-foreach/_simulate
{
"docs": [
{
"_index": "testindex1",
"_id": "1",
"_source": {
"protocols": ["HTTP","HTTPS","TCP","UDP"]
}
}
]
}
Response
The following example response confirms that the pipeline is working as expected, showing that the four elements have been lowercased:
{
"docs": [
{
"doc": {
"_index": "testindex1",
"_id": "1",
"_source": {
"protocols": [
"http",
"https",
"tcp",
"udp"
]
},
"_ingest": {
"_value": null,
"timestamp": "2024-05-23T02:44:10.8201Z"
}
}
}
]
}
Step 3: Ingest a document
The following query ingests a document into an index named testindex1
:
POST testindex1/_doc/1?pipeline=test-foreach
{
"protocols": ["HTTP","HTTPS","TCP","UDP"]
}
Response
The request indexes the document into the index testindex1
and applies the pipeline before indexing:
{
"_index": "testindex1",
"_id": "1",
"_version": 6,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 5,
"_primary_term": 67
}
Step 4 (Optional): Retrieve the document
To retrieve the document, run the following query:
GET testindex1/_doc/1
Response
The response shows the document with the extracted JSON data from the users
field:
{
"_index": "testindex1",
"_id": "1",
"_version": 6,
"_seq_no": 5,
"_primary_term": 67,
"found": true,
"_source": {
"protocols": [
"http",
"https",
"tcp",
"udp"
]
}
}
{
"docs": [
{
"doc": {
"_index": "testindex1",
"_id": "1",
"_source": {
"protocols": [
"http",
"https",
"tcp",
"udp"
]
},
"_ingest": {
"_value": null,
"timestamp": "2024-05-23T02:44:10.8201Z"
}
}
}
]
}