You're viewing version 1.0 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.
Index rollups API
Use the index rollup operations to programmatically work with index rollup jobs.
Table of contents
- Create or update an index rollup job
- Get an index rollup job
- Delete an index rollup job
- Start or stop an index rollup job
- Explain an index rollup job
Create or update an index rollup job
Introduced 1.0
Creates or updates an index rollup job. You must provide the seq_no and primary_term parameters.
Request
PUT _plugins/_rollup/jobs/<rollup_id> // Create
PUT _plugins/_rollup/jobs/<rollup_id>?if_seq_no=1&if_primary_term=1 // Update
{
"rollup": {
"source_index": "nyc-taxi-data",
"target_index": "rollup-nyc-taxi-data",
"schedule": {
"interval": {
"period": 1,
"unit": "Days"
}
},
"description": "Example rollup job",
"enabled": true,
"page_size": 200,
"delay": 0,
"roles": [
"rollup_all",
"nyc_taxi_all",
"example_rollup_index_all"
],
"continuous": false,
"dimensions": {
"date_histogram": {
"source_field": "tpep_pickup_datetime",
"fixed_interval": "1h",
"timezone": "America/Los_Angeles"
},
"terms": {
"source_field": "PULocationID"
},
"metrics": [
{
"source_field": "passenger_count",
"metrics": [
{
"avg": {}
},
{
"sum": {}
},
{
"max": {}
},
{
"min": {}
},
{
"value_count": {}
}
]
}
]
}
}
}
You can specify the following options.
| Options | Description | Type | Required |
|---|---|---|---|
source_index | The name of the detector. | String | Yes |
target_index | Specify the target index that the rolled up data is ingested into. You could either create a new target index or use an existing index. The target index cannot be a combination of raw and rolled up data. | String | Yes |
schedule | Schedule of the index rollup job which can be an interval or a cron expression. | Object | Yes |
schedule.interval | Specify the frequency of execution of the rollup job. | Object | No |
schedule.interval.start_time | Start time of the interval. | Timestamp | Yes |
schedule.interval.period | Define the interval period. | String | Yes |
schedule.interval.unit | Specify the time unit of the interval. | String | Yes |
schedule.interval.cron | Optionally, specify a cron expression to define therollup frequency. | List | No |
schedule.interval.cron.expression | Specify a Unix cron expression. | String | Yes |
schedule.interval.cron.timezone | Specify timezones as defined by the IANA Time Zone Database. Defaults to UTC. | String | No |
description | Optionally, describe the rollup job. | String | No |
enabled | When true, the index rollup job is scheduled. Default is true. | Boolean | Yes |
continuous | Specify whether or not the index rollup job continuously rolls up data forever or just executes over the current data set once and stops. Default is false. | Boolean | Yes |
error_notification | Set up a Mustache message template sent for error notifications. For example, if an index rollup job fails, the system sends a message to a Slack channel. | Object | No |
page_size | Specify the number of buckets to paginate through at a time while rolling up. | Number | Yes |
delay | The number of milliseconds to delay execution of the index rollup job. | Long | No |
dimensions | Specify aggregations to create dimensions for the roll up time window. | Object | Yes |
dimensions.date_histogram | Specify either fixed_interval or calendar_interval, but not both. Either one limits what you can query in the target index. | Object | No |
dimensions.date_histogram.fixed_interval | Specify the fixed interval for aggregations in milliseconds, seconds, minutes, hours, or days. | String | No |
dimensions.date_histogram.calendar_interval | Specify the calendar interval for aggregations in minutes, hours, days, weeks, months, quarters, or years. | String | No |
dimensions.date_histogram.field | Specify the date field used in date histogram aggregation. | String | No |
dimensions.date_histogram.timezone | Specify the timezones as defined by the IANA Time Zone Database. The default is UTC. | String | No |
dimensions.terms | Specify the term aggregations that you want to roll up. | Object | No |
dimensions.terms.fields | Specify terms aggregation for compatible fields. | Object | No |
dimensions.histogram | Specify the histogram aggregations that you want to roll up. | Object | No |
dimensions.histogram.field | Add a field for histogram aggregations. | String | Yes |
dimensions.histogram.interval | Specify the histogram aggregation interval for the field. | Long | Yes |
dimensions.metrics | Specify a list of objects that represent the fields and metrics that you want to calculate. | Nested object | No |
dimensions.metrics.field | Specify the field that you want to perform metric aggregations on. | String | No |
dimensions.metrics.field.metrics | Specify the metric aggregations you want to calculate for the field. | Multiple strings | No |
Sample response
{
"_id": "rollup_id",
"_seqNo": 1,
"_primaryTerm": 1,
"rollup": { ... }
}
Get an index rollup job
Introduced 1.0
Returns all information about an index rollup job based on the rollup_id.
Request
GET _plugins/_rollup/jobs/<rollup_id>
Sample response
{
"_id": "my_rollup",
"_seqNo": 1,
"_primaryTerm": 1,
"rollup": { ... }
}
Delete an index rollup job
Introduced 1.0
Deletes an index rollup job based on the rollup_id.
Request
DELETE _plugins/_rollup/jobs/<rollup_id>
Sample response
200 OK
Start or stop an index rollup job
Introduced 1.0
Start or stop an index rollup job.
Request
POST _plugins/_rollup/jobs/<rollup_id>/_start
POST _plugins/_rollup/jobs/<rollup_id>/_stop
Sample response
200 OK
Explain an index rollup job
Introduced 1.0
Returns detailed metadata information about the index rollup job and its current progress.
Request
GET _plugins/_rollup/jobs/<rollup_id>/_explain
Sample response
{
"example_rollup": {
"rollup_id": "example_rollup",
"last_updated_time": 1602014281,
"continuous": {
"next_window_start_time": 1602055591,
"next_window_end_time": 1602075591
},
"status": "running",
"failure_reason": null,
"stats": {
"pages_processed": 342,
"documents_processed": 489359,
"rollups_indexed": 3420,
"index_time_in_ms": 30495,
"search_time_in_ms": 584922
}
}
}