Error log: “no search context found” / “SearchContextMissingException”

Error Log: You are using a scroll query to paginate through a large result set, but after a few requests, your scroll fails with this error.

JSON

None
{

  "error": {

    "root_cause": [

      {

        "type": "search_context_missing_exception",

        "reason": "No search context found for id [scroll-id-string...]"

      }

    ],

    "type": "search_context_missing_exception",

    "reason": "No search context found for id [scroll-id-string...]"

  },

  "status": 404

}

Why… is this happening? This error means the “search context” (a lightweight, temporary snapshot of your search results) that you were paginating through has expired and been deleted by OpenSearch.

When you start a scroll query (e.g., POST /my-index/_search?scroll=1m), you are telling OpenSearch: “Please keep these search results alive for 1 minute.” OpenSearch does this and gives you a _scroll_id. Every time you send a new request with that _scroll_id, the 1-minute timer resets.

This error happens for two main reasons:

1. You waited too long: Your application took longer than the scroll timeout (e.g., longer than 1m) to send the next request. By the time the request arrived, OpenSearch had already cleaned up the expired search context.
2. Too many scrolls: OpenSearch has a limit on the total number of scroll contexts allowed open at one time (default is 500). If you open too many without closing them, the cluster will start aggressively expiring the oldest ones, even if their individual timers haven’t run out.

Best Practice:

1. Use search_after (The Modern Fix): The scroll API is an older technology. For most deep-pagination use cases, search_after is the modern, preferred, and stateless alternative. It doesn’t create a search context and is much more efficient.
2. Use point-in-time (PIT): If you need a stable snapshot of data for pagination, the Point-in-time (PIT) API is the successor to scroll. You create a PIT, which gives you an ID, and you use that ID with search_after to paginate. It’s designed to be more reliable.
3. If you must use scroll:
   • Ensure your scroll timeout (e.g., 1m, 5m) is long enough for your application to process one batch and request the next.
   • Always explicitly delete your scroll context with DELETE /_search/scroll when you are finished to free up resources.
   • Monitor GET /_nodes/stats/indices/search to check the open_contexts count.

What else can I do? Unsure whether to use scroll, search_after, or PIT? This is a common architectural question. Post your use case to the OpenSearch community forums for advice, or contact us in The OpenSearch Slack Channel in #General.