You're viewing version 2.16 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.

Neural sparse query two-phase processor

Introduced 2.15

The neural_sparse_two_phase_processor search processor is designed to provide faster search pipelines for neural sparse search. It accelerates the neural sparse query by dividing the original method of scoring all documents with all tokens into two steps:

1. High-weight tokens score the documents and filter out the top documents.
2. Low-weight tokens rescore the top documents.

Request fields

The following table lists all available request fields.

Example

The following example creates a search pipeline with a neural_sparse_two_phase_processor search request processor.

Create search pipeline

The following example request creates a search pipeline with a neural_sparse_two_phase_processor search request processor. The processor sets a custom model ID at the index level and provides different default model IDs for two specific index fields:

PUT /_search/pipeline/two_phase_search_pipeline
{
  "request_processors": [
    {
      "neural_sparse_two_phase_processor": {
        "tag": "neural-sparse",
        "description": "This processor is making two-phase processor.",
        "enabled": true,
        "two_phase_parameter": {
          "prune_ratio": custom_prune_ratio,
          "expansion_rate": custom_expansion_rate,
          "max_window_size": custom_max_window_size
        }
      }
    }
  ]
}

Set search pipeline

After the two-phase pipeline is created, set the index.search.default_pipeline setting to the name of the pipeline for the index on which you want to use the two-phase pipeline:

PUT /index-name/_settings 
{
  "index.search.default_pipeline" : "two_phase_search_pipeline"
}

Limitation

The neural_sparse_two_phase_processor has the following limitations.

Version support

The neural_sparse_two_phase_processor can only be used with OpenSearch 2.15 or later.

Compound query support

As of OpenSearch 2.15, only the Boolean compound query is supported.

Neural sparse queries and Boolean queries with a boost parameter (not boosting queries) are also supported.

Examples

The following examples show neural sparse queries with the supported query types.

Single neural sparse query

GET /my-nlp-index/_search
{
  "query": {
    "neural_sparse": {
      "passage_embedding": {
        "query_text": "Hi world"
        "model_id": <model-id>
      }
    }
  }
}

Neural sparse query nested in a Boolean query

GET /my-nlp-index/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "neural_sparse": {
            "passage_embedding": {
              "query_text": "Hi world",
              "model_id": <model-id>
            },
            "boost": 2.0
          }
        }
      ]
    }
  }
}

P99 latency metrics

Using an OpenSearch cluster set up on three m5.4xlarge Amazon Elastic Compute Cloud (Amazon EC2) instances, OpenSearch conducts neural sparse query P99 latency tests on indexes corresponding to more than 10 datasets.

Doc-only mode latency metric

In doc-only mode, the two-phase processor can significantly decrease query latency, as shown by the following latency metrics:

• Average latency without the two-phase processor: 53.56 ms
• Average latency with the two-phase processor: 38.61 ms

This results in an overall latency reduction of approximately 27.92%. Most indexes show a significant latency reduction when using the two-phase processor, with reductions ranging from 5.14 to 84.6%. The specific latency optimization values depend on the data distribution within the indexes.

Bi-encoder mode latency metric

In bi-encoder mode, the two-phase processor can significantly decrease query latency, as shown by the following latency metrics:

• Average latency without the two-phase processor: 300.79 ms
• Average latency with the two-phase processor: 121.64 ms

This results in an overall latency reduction of approximately 59.56%. Most indexes show a significant latency reduction when using the two-phase processor, with reductions ranging from 1.56 to 82.84%. The specific latency optimization values depend on the data distribution within the indexes.