Advancing search with OpenSearch v3 neural sparse models and a multilingual retrieval model

Neural sparse search is a powerful and efficient method for semantic retrieval in OpenSearch. It encodes text into (token, weight) entries, allowing OpenSearch to index and search efficiently using Lucene’s inverted index. Since its introduction in OpenSearch 2.11 and the improvements brought by the v2 series, neural sparse search has delivered strong search relevance alongside the efficiency benefits of inference‑free retrieval. Today, we’re excited to share two major advancements:

• The v3 series of neural sparse models – Our most accurate sparse retrieval models to date, delivering substantial gains in search relevance while maintaining lightweight, inference‑free efficiency.
• A new multilingual retrieval model – The first multilingual neural sparse retrieval model in OpenSearch.

Neural sparse search v3 models: Advancing search relevance

We are excited to announce the release of our v3 neural sparse models:

• v3-distill: Building on the success of the v2-distill model, v3-distill delivers higher search relevance (0.517 NDCG@10 vs. 0.504 for v2-distill) through improved training while retaining its lightweight architecture for fast ingestion and low memory usage.
• v3-gte: Our most accurate v3 model, offering the best search relevance across all benchmarks (0.546 NDCG@10 vs. 0.517 for v3-distill) while maintaining the high efficiency and low-latency performance of doc-only sparse retrieval.

The v3 models are now available in both OpenSearch and Hugging Face.

All v3 models achieve better search relevance than their v2 counterparts. The following table compares search relevance across model generations.

From v2 to v3 series models

The transition from v2 to v3 models in OpenSearch represents a significant leap forward in neural sparse search relevance while maintaining the hallmark efficiency of the v2 series.

Limitations of v2 models

The v2 series models made neural sparse search widely accessible by significantly reducing the number of model parameters and improving ingestion throughput while maintaining nearly the same search relevance. However, as search workloads and datasets grew in complexity, certain challenges emerged:

• Relevance bottleneck: While v2 models delivered strong efficiency and solid performance, their inference-free design still trailed behind well-trained Siamese dense or sparse retrievers in retrieval quality.
• Limited teacher guidance: v2 models relied primarily on heterogeneous bi-encoder teachers for distillation, without using the richer ranking signals from stronger models, such as large language models (LLMs).

These limitations motivated us to rethink both training strategies and model architecture for the next-generation models.

Advancements in v3 models

For the v3 series, our primary goal was to push search relevance to a new level while retaining the lightweight and low-latency characteristics of v2 models. Key advancements include the following:

• v3-distill: Builds on v2-distill by incorporating ℓ0-based sparsification techniques and training on a larger and more diverse dataset. This combination improves search relevance while maintaining the same lightweight architecture for fast ingestion and low memory usage.
• v3-gte: Replaces the v3-distill backbone with a General Text Embedding (GTE) architecture, providing stronger semantic representation and support for 8192-token context windows. This model employs LLM teacher models, capturing richer semantic nuances and setting a new benchmark for sparse retrieval relevance in OpenSearch.

The technology behind v3 models

Two core techniques drive the technology improvements in v3 models: ℓ0-based sparsification for efficient document representation and GTE architecture with LLM teachers for enhanced training quality.

With these advancements, the v3 series delivers substantial improvements in search relevance while preserving the hallmark speed, efficiency, and inference-free advantages of previous generations. This ensures that you can achieve state-of-the-art retrieval performance without compromising scalability or latency.

ℓ0-based sparsification

The ℓ0-based approach selectively sparsifies document-side representations to balance efficiency and ranking quality:

• ℓ0 mask loss: Regularizes only document vectors exceeding the desired sparsity threshold.
• ℓ0 approximation activation: Provides a differentiable approximation for ℓ0, enabling precise sparsity control during training.

Combined with expanded training data, this enables v3-distill to achieve higher relevance without sacrificing efficiency.

GTE architecture with LLM teachers

The GTE architecture strengthens semantic representation and handles much longer inputs, while LLM-based teacher signals offer richer ranking guidance. This combination allows v3-gte to deliver the highest relevance scores among all OpenSearch sparse retrievers.

Search relevance benchmarks

Similarly to the tests described in our previous blog post, we evaluated the search relevance of the models on the BEIR benchmark. The search relevance results are shown in the following table. All v3 series models outperform their v2 and v1 counterparts, with v3‑gte achieving the highest relevance scores across all tests and setting a new record for OpenSearch neural sparse retrieval models.

Multilingual sparse retrieval

We are also excited to announce multilingual-v1, the first multilingual neural sparse retrieval model in OpenSearch. Using the same proven training techniques as the English-language v2 series, multilingual-v1 brings high‑quality sparse retrieval to a wide range of languages, achieving strong relevance across multilingual benchmarks while maintaining the same efficiency as our English-language models.

The following table shows the detailed search relevance evaluation of multilingual-v1 across different languages, compared to BM25. Results are reported using the MIRACL benchmark. multilingual-v1 delivers substantial improvements over BM25 in all languages, demonstrating the effectiveness of applying our neural sparse retrieval techniques beyond the English language. The table also presents results for a pruned version of multilingual-v1 (using a prune ratio of 0.1), which maintains competitive relevance while reducing index size.

Further reading

For more information about neural sparse search, see these previous blog posts:

• Neural sparse models are now available in Hugging Face Sentence Transformers
• Improving search efficiency and accuracy with the newest v2 neural sparse models
• Improving document retrieval with sparse semantic encoders
• A deep dive into faster semantic sparse retrieval in OpenSearch 2.12
• Introducing the neural sparse two-phase algorithm

Next steps

Try our newest v3 neural sparse models in your OpenSearch cluster and share your experience with us on the OpenSearch forum. Your feedback helps us improve future versions.