OpenSearch 3.0 has arrived! The OpenSearch Project’s first major version since 2022 propels OpenSearch forward with significant advancements in performance, data management, vector database functionality, and much more.
Why three years between major versions? The project steered clear of a dot zero release in keeping with semantic versioning, which means that OpenSearch will only release breaking changes in major versions. The inherent performance benefits that flowed from the release of Apache Lucene 10 were a catalyst in the decision to move forward with 3.0, and the community of OpenSearch developers seized the moment to deliver a slew of technical innovations that elevate OpenSearch to a new plane of capability, performance, and versatility. Thanks to their efforts, and with the oversight of our Technical Steering Committee, the OpenSearch 3.x line is here and ready to help you tackle the high-powered workloads and massive datasets that tomorrow’s applications require. Read on for highlights from this release and see the release notes for a deeper dive into the new functionality and a comprehensive list of breaking changes.
Cost, performance, and scalability
OpenSearch has been making significant advancements in performance (and prevailing in benchmark comparisons) as the project strives to stay ahead of the performance demands of data-intensive search, observability, and generative AI workloads. The 3.0 release further accelerates these performance gains, delivering improvements of 20% on aggregate across high-impact operations such as desc_sort_after_timestamp, query_string_on_messagem, and cardinality_agg_high compared to OpenSearch 2.19; compared to OpenSearch 1.3, this release performs more than 9.5x faster across key query types.
Benefit from Lucene 10 upgrades
The latest version of Apache Lucene offers significant improvements in performance, efficiency, and vector search functionality. These types of improvements pave the way for larger vector and search deployments, enabling AI workloads to scale factorially over time. The Lucene 10 release post includes all of the details; here are some highlights:
- Indexing performance: Lucene 10 optimized the indexing process for vector fields, resulting in faster indexing times and reduced index sizes for vector data.
- Sparse indexing: When used with index sorting to group similar documents together, sparse indexing creates CPU and storage efficiency improvements.
- Vector quantization: New compression techniques for vector representations have been created, reducing memory usage and improving query performance.
Additional improvements include better hybrid search through text/semantic search integration, enhanced in-engine performance monitoring, and more.
Improve data transport performance with protobuf and gRPC
OpenSearch 3.0 takes a major step toward higher-performance data transport between clients and servers with experimental support for protocol buffers (protobuf) over gRPC transport. Benefiting from underlying HTTP/2 infrastructure, gRPC supports multiplexing and bidirectional data streams, enabling clients to send and receive requests concurrently over the same TCP connection. Performance gains can be especially pronounced for users working with large and complex queries, where the overhead of deserializing requests can compound when using JSON. Users who want to cut the cost of serializing JSON or are looking for an easier way to integrate OpenSearch into their existing gRPC ecosystems should stay tuned for expanded support for protobuf and gRPC as we approach general availability in an upcoming release.
Index streaming data with pull-based ingestion
This release brings pull-based ingestion to OpenSearch as an experimental feature. Now users can configure an OpenSearch index to fetch data from a streaming data source or continue to load data into OpenSearch through the REST API as before. A key advantage of pull-based ingestion is its ability to inherently handle backpressure and pace control of data ingestion, ensuring more stable and resilient data pipelines. This release includes plugins to support pull-based ingestion from Apache Kafka and Amazon Kinesis data sources. Many thanks to the Search Platform Team at Uber for their contributions to bringing this functionality to the project.
Improve range query performance by 25%
OpenSearch 3.0 incorporates smarter strategies for range queries on numeric and date fields. By using approximation techniques like multi-level interval tree filters, OpenSearch can answer range filters with far fewer I/O operations. In practice, this means that queries like timestamp >= X AND timestamp <= Y, or numeric filters on large data, will run faster with less load on the cluster. These enhancements especially benefit log analytics and time-series use cases, delivering 25% faster performance in the Big5 benchmarks compared to OpenSearch 2.19.
Reduce latency for high-cardinality aggregations by up to 75%
When dealing with analytics queries on fields with extremely high cardinality (a large number of unique values) or wide numeric ranges, OpenSearch 3.0 offers new optimizations to speed up results. One improvement is an execution hint for the cardinality aggregation that estimates unique value counts. In datasets with millions of unique terms, the cardinality aggregator can consume significant memory. Now this execution hint lets you choose an algorithmic strategy (such as precision_threshold) to balance accuracy and performance, avoiding heavy memory use on high-cardinality fields. The net effect on p90 latency delivered a 75% reduction in latency in benchmark testing compared to OpenSearch 2.19.
Support sub-aggregations in filter rewrite optimization
As part of an ongoing effort to reduce query latency for date histogram aggregation workloads, OpenSearch 3.0 enhances filter rewrite optimization for aggregations with sub-aggregation support. Previously, this optimization only applied to single-level aggregations without any sub-aggregations; now its performance gains apply to any sub-aggregations as well, expanding the functionality to more complex real-world use cases. This optimization demonstrated improvements of 30% to 40% for relevant operations in the Big5 workload.
Vector database and generative AI
OpenSearch 3.0 is packed with new functionality to boost performance and simplify the task of building powerful vector-driven applications for search, observability, and generative AI workloads.
Power agentic AI with native MCP protocol support
With version 3.0, OpenSearch introduces native Model Context Protocol (MCP) support on both the server and client side as an experimental feature, enabling seamless integration with AI agents. The open protocol standardizes communications between large language model (LLM) applications, data sources, and tools; with MCP client support, users can incorporate their existing tools and systems into the ML Commons agent framework, enabling more comprehensive and customizable AI-powered solutions. On the server side, OpenSearch can now expose operations such as searching indexes and pulling index stats as tools, making it easier to integrate OpenSearch with external AI agents such as Anthropic, LangChain, and OpenAI.
Deploy the plan-execute-reflect agent to support complex analyses
Another addition to the ML Commons agent framework is the experimental plan-execute-reflect agent, which can autonomously solve complex tasks using planning and reflection. This agent breaks down intricate questions into manageable steps, selects suitable tools for execution, and iteratively improves its strategies through reflection. For example, when analyzing a service error, the agent can automatically analyze logs, query relevant indexes, search the web for related issues, and summarize findings to assist in troubleshooting.
Cut storage costs by as much as 3x with derived source for vectors
First released in 2.19 as an experimental feature, derived source for k-NN vectors is production-ready in OpenSearch 3.0 for all vector indexes. Derived source lets you access vector values during searching and reindexing without the need to store the entire source vector in OpenSearch. By removing vectors from the JSON source and dynamically injecting them when needed, derived vectors improve query performance, delivering up to a 30x improvement in p90 cold start query latencies for the Lucene engine and reducing merge times by up to 40% across all engines. This functionality has also demonstrated a 3x reduction in storage costs across the Faiss, Lucene, and NMSLIB libraries, unlocking further optimizations for vector workloads.
Build faster vector solutions with GPU acceleration
Vector operations are computationally intensive, particularly at large scale. This makes them ideally suited to parallel processing, where GPUs excel. As an experimental feature, this release unlocks the ability to deploy GPUs to power index builds for significantly faster index creation, lower operational costs, and enhanced fault tolerance. According to benchmark tests, GPU acceleration can improve indexing speed by 9.3x while reducing costs by 3.75x compared to CPU-based solutions. For more on how GPUs can help optimize your vector search applications, check out our blog post GPU-accelerated vector search in OpenSearch: A new frontier. Special thanks to the NVIDIA team for their contributions to this groundbreaking functionality.
Improve relevance across search results with semantic sentence highlighting
Semantic sentence highlighting enhances search result relevance by using machine learning to identify and highlight relevant sentences based on meaning—not just keyword matches. Unlike traditional highlighting that relies on exact text matching, this feature captures context and preserves complete thoughts by highlighting entire sentences. Semantic highlighting works with all query types, including traditional, neural, and hybrid searches, making search results more meaningful even when exact keywords aren’t present. The release includes a pretrained opensearch-semantic-highlighter-v1 model optimized for identifying semantically relevant sentences across general domains. Users can easily deploy this model to their cluster and reference it in highlight configurations, eliminating the need to train custom models for basic semantic highlighting use cases.
Boost binary quantization (32x) performance by 2.5x with concurrent segment search
Concurrent segment search is now enabled by default for OpenSearch k-NN queries. With concurrent segment search enabled, you can accelerate the performance of k-NN queries by up to 2.5x with no impact on recall by parallelizing search queries across multiple threads. This release also enables changes to the floor segment size setting in the merge policy to create more balanced segments and improve tail latencies by up to 20%.
Gain insights into vector search scoring with the Explain API
As of this release, you can use the explain parameter to gain insights into how scores are calculated, normalized, and combined in k-NN queries using the Faiss engine. This functionality offers detailed information about the scoring process for each search result, exposing the score normalization techniques used, how different scores were combined, and the calculations for individual subquery scores. This comprehensive insight makes it easier to understand and optimize your query results.
Search
OpenSearch 3.0 also brings a range of new functionality to enhance search operations and improve search results.
Enhance hybrid search results with Z-score normalization
This release introduces Z-score normalization, a new statistical approach to score normalization in hybrid search. While the default score-based normalization method—min-max normalization—can struggle with outliers, Z-score normalization excels by transforming raw scores into standardized units. This technique considers both the score distribution of each subquery and handles outliers effectively, making scores directly comparable across different query types. This approach significantly improves hybrid search reliability by reducing the impact of outliers and different score scales, resulting in more consistent and meaningful search results.
Improve hybrid search result relevance with a lower bound for min-max normalization
This release addresses a key limitation in hybrid search relevance with the addition of a lower bound for min-max normalization. Traditional min-max normalization can disproportionately elevate documents with extremely low scores when combining neural and keyword search results, leading to suboptimal rankings. A lower bound lets you establish a minimum threshold during normalization, preventing the over-amplification of negligible scores that previously could dominate search results and ensuring that the normalized values remain meaningful and proportionate to their actual relevance. When scores fall below the specified lower bound, they’re adjusted to maintain an appropriate ranking order, which is particularly beneficial for queries where one search method yields minimal relevance signals.
Improve hybrid query results with inner hits support
This release adds support for inner hits in hybrid queries. Inner hits are the underlying hits that are hidden by default when performing a search operation on nested fields or parent-join fields. For hybrid queries, inner hits are extracted based on each parent document present in the final search results. The parent document will show the hybrid score, and the inner hits will show raw scores prior to normalization. To use this feature, users will need to add an inner_hits clause in the search request.
Reduce query overhead for aggregations with star-tree indexing
For aggregation-heavy workloads, OpenSearch 3.0 continues to build upon the star-tree indexing feature, introduced in 2.19, which can dramatically speed up certain aggregations. In this release, star-tree support is expanded to handle metric aggregations and filtered terms queries within aggregations; instead of scanning every document for a heavy aggregation (for example, computing a histogram over billions of records), OpenSearch can answer the query by reading much smaller pre-aggregated segments. Early results have shown up to a 100x reduction in query work and 30x lower cache usage for star-tree powered aggregations, with particularly significant benefits for high-cardinality group-by operations and multi-level aggregations.
Observability, log analytics, and security analytics
This release includes new tools to support observability, log analytics, and security analytics workloads.
Enrich and manage log data more efficiently with PPL query advancements
This release adds powerful capabilities to OpenSearch’s Piped Processing Language (PPL) tools with the addition of lookup, join, and subsearch commands. These features enable users—especially those in observability and security domains—to enrich, correlate, and filter logs more efficiently across large datasets. For example, security engineers can now join authentication and application logs to investigate incidents or use lookup to enrich logs with geo-context in real time. Backed by Apache Calcite, these enhancements also bring improved query planning and execution, laying the groundwork for more advanced analytics capabilities in future releases. This release marks a significant step forward in making PPL a more expressive and performant language for interactive data exploration.
Improve query monitoring and performance analysis
OpenSearch 3.0 enhances Query Insights with several powerful features that improve query monitoring and performance analysis. A new Live Queries API provides real-time visibility into currently executing search queries across your cluster, helping you identify long-running or resource-intensive operations as they happen. This allows users to assess the impact of resource-intensive queries and potentially take proactive measures, such as query cancellation, to maintain system stability and resiliency. The verbose parameter optimizes dashboard performance by allowing users to fetch lightweight versions of query data when full details aren’t needed, significantly reducing payload size and improving responsiveness. New dynamic columns in the Query Insights dashboard can intelligently adapt based on your filter selections, showing only relevant metrics for queries, groups, or combined views. Together, these features deliver more efficient monitoring and troubleshooting capabilities while maintaining dashboard performance even with thousands of query records.
Explore anomalies directly from the Discover interface
This release also brings further enhancements to the user experience for observability and log analytics workloads. As an example, with contextual launch, the Discover interface now lets you launch an anomaly detector from the main dashboard and populate all the relevant logs in the Discover view. This eliminates the need to manually query your logs after an anomaly is detected; now OpenSearch takes care of that for you.
Modular architecture
This release includes updates to OpenSearch’s modular architecture, such as the following.
Improve resource utilization with separate reads and writes for remote store clusters
As of this release, OpenSearch will support an architecture that separates indexing and search traffic within remote-store-enabled clusters. Separating indexing and search operations across the cluster provides failure isolation, enables independent scaling, and offers improved performance and cost efficiency, particularly for use cases that require high-volume indexing along with intensive search operations. For write-once, ready-many use cases, such as log analytics, this release introduces a new _scale API that allows you to turn of all writers and make the index search only.
Security
OpenSearch 3.0 includes several significant updates to help you maintain the security of your OpenSearch deployments.
Replacing the Java Security Manager in OpenSearch 3.0
In JDK 17, Java marked the Java Security Manager for deprecation and eventual removal. With JDK 24, Java has now permanently disabled the Security Manager and rendered this part of the Java language obsolete. OpenSearch has relied on this Java functionality to provide proper sandboxing to plugins that run in the same JVM as the OpenSearch process. In OpenSearch 3.0, the Java Security Manager has been replaced with a new Java agent that provides low-level instrumentation allowing OpenSearch to intercept privileged calls and ensure that the caller performing the privileged action has been explicitly granted permissions. The new Java agent is configured in the same manner as the Security Manager, with policy files that grant permissions to individual codebases that specify the privileged actions they are allowed to perform.
Performance improvements for security operations
OpenSearch 2.19 introduced a significant performance optimization in the Security plugin that made privilege evaluation scale better with the number of indexes and roles in a cluster. This release brings further performance improvements around the implementation of field-level security, field masking, and document-level security that result in less data being internally serialized and deserialized with internodal communication, providing significant benefits for clusters using these advanced security features.
New PGP key for 3.x versions
Please note that a new PGP public key (associated with release@opensearch.org) is available for artifact verification in OpenSearch version 3.0 and later. OpenSearch’s current PGP public key (associated with opensearch@amazon.com) will be reserved for 2.x releases only. Please visit https://opensearch.org/verify-signatures.html to download the new public key, which is scheduled to expire on March 6, 2027.
Get started with OpenSearch 3.0
Ready to explore what you can do with OpenSearch 3.0? Head to our downloads page for your preferred distribution, or check out the latest visualization tools on OpenSearch Playground. For more information, see the release notes, documentation release notes, and updated documentation. As always, we welcome your feedback on our community forum, and we look forward to hearing about your experiences with OpenSearch 3.0!
