OpenSearch Project Blog

Data Prepper 2.13 adds Prometheus sink support, native OpenSearch data streams, cross-region S3 ingestion, and 20% performance improvements. Download it now.

Learn how OpenSearch 3.3's agentic memory system enables AI agents to maintain context and learn from past interactions using persistent memory and intelligent processing capabilities.

Learn how asymmetric distance computation (ADC) in OpenSearch achieves 32-fold memory compression for vector search while maintaining better recall than traditional binary quantization methods.

OpenSearch 3.3, PPL, Piped Processing Language, log analytics, observability, Apache Calcite, data analysis, time-series analysis, log processing, data visualization, OpenSearch 3.3, database querying, data monitoring, security analytics

With OpenSearch 3.3, we are excited to introduce _agentic search_, a major evolution in the way you interact with your data. Agentic search lets you explore your data using a natural language interface, without needing to construct query domain-specific language (DSL) queries manually.

Segment replication is a powerful index replication strategy for high QPS workloads that allows physical isolation and decoupling of indexing and search workloads, rapid replica failovers, and seamless point-in-time restores. However, this efficiency comes with its own unique challenges. During high indexing bursts, or if replication is delayed due to network issues, the system can accumulate large replication checkpoints. Searchers (replica shards) struggle to absorb these checkpoints, experiencing high page faults, thrashing for in-flight queries, and degraded search performance. This blog post describes how to address these issues and build a resilient segment-replicated system using Lucene's powerful new hooks for "Adaptive Refresh".

Learn how to integrate Amazon Q CLI with OpenSearch's log pattern analysis and data distribution tools for streamlined command-line diagnostics and enhanced system troubleshooting capabilities.

In today’s fast-paced digital world, waiting isn’t an option, especially when it comes to AI-powered applications. Streaming technology has emerged as a critical solution to this challenge, fundamentally changing how...

Learn how to improve search result diversity using OpenSearch 3.3's native Maximal Marginal Relevance (MMR) for k-NN and neural queries. Includes implementation guide and performance benchmarks.

Discover how to build reliable RAG pipelines by combining OpenSearch and Docling, an open-source tool that transforms complex documents into structured, metadata-rich data suitable for AI workflows.

Learn how OpenSearch automatically generates and executes search processors during query evaluation, how plugin developers can extend this mechanism, and how to monitor system-generated processors using the Search Pipeline Stats API.

Explore the diverse range of AI agents in OpenSearch, from simple flow agents to advanced ReAct multi-agent systems. Learn how to choose the right agent for tasks like RAG, chatbots, research, and automated root cause analysis.

OpenSearch 3.3 introduces batch processing for remote semantic highlighting models, reducing ML inference calls from N to 1 per search query. Our benchmarks demonstrate 2-14× performance improvements depending on document length and result set size.

Neural sparse search combines the semantic understanding of neural models with the efficiency of sparse vector representations. This technique has proven effective for semantic retrieval while maintaining the advantages of...

Storage is a key factor driving the infrastructure cost of your OpenSearch cluster. As your data grows, storage requirements can increase multifold, depending on whether OpenSearch stores documents in multiple...

As we mark the first anniversary of the OpenSearch Software Foundation’s Technical Steering Committee (TSC), it’s important to celebrate the milestones we’ve achieved and acknowledge the dedicated individuals who helped...

What does it feel like to attend OpenSearchCon for the very first time?  When I walked in, I was nervous. I’ve been working in infrastructure and observability, but this was...

In an era of explosive data growth, organizations are facing a critical challenge: how to manage, analyze, and derive value from their logs without breaking the bank. Traditional proprietary solutions like Splunk have become increasingly expensive, leaving teams searching for a more sustainable approach to observability.

Search functionality is the cornerstone of any successful e-commerce platform. It’s the point of connection between customers and your product catalog. Your store’s search features directly influence the user experience,...