OpenSearch Data Prepper is a server-side data collector capable of filtering, enriching, transforming, normalizing, and aggregating data for downstream analysis and visualization. As OpenSearch’s native ingestion tool, it can handle almost any use case while processing large, complex datasets.
Data Prepper lets you build custom pipelines to improve your operational view of applications across different methodologies, including trace analytics and log analytics. These capabilities help you visualize event flows, identify performance problems, enhance your search capabilities, conduct comprehensive analysis, and gain insights into your applications’ performance and behavior.
From pipeline to comprehensive observability
Customizable pipelines
Customize pluggable components to fit your needs and even connect them to your own implementations.
Unified OTLP source
Streamline telemetry data ingestion using a single, consolidated configuration with OpenTelemetry protocol (OTLP).
Support for trace and log analytics
Visualize event flows and identify performance problems while enhancing search capabilities, conducting analysis, and gaining insights.
Enhanced interoperability
Enhance interoperability for your observability pipeline by seamlessly exporting processed trace data to AWS X-Ray.
Durability and flexibility for any scenario
Support for extensive use cases
Use OpenSearch Data Prepper for key scenarios, including trace analytics, log analytics, Amazon S3 log analytics, and metrics ingestion.
Flexible configuration
Options for both minimal and comprehensive configurations give you the flexibility you need to tailor your pipelines to your unique needs.
Durable and reliable data delivery
Data Prepper helps ensure reliable and durable data delivery from sources to sinks through end-to-end (E2E) acknowledgments.
Simplified management
Administrator functions include system configuration, interacting with core APIs, Log4j configuration, and monitoring.
Prepare your pipeline for enhanced observability
Follow these steps to feed your OpenSearch observability ecosystem with a robust and secure data pipeline.
| Get started | Install OpenSearch Data Prepper by running the Docker image or building from source. |
| Create your pipelines | Establish your pipeline for ingesting data into OpenSearch from source to sink. |
| Configure your collector | Customize your configuration in your Data Prepper installation by choosing from a wide array of options. |
| Migrate from Logstash | Run OpenSearch Data Prepper with a Logstash configuration. |
| Migrate from Open Distro | Migrate from the Open Distro Data Prepper to OpenSearch Data Prepper |
Get started with OpenSearch
Unlock actionable insights today. Explore our documentation for OpenSearch Observability.
