Link Search Menu Expand Document Documentation Menu
Documentation

Metric analytics

Introduced 2.4

With the release of OpenSearch 2.4, you can now ingest and visualize metric data stored directly in OpenSearch using the Metrics tool. This equips you with tools to analyze and correlate data across logs, traces, and metrics.

Before the introduction of this feature, you could only ingest and visualize logs and traces from your monitored environments. With the Metrics tool, you can now observe your digital assets with more granularity, gain deeper insight into the health of your infrastructure, and better inform your root cause analysis.

The Metrics tool offers federated visualization capabilities in addition to the following:

The following image displays the flow for retrieving metrics from Prometheus and displaying them on a visualization dashboard.

Prometheus data source

The following image displays an observability dashboard that visualizes metric data from the OpenSearch index using OTel queries.

OTel data source

Configuring Prometheus to send metric data to OpenSearch

You must first create a connection from Prometheus to OpenSearch using the SQL plugin. You can then configure a connection to Prometheus by using the _datasources API endpoint.

The following example shows a request that configures a Prometheus data source without any authentication:

POST _plugins/_query/_datasources 
{
    "name" : "my_prometheus",
    "connector": "prometheus",
    "properties" : {
        "prometheus.uri" : "http://localhost:9090"
    }
}

The following example shows how to configure a Prometheus data source using AWS Signature Version 4 authentication:

POST _plugins/_query/_datasources
{
    "name" : "my_prometheus",
    "connector": "prometheus",
    "properties" : {
        "prometheus.uri" : "http://localhost:8080",
        "prometheus.auth.type" : "awssigv4",
        "prometheus.auth.region" : "us-east-1",
        "prometheus.auth.access_key" : ""
        "prometheus.auth.secret_key" : ""
    }
}

After configuring the connection, you can view Prometheus metrics in OpenSearch Dashboards by going to the Observability > Metrics page, as shown in the following image.

Prometheus metrics displayed on a dashboard

Developer resources

See the following developer resources for sample code, articles, tutorials, and API references:

Experimenting with OpenTelemetry Metrics in the OpenSearch demo environment

The OpenSearch opentelemetry-demo repository provides a practical demonstration of collecting, processing, and visualizing metric data through OpenTelemetry Metrics from OpenTelemetry and using the Metrics tool in OpenSearch Dashboards.

Visualizing OTel metrics in OpenSearch

To visualize OTel metric data in OpenSearch, follow these steps:

  1. Install the opentelemetry-demo repository. See the Getting Started guide for instructions.
  2. Collect the OTel signals, including metric signals. See the OTel Collector guide for instructions.
  3. Configure the OTel pipeline to emit metric signals. See the OTel Collector Pipeline guide for instructions.

Example YAML config file

    service:
      extensions: [basicauth/client]
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch]
          exporters: [otlp, debug, spanmetrics, otlp/traces, opensearch/traces]
        metrics:
          receivers: [otlp, spanmetrics]
          processors: [filter/ottl, transform, batch]
          exporters: [otlphttp/prometheus, otlp/metrics, debug]
        logs:
          receivers: [otlp]
          processors: [batch]
          exporters: [otlp/logs,  opensearch/logs, debug]

  1. Configure the Data Prepper pipeline to emit the collected metric signals into the OpenSearch metrics index.

Example YAML config file

    otel-metrics-pipeline:
      workers: 8
      delay: 3000
      source:
        otel_metrics_source:
          health_check_service: true
          ssl: false
      buffer:
        bounded_blocking:
          buffer_size: 1024 # max number of records the buffer accepts
          batch_size: 1024 # max number of records the buffer drains after each read
      processor:
        - otel_metrics:
            calculate_histogram_buckets: true
            calculate_exponential_histogram_buckets: true
            exponential_histogram_max_allowed_scale: 10
            flatten_attributes: false
      sink:
        - opensearch:
            hosts: ["https://opensearch-node1:9200"]
            username: "admin"
            password: "my_%New%_passW0rd!@#"
            insecure: true
            index_type: custom
            template_file: "templates/ss4o_metrics.json"
            index: ss4o_metrics-otel-%{yyyy.MM.dd}
            bulk_size: 4

  1. Ingest metric data into OpenSearch. As the demo starts generating data, the metric signals will be added to the OpenSearch index that supports the OpenTelemetry Metrics schema format.
  2. On the Metrics page, choose Otel-Index from the Data sources dropdown menu and Simple Schema for Observability Index from the OTel index dropdown menu. A visualization is displayed, as shown in the following image.

OTel metrics dashboard

Visualizing metrics in remote clusters

Introduced 2.14

You can view metrics from remote OpenSearch clusters by using the Metrics tool. Select the database icon on the upper-right toolbar and choose a cluster from the DATA SOURCES dropdown menu, as shown in the following image. You can switch from a local cluster to a remote cluster.

Switching clusters using the Metrics analytics tool

You can also view metric visualizations from other sources alongside local metric visualizations. From the DATA SOURCES dropdown menu, choose the remote metric visualization to add it to the group of visualizations already shown on the dashboard. An example dashboard is shown in the following image.

Metrics dashboard

To learn about multi-cluster support for data sources, see Enable OpenSearch Dashboards to support multiple OpenSearch clusters.

Creating visualizations based on custom metrics

You can create visualizations using the metric data collected by your OpenSearch cluster, including Prometheus metrics and custom metrics.

To create these visualizations, follow these steps:

  1. From the OpenSearch Dashboards main menu, navigate to Observability > Metrics > Available Metrics.
  2. Choose the metrics to add to your visualization and then select Save.
  3. When prompted for a Custom operational dashboards/application, choose one of the listed options. You can edit the predefined name values in the Metric Name field.
  4. Select Save to save your visualization. An example visualization is shown in the following image.

Metrics analytics dashboard with visualizations

Defining PPL queries for Prometheus metrics

You can define Piped Processing Language (PPL) queries to interact with metrics collected by Prometheus. The following is an example PPL query for a Prometheus metric:

source = my_prometheus.prometheus_http_requests_total | stats avg(@value) by span(@timestamp,15s), handler, code

Creating a custom visualization based on the PPL query

To create a custom visualization based on the PPL query, follow these steps:

  1. From the Logs page, select > Event Explorer.
  2. On the Explorer page, enter your PPL query and select Run. Then select Save.
  3. When prompted to choose a Custom Operational Dashboards/Application, select one of the listed options. Optionally, you can edit the predefined name values in the Metric Name fields and can choose to save the visualization as a metric.
  4. Select Save to save your custom visualization.

Only queries that include a time-series visualization and statistics or span information can be saved as a metric, as shown in the following image.

Saving queries as metrics

350 characters left

Have a question? .

Want to contribute? or .