execute-test
Whether you’re using the included OpenSearch Benchmark workloads or a custom workload, use the execute-test
command to gather data about the performance of your OpenSearch cluster according to the selected workload.
Usage
The following example executes a test using the geonames
workload in test mode:
opensearch-benchmark execute-test --workload=geonames --test-mode
After the test runs, OpenSearch Benchmark responds with a summary of the benchmark metrics:
------------------------------------------------------
_______ __ _____
/ ____(_)___ ____ _/ / / ___/_________ ________
/ /_ / / __ \/ __ `/ / \__ \/ ___/ __ \/ ___/ _ \
/ __/ / / / / / /_/ / / ___/ / /__/ /_/ / / / __/
/_/ /_/_/ /_/\__,_/_/ /____/\___/\____/_/ \___/
------------------------------------------------------
| Metric | Task | Value | Unit |
|-------------------------------:|---------------------:|----------:|-------:|
| Total indexing time | | 28.0997 | min |
| Total merge time | | 6.84378 | min |
| Total refresh time | | 3.06045 | min |
| Total flush time | | 0.106517 | min |
| Total merge throttle time | | 1.28193 | min |
| Median CPU usage | | 471.6 | % |
| Total Young Gen GC | | 16.237 | s |
| Total Old Gen GC | | 1.796 | s |
| Index size | | 2.60124 | GB |
| Total written | | 11.8144 | GB |
| Heap used for segments | | 14.7326 | MB |
| Heap used for doc values | | 0.115917 | MB |
| Heap used for terms | | 13.3203 | MB |
| Heap used for norms | | 0.0734253 | MB |
| Heap used for points | | 0.5793 | MB |
| Heap used for stored fields | | 0.643608 | MB |
| Segment count | | 97 | |
| Min Throughput | index-append | 31925.2 | docs/s |
| Median Throughput | index-append | 39137.5 | docs/s |
| Max Throughput | index-append | 39633.6 | docs/s |
| 50.0th percentile latency | index-append | 872.513 | ms |
| 90.0th percentile latency | index-append | 1457.13 | ms |
| 99.0th percentile latency | index-append | 1874.89 | ms |
| 100th percentile latency | index-append | 2711.71 | ms |
| 50.0th percentile service time | index-append | 872.513 | ms |
| 90.0th percentile service time | index-append | 1457.13 | ms |
| 99.0th percentile service time | index-append | 1874.89 | ms |
| 100th percentile service time | index-append | 2711.71 | ms |
| ... | ... | ... | ... |
| ... | ... | ... | ... |
| Min Throughput | painless_dynamic | 2.53292 | ops/s |
| Median Throughput | painless_dynamic | 2.53813 | ops/s |
| Max Throughput | painless_dynamic | 2.54401 | ops/s |
| 50.0th percentile latency | painless_dynamic | 172208 | ms |
| 90.0th percentile latency | painless_dynamic | 310401 | ms |
| 99.0th percentile latency | painless_dynamic | 341341 | ms |
| 99.9th percentile latency | painless_dynamic | 344404 | ms |
| 100th percentile latency | painless_dynamic | 344754 | ms |
| 50.0th percentile service time | painless_dynamic | 393.02 | ms |
| 90.0th percentile service time | painless_dynamic | 407.579 | ms |
| 99.0th percentile service time | painless_dynamic | 430.806 | ms |
| 99.9th percentile service time | painless_dynamic | 457.352 | ms |
| 100th percentile service time | painless_dynamic | 459.474 | ms |
----------------------------------
[INFO] SUCCESS (took 2634 seconds)
----------------------------------
Options
Use the following options to customize the execute-test
command for your use case. Options in this section are categorized by their use case.
General settings
The following options shape how each test runs and how results appear:
--test-mode
: Runs the given workload in test mode, which is useful when checking a workload for errors.--user-tag
: Defines user-specific key-value pairs to be used in metric record as meta information, for example,intention:baseline-ticket-12345
.--results-format
: Defines the output format for the command line results, eithermarkdown
orcsv
. Default ismarkdown
.--results-number-align
: Defines the column number alignment for when thecompare
command outputs results. Default isright
.--results-file
: When provided a file path, writes the compare results to the file indicated in the path.--show-in-results
: Determines whether or not to include the comparison in the results file.
Distributions
The following options set which version of OpenSearch and the OpenSearch plugins the benchmark test uses:
--distribution-version
: Downloads the specified OpenSearch distribution based on version number. For a list of released OpenSearch versions, see Version history.--distribution-repository
: Defines the repository from where the OpenSearch distribution should be downloaded. Default isrelease
.--revision
: Defines the current source code revision to use for running a benchmark test. Default iscurrent
.current
: Uses the source tree’s current revision based on your OpenSearch distribution.latest
: Fetches the latest revision from the main branch of the source tree.- You can also use a timestamp or commit ID from the source tree. When using a timestamp, specify
@ts
, where “ts” is a valid ISO 8601 timestamp, for example,@2013-07-27T10:37:00Z
.
--opensearch-plugins
: Defines which OpenSearch plugins to install. By default, no plugins are installed.--plugin-params:
Defines a comma-separated list of key:value pairs that are injected verbatim into all plugins as variables.--runtime-jdk
: The major version of JDK to use.--client-options
: Defines a comma-separated list of clients to use. All options are passed to the OpenSearch Python client. Default istimeout:60
.
Cluster
The following option relates to the target cluster of the benchmark.
--target-hosts
: Defines a comma-separated list of host-port pairs that should be targeted if using the pipelinebenchmark-only
. Default islocalhost:9200
.
Distributed workload generation
The following options help those who want to use multiple hosts to generate load to the benchmark cluster:
--load-worker-coordinator-hosts
: Defines a comma-separated list of hosts that coordinate loads. Default islocalhost
.--enable-worker-coordinator-profiling
: Enables an analysis of the performance of OpenSearch Benchmark’s worker coordinator. Default isfalse
.
Provisioning
The following options help customize how OpenSearch Benchmark provisions OpenSearch and workloads:
--provision-config-repository
: Defines the repository from which OpenSearch Benchmark loadsprovision-configs
andprovision-config-instances
.--provision-config-path
: Defines the path to the--provision-config-instance
and any OpenSearch plugin configurations to use.--provision-config-revision
: Defines a specific Git revision in theprovision-config
that OpenSearch Benchmark should use.--provision-config-instance
: Defines the--provision-config-instance
to use. You can see possible configuration instances using the commandopensearch-benchmark list provision-config-instances
.--provision-config-instance-params
: A comma-separated list of key-value pairs injected verbatim as variables for theprovision-config-instance
.
Workload
The following options determine which workload is used to run the test:
--workload-repository
: Defines the repository from which OpenSearch Benchmark loads workloads.--workload-path
: Defines the path to a downloaded or custom workload.--workload-revision
: Defines a specific revision from the workload source tree that OpenSearch Benchmark should use.--workload
: Defines the workload to use based on the workload’s name. You can find a list of preloaded workloads usingopensearch-benchmark list workloads
.
Test procedures
The following options define what test procedures the test uses and which operations are contained inside the procedure:
--test-execution-id
: Defines a unique ID for this test run. Defines the test procedures to use with each workload. You can find a list of test procedures that the workload supports by specifying the workload in theinfo
command, for example,opensearch-benchmark info --workload=<workload_name>
. To look up information on a specific test procedure, use the commandopensearch-benchmark info --workload=<workload_name> --test-procedure=<test-procedure>
.--include-tasks
: Defines a comma-separated list of test procedure tasks to run. By default, all tasks listed in a test procedure array are run.--exclude-tasks
: Defines a comma-separated list of test procedure tasks not to run.--enable-assertions
: Enables assertion checks for tasks. Default isfalse
.
Pipelines
The --pipeline
option selects a pipeline to run. You can find a list of pipelines supported by OpenSearch Benchmark by running opensearch-benchmark list pipelines
.
Telemetry
The following options enable telemetry devices on OpenSearch Benchmark:
--telemetry
: Enables the provided telemetry devices when the devices are provided using a comma-separated list. You can find a list of possible telemetry devices by usingopensearch-benchmark list telemetry
.--telemetry-params
: Defines a comma-separated list of key-value pairs that are injected verbatim into the telemetry devices as parameters.
Errors
The following options set how OpenSearch Benchmark handles errors when running tests:
--on-error
: Controls how OpenSearch Benchmark responds to errors. Default iscontinue
.continue
: Continues to run the test despite the error.abort
: Aborts the test when an error occurs.
--preserve-install
: Keeps the Benchmark candidate and its index. Default isfalse
.--kill-running-processes
: When set totrue
, stops any OpenSearch Benchmark processes currently running and allows OpenSearch Benchmark to continue to run. Default isfalse
.