You're viewing version 2.16 of the OpenSearch documentation. This version is no longer maintained. For the latest version, see the current documentation. For information about OpenSearch version maintenance, see Release Schedule and Maintenance Policy.
Nodes hot threads
Introduced 1.0
The nodes hot threads endpoint provides information about busy JVM threads for selected cluster nodes. It provides a unique view of the of activity each node.
Example
GET /_nodes/hot_threads
Path and HTTP methods
GET /_nodes/hot_threads
GET /_nodes/<nodeId>/hot_threads
Path parameters
You can include the following optional path parameter in your request.
| Parameter | Type | Description |
|---|---|---|
| nodeId | String | A comma-separated list of node IDs used to filter results. Supports node filters. Defaults to _all. |
Query parameters
You can include the following query parameters in your request. All query parameters are optional.
| Parameter | Type | Description |
|---|---|---|
| snapshots | Integer | The number of samples of thread stacktraces. Defaults to 10. |
| interval | Time | The interval between consecutive samples. Defaults to 500ms. |
| threads | Integer | The number of the busiest threads to return information about. Defaults to 3. |
| ignore_idle_threads | Boolean | Don’t show threads that are in known idle states, such as waiting on a socket select or pulling from an empty task queue. Defaults to true. |
| type | String | Supported thread types are cpu, wait, or block. Defaults to cpu. |
| timeout | Time | Sets the time limit for node response. Default value is 30s. |
Example request
GET /_nodes/hot_threads
Example response
::: {opensearch}{F-ByTQzVQ3GQeYzQJArJGQ}{GxbcLdCATPWggOuQHJAoCw}{127.0.0.1}{127.0.0.1:9300}{dimr}{shard_indexing_pressure_enabled=true}
Hot threads at 2022-09-29T19:46:44.533Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
0.1% (455.5micros out of 500ms) cpu usage by thread 'ScheduledMetricCollectorsExecutor'
10/10 snapshots sharing following 2 elements
java.base@17.0.4/java.lang.Thread.sleep(Native Method)
org.opensearch.performanceanalyzer.collectors.ScheduledMetricCollectorsExecutor.run(ScheduledMetricCollectorsExecutor.java:100)
Example response
Unlike the majority of OpenSearch API responses, this response is in a text format.
It consists of one section per each cluster node included in the response.
Each section starts with a single line containing the following segments:
| Line segment | Description |
|---|---|
::: | Line start (a distinct visual symbol). |
{global-eu-35} | Node name. |
{uFPbKLDOTlOmdnwUlKW8sw} | NodeId. |
{OAM8OT5CQAyasWuIDeVyUA} | EphemeralId. |
{global-eu-35.local} | Host name. |
{[gdv2:a284:2acv:5fa6:0:3a2:7260:74cf]:9300} | Host address. |
{dimr} | Node roles (d=data, i=ingest, m=cluster manager, r=remote cluster client). |
{zone=west-a2, shard_indexing_pressure_enabled=true} | Node attributes. |
Then information about threads of the selected type is provided.
::: {global-eu-35}{uFPbKLDOTlOmdnwUlKW8sw}{OAM8OT5CQAyasWuIDeVyUA}{global-eu-35.local}{[gdv2:a284:2acv:5fa6:0:3a2:7260:74cf]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}
Hot threads at 2022-04-01T15:15:27.658Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
0.1% (645micros out of 500ms) cpu usage by thread 'opensearch[global-eu-35][transport_worker][T#7]'
4/10 snapshots sharing following 3 elements
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
java.base@11.0.14.1/java.lang.Thread.run(Thread.java:829)
::: {global-eu-62}{4knOxAdERlOB19zLQIT1bQ}{HJuZs2HiQ_-8Elj0Fvi_1g}{global-eu-62.local}{[gdv2:a284:2acv:5fa6:0:3a2:bba6:fe3f]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}
Hot threads at 2022-04-01T15:15:27.659Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
18.7% (93.4ms out of 500ms) cpu usage by thread 'opensearch[global-eu-62][transport_worker][T#3]'
6/10 snapshots sharing following 3 elements
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
java.base@11.0.14.1/java.lang.Thread.run(Thread.java:829)
::: {global-eu-44}{8WW3hrkcTwGvgah_L8D_jw}{Sok7spHISFyol0jFV6i0kw}{global-eu-44.local}{[gdv2:a284:2acv:5fa6:0:3a2:9120:e79e]:9300}{dimr}{zone=west-a2, shard_indexing_pressure_enabled=true}
Hot threads at 2022-04-01T15:15:27.659Z, interval=500ms, busiestThreads=3, ignoreIdleThreads=true:
42.6% (212.7ms out of 500ms) cpu usage by thread 'opensearch[global-eu-44][write][T#5]'
2/10 snapshots sharing following 43 elements
java.base@11.0.14.1/sun.nio.ch.IOUtil.write1(Native Method)
java.base@11.0.14.1/sun.nio.ch.EPollSelectorImpl.wakeup(EPollSelectorImpl.java:254)
io.netty.channel.nio.NioEventLoop.wakeup(NioEventLoop.java:787)
io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:846)
io.netty.util.concurrent.SingleThreadEventExecutor.execute(SingleThreadEventExecutor.java:815)
io.netty.channel.AbstractChannelHandlerContext.safeExecute(AbstractChannelHandlerContext.java:989)
io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:796)
io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:758)
io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1020)
io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:311)
org.opensearch.transport.netty4.Netty4TcpChannel.sendMessage(Netty4TcpChannel.java:159)
app//org.opensearch.transport.OutboundHan...
Required permissions
If you use the Security plugin, make sure you set the following permissions: cluster:monitor/nodes/hot_threads.