Link Search Menu Expand Document Documentation Menu

Auto-follow for cross-cluster replication

Auto-follow lets you automatically replicate indexes created on the leader cluster based on matching patterns. When you create an index on the leader cluster with a name that matches a specified pattern (for example, index-01*), a corresponding follower index is automatically created on the follower cluster.

You can configure multiple replication rules for a single cluster. The patterns currently only support wildcard matching.

Prerequisites

You need to set up a cross-cluster connection between two clusters before you can enable auto-follow.

Permissions

If the security plugin is enabled, make sure that non-admin users are mapped to the appropriate permissions so they can perform replication actions. For index and cluster-level permissions requirements, see Cross-cluster replication permissions.

Get started with auto-follow

Replication rules are a collection of patterns that you create against a single remote cluster. When you create a replication rule, it automatically starts replicating any new indexes that match the pattern, but does not replicate matching indexes that were previously created.

Create a replication rule on the follower cluster:

curl -XPOST -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "my-connection-alias",
   "name": "my-replication-rule",
   "pattern": "movies*",
   "use_roles":{
      "leader_cluster_role": "all_access",
      "follower_cluster_role": "all_access"
   }
}'

If the security plugin is disabled, you can leave out the use_roles parameter. If it’s enabled, however, you need to specify the leader and follower cluster roles that OpenSearch uses to authenticate requests. This example uses all_access for simplicity, but we recommend creating a replication user on each cluster and mapping it accordingly.

To test the rule, create a matching index on the leader cluster:

curl -XPUT -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://localhost:9201/movies-0001?pretty'

And confirm its replica shows up on the follower cluster:

curl -XGET -u 'admin:admin' -k 'https://localhost:9200/_cat/indices?v'

It might take several seconds for the index to appear.

health status index        uuid                     pri rep docs.count docs.deleted store.size pri.store.size
yellow open   movies-0001  kHOxYYHxRMeszLjTD9rvSQ     1   1          0            0       208b           208b

Retrieve replication rules

To retrieve a list of existing replication rules that are configured on a cluster, send the following request:

curl -XGET -u 'admin:admin' -k 'https://localhost:9200/_plugins/_replication/autofollow_stats'

{
   "num_success_start_replication": 1,
   "num_failed_start_replication": 0,
   "num_failed_leader_calls": 0,
   "failed_indices":[
      
   ],
   "autofollow_stats":[
      {
         "name":"my-replication-rule",
         "pattern":"movies*",
         "num_success_start_replication": 1,
         "num_failed_start_replication": 0,
         "num_failed_leader_calls": 0,
         "failed_indices":[
            
         ]
      }
   ]
}

Delete a replication rule

To delete a replication rule, send the following request:

curl -XDELETE -k -H 'Content-Type: application/json' -u 'admin:admin' 'https://localhost:9200/_plugins/_replication/_autofollow?pretty' -d '
{
   "leader_alias" : "my-conection-alias",
   "name": "my-replication-rule"
}'

OpenSearch stops replicating new indexes that match the pattern, but existing indexes that the rule previously created remain read-only and continue to replicate. If you need to stop existing replication activity and open the indexes up for writes, use the stop replication API operation.