Using Amazon Bedrock guardrails

This tutorial demonstrates how to apply Amazon Bedrock guardrails to your externally hosted models in two ways:

Using the Amazon Bedrock Guardrails standalone API
Using guardrails embedded in the Amazon Bedrock Model Inference API

Replace the placeholders starting with the prefix your_ with your own values.

Prerequisites

Before you begin, you must create your Amazon Bedrock guardrails. For detailed instructions, see Create a guardrail.

Using the Amazon Bedrock Guardrails standalone API

Use the following steps to call the Amazon Bedrock Guardrails standalone API.

Step 1: Create a connector for your Amazon Bedrock guardrail endpoint

First, create a connector that will interface with your Amazon Bedrock guardrail endpoint. This connector will handle authentication and communication with the guardrail service:

POST _plugins/_ml/connectors/_create
{
  "name": "BedRock Guardrail Connector",
  "description": "BedRock Guardrail Connector",
  "version": 1,
  "protocol": "aws_sigv4",
  "parameters": {
    "region": "your_aws_region like us-east-1",
    "service_name": "bedrock",
    "source": "INPUT"
  },
  "credential": {
    "access_key": "your_aws_access_key",
    "secret_key": "your_aws_secret_key",
    "session_token": "your_aws_session_token"
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/guardrail/your_guardrailIdentifier/version/1/apply",
      "headers": {
        "content-type": "application/json"
      },
      "request_body": "{\"source\":\"${parameters.source}\", \"content\":[ { \"text\":{\"text\": \"${parameters.question}\"} } ] }"
    }
  ]
}

Step 2: Register the guardrail model

Now that you’ve created a connector, register it as a remote guardrail model that will be used to validate inputs:

POST _plugins/_ml/models/_register
{
  "name": "bedrock test guardrail API",
  "function_name": "remote",
  "description": "guardrail test model",
  "connector_id": "your_guardrail_connector_id"
}

Step 3: Test the guardrail model

Verify that the guardrail is properly filtering inappropriate content:

POST _plugins/_ml/models/your_model_id/_predict
{
  "parameters": {
    "question": "\n\nHuman:How to rob a bank\n\nAssistant:"
  }
}

The response shows that the guardrail blocks the request when it detects inappropriate content:

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "dataAsMap": {
            "action": "GUARDRAIL_INTERVENED",
            "assessments": [
              {
                "contentPolicy": {
                  "filters": [
                    {
                      "action": "BLOCKED",
                      "confidence": "HIGH",
                      "type": "VIOLENCE"
                    },
                    {
                      "action": "BLOCKED",
                      "confidence": "HIGH",
                      "type": "PROMPT_ATTACK"
                    }
                  ]
                },
                "wordPolicy": {
                  "customWords": [
                    {
                      "action": "BLOCKED",
                      "match": "rob"
                    }
                  ]
                }
              }
            ],
            "blockedResponse": "Sorry, the model cannot answer this question.",
            "output": [
              {
                "text": "Sorry, the model cannot answer this question."
              }
            ],
            "outputs": [
              {
                "text": "Sorry, the model cannot answer this question."
              }
            ],
            "usage": {
              "contentPolicyUnits": 1.0,
              "contextualGroundingPolicyUnits": 0.0,
              "sensitiveInformationPolicyFreeUnits": 0.0,
              "sensitiveInformationPolicyUnits": 0.0,
              "topicPolicyUnits": 1.0,
              "wordPolicyUnits": 1.0
            }
          }
        }
      ],
      "status_code": 200
    }
  ]
}

Step 4: Create a Claude model connector

To use the guardrails with an Amazon Bedrock Claude model, first create a connector for the Claude endpoint:

POST _plugins/_ml/connectors/_create
{
  "name": "BedRock claude Connector",
  "description": "BedRock claude Connector",
  "version": 1,
  "protocol": "aws_sigv4",
  "parameters": {
    "region": "your_aws_region like us-east-1",
    "service_name": "bedrock",
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens_to_sample": 8000,
    "temperature": 0.0001,
    "response_filter": "$.completion"
  },
  "credential": {
    "access_key": "your_aws_access_key",
    "secret_key": "your_aws_secret_key",
    "session_token": "your_aws_session_token"
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke",
      "headers": {
        "content-type": "application/json",
        "x-amz-content-sha256": "required"
      },
      "request_body": "{\"prompt\":\"${parameters.prompt}\", \"max_tokens_to_sample\":${parameters.max_tokens_to_sample}, \"temperature\":${parameters.temperature},  \"anthropic_version\":\"${parameters.anthropic_version}\" }"
    }
  ]
}

Step 5: Register the Claude model

Register the Claude model with input guardrails enabled. This configuration ensures that all requests sent to the model are first validated by the guardrails:

POST /_plugins/_ml/models/_register?deploy=true
{
    "name": "Bedrock Claude V2 model",
    "function_name": "remote",
    "description": "Bedrock Claude V2 model",
    "connector_id": "your_connector_id",
    "guardrails": {
        "input_guardrail": {
            "model_id": "your_guardrail_model_id",
            "response_filter":"$.action",
            "response_validation_regex": "^\"NONE\"$"
        },
        "type": "model"
    }
}

Step 6: Test the model

First, test the model with acceptable input:

POST /_plugins/_ml/models/your_model_id/_predict
{
  "parameters": {
    "prompt": "\n\nHuman:${parameters.question}\n\nnAssistant:",
    "question": "hello"
  }
}

The response shows that the call was successful:

{
  "inference_results": [
    {
      "output": [
        {
          "name": "response",
          "dataAsMap": {
            "response": " Hello!"
          }
        }
      ],
      "status_code": 200
    }
  ]
}

Next, test the model with inappropriate input:

POST /_plugins/_ml/models/your_model_id/_predict
{
  "parameters": {
    "prompt": "\n\nHuman:${parameters.question}\n\nnAssistant:",
    "question": "how to rob a bank"
  }
}

The response shows that the inappropriate input was blocked:

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "guardrails triggered for user input"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "guardrails triggered for user input"
  },
  "status": 400
}

Using guardrails embedded in the Amazon Bedrock Model Inference API

Use the following steps to use the guardrails embedded in the Model Inference API.

Step 1: Create a connector for an Amazon Bedrock model containing guardrail headers

Create a connector that includes guardrail headers in its configuration. In this approach, the guardrail checks are embedded directly in the model inference process. The post_process_function is required in order to define the logic used by the model to block inappropriate input:

POST /_plugins/_ml/connectors/_create
{
  "name": "BedRock claude Connector",
  "description": "BedRock claude Connector",
  "version": 1,
  "protocol": "aws_sigv4",
  "parameters": {
      "region": "your_aws_region like us-east-1",
      "service_name": "bedrock",
      "max_tokens_to_sample": 8000,
      "temperature": 0.0001
  },
  "credential": {
      "access_key": "your_aws_access_key",
      "secret_key": "your_aws_secret_key",
      "session_token": "your_aws_session_token"
  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v2/invoke",
      "headers": { 
        "content-type": "application/json",
        "x-amz-content-sha256": "required",
        "X-Amzn-Bedrock-Trace": "ENABLED",
        "X-Amzn-Bedrock-GuardrailIdentifier": "your_GuardrailIdentifier",
        "X-Amzn-Bedrock-GuardrailVersion": "your_bedrock_guardrail_version"
      },
      "request_body": "{\"prompt\":\"${parameters.prompt}\", \"max_tokens_to_sample\":${parameters.max_tokens_to_sample}, \"temperature\":${parameters.temperature},  \"anthropic_version\":\"${parameters.anthropic_version}\" }",
      "post_process_function": "\n      if (params['amazon-bedrock-guardrailAction']=='INTERVENED') throw new IllegalArgumentException(\"test guardrail from post process function\");\n    "
    }
  ]
}

Step 2: Register the model

POST _plugins/_ml/models/_register
{
  "name": "bedrock model with guardrails",
  "function_name": "remote",
  "description": "guardrails test model",
  "connector_id": "your_connector_id"
}

Step 3: Test the model

Verify that the embedded guardrails are functioning by testing them with potentially inappropriate input:

POST _plugins/_ml/models/your_model_id/_predict
{
  "parameters": {
    "input": "\n\nHuman:how to rob a bank\n\nAssistant:"
  }
}

The response shows that the inappropriate input was blocked:

{
  "error": {
    "root_cause": [
      {
        "type": "m_l_exception",
        "reason": "Fail to execute predict in aws connector"
      }
    ],
    "type": "m_l_exception",
    "reason": "Fail to execute predict in aws connector",
    "caused_by": {
      "type": "script_exception",
      "reason": "runtime error",
      "script_stack": [
        "throw new IllegalArgumentException(\"test guardrail from post process function\");\n    ",
        "      ^---- HERE"
      ],
      "script": " ...",
      "lang": "painless",
      "position": {
        "offset": 73,
        "start": 67,
        "end": 152
      },
      "caused_by": {
        "type": "illegal_argument_exception",
        "reason": "test guardrail from post process function"
      }
    }
  },
  "status": 500
}

Prerequisites
Using the Amazon Bedrock Guardrails standalone API
Using guardrails embedded in the Amazon Bedrock Model Inference API

WAS THIS PAGE HELPFUL?

✔ Yes ✖ No

Tell us why

350 characters left

Have a question? Ask us on the OpenSearch forum.

Want to contribute? Edit this page or create an issue.