Follow this guide to learn how to automate CloudWatch alerts, Kubernetes remediation, and incident reporting using multi-agent AI workflows with the AWS Strands Agents SDK. The SRE Incident Response Agent is a multi-agent sample that ships with the AWS Strands Agents SDK. It automatically discovers active CloudWatch alarms, performs AI-powered root cause analysis using Claude Sonnet 4 on Amazon Bedrock, proposes Kubernetes or Helm remediations, and posts a structured incident report to Slack. AWS Strands Agents SDK Kubernetes This guide covers everything you need to clone the repo and run it yourself. Prerequisites Before you begin, make sure the following are in place: Python 3.11+ installed on your machine
AWS credentials configured (aws configure or an active IAM role)
Amazon Bedrock access enabled for Claude Sonnet 4 in your target region
kubectl and helm v3 installed — only required if you plan to run live remediations. Dry-run mode works without them. Python 3.11+ installed on your machine Python 3.11+ AWS credentials configured (aws configure or an active IAM role) AWS credentials Amazon Bedrock access enabled for Claude Sonnet 4 in your target region Amazon Bedrock access kubectl and helm v3 installed — only required if you plan to run live remediations. Dry-run mode works without them. kubectl helm v3 Step 1: Clone the Repository The sample lives inside the strands-agents/samples open source repository. Clone it and navigate to the SRE agent directory: git clone https://github.com/strands-agents/samples.git
cd samples/02-samples/sre-incident-response-agent git clone https://github.com/strands-agents/samples.git
cd samples/02-samples/sre-incident-response-agent The directory contains the following files: sre-incident-response-agent/

├── sre_agent.py           # Main agent: 4 agents + 8 tools

├── test_sre_agent.py      # Pytest unit tests (12 tests, mocked AWS)

├── requirements.txt

├── .env.example

└── README.md sre-incident-response-agent/

├── sre_agent.py           # Main agent: 4 agents + 8 tools

├── test_sre_agent.py      # Pytest unit tests (12 tests, mocked AWS)

├── requirements.txt

├── .env.example

└── README.md Step 2: Create a Virtual Environment and Install Dependencies python -m venv .venv
source .venv/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt python -m venv .venv
source .venv/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt The requirements.txt pins the core dependencies: strands-agents>=0.1.0
strands-agents-tools>=0.1.0
boto3>=1.38.0
botocore>=1.38.0 strands-agents>=0.1.0
strands-agents-tools>=0.1.0
boto3>=1.38.0
botocore>=1.38.0 Step 3: Configure Environment Variables Copy .env.example to .env and fill in your values: cp .env.example .env cp .env.example .env Open .env and set the following: # AWS region where your CloudWatch alarms live
AWS_REGION=us-east-1

# Amazon Bedrock model ID (Claude Sonnet 4 is the default)
BEDROCK_MODEL_ID=us.anthropic.claude-sonnet-4-20250514-v1:0

# DRY_RUN=true means kubectl/helm commands are printed, not executed.
# Set to false only when you are ready for live remediations.
DRY_RUN=true
# Optional: post the incident report to Slack.
# Leave blank to print to stdout instead.
SLACK_WEBHOOK_URL= # AWS region where your CloudWatch alarms live
AWS_REGION=us-east-1

# Amazon Bedrock model ID (Claude Sonnet 4 is the default)
BEDROCK_MODEL_ID=us.anthropic.claude-sonnet-4-20250514-v1:0

# DRY_RUN=true means kubectl/helm commands are printed, not executed.
# Set to false only when you are ready for live remediations.
DRY_RUN=true
# Optional: post the incident report to Slack.
# Leave blank to print to stdout instead.
SLACK_WEBHOOK_URL= Step 4: Grant IAM Permissions The agent needs read-only access to CloudWatch alarms, metric statistics, and log events. No write permissions to CloudWatch are required. Attach the following policy to the IAM role or user running the agent: CloudWatch {
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "cloudwatch:DescribeAlarms",
      "cloudwatch:GetMetricStatistics",
      "logs:FilterLogEvents",
      "logs:DescribeLogGroups"
    ],
    "Resource": "*"
  }]
} {
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "cloudwatch:DescribeAlarms",
      "cloudwatch:GetMetricStatistics",
      "logs:FilterLogEvents",
      "logs:DescribeLogGroups"
    ],
    "Resource": "*"
  }]
} Step 5: Run the Agent There are two ways to trigger the agent. Option A: Automatic Alarm Discovery Let the agent discover all active CloudWatch alarms on its own. This is the recommended mode for a real on-call scenario: python sre_agent.py python sre_agent.py Option B: Targeted Investigation Pass a natural-language description of the triggering event. The agent will focus its investigation on the service and symptom you describe: python sre_agent.py "High CPU alarm fired on ECS service my-api in prod namespace" python sre_agent.py "High CPU alarm fired on ECS service my-api in prod namespace" Example Output Running the targeted trigger above produces output similar to the following: Starting SRE Incident Response
   Trigger: High CPU alarm fired on ECS service my-api in prod namespace

[cloudwatch_agent] Fetching active alarms...
  Found alarm: my-api-HighCPU (CPUUtilization > 85% for 5m)
  Metric stats: avg 91.3%, max 97.8% over last 30 min
  Log events: 14 OOMKilled events in /ecs/my-api

[rca_agent] Performing root cause analysis...
  Root cause: Memory leak causing CPU spike as GC thrashes
  Severity: P2 - single service, <5% of users affected
  Recommended fix: Rolling restart to clear heap; monitor for recurrence

[remediation_agent] Applying remediation...
  [DRY-RUN] kubectl rollout restart deployment/my-api -n prod

================================================================
*[P2] SRE Incident Report - 2025-10-14 09:31 UTC*

What happened: CloudWatch alarm my-api-HighCPU fired at 09:18 UTC.
CPU reached 97.8% (threshold 85%). 14 OOMKilled events in 15 min.

Root cause: Memory leak in application heap leading to aggressive GC,
causing CPU saturation. Likely introduced in the last deployment.

Remediation: Rolling restart of deployment/my-api in namespace prod
initiated (dry-run). All pods will be replaced with fresh instances.

Follow-up:
  - Monitor CPUUtilization for next 30 min
  - Review recent commits for memory allocation changes
  - Consider setting memory limits in the Helm chart
================================================================ Starting SRE Incident Response
   Trigger: High CPU alarm fired on ECS service my-api in prod namespace

[cloudwatch_agent] Fetching active alarms...
  Found alarm: my-api-HighCPU (CPUUtilization > 85% for 5m)
  Metric stats: avg 91.3%, max 97.8% over last 30 min
  Log events: 14 OOMKilled events in /ecs/my-api

[rca_agent] Performing root cause analysis...
  Root cause: Memory leak causing CPU spike as GC thrashes
  Severity: P2 - single service, <5% of users affected
  Recommended fix: Rolling restart to clear heap; monitor for recurrence

[remediation_agent] Applying remediation...
  [DRY-RUN] kubectl rollout restart deployment/my-api -n prod

================================================================
*[P2] SRE Incident Report - 2025-10-14 09:31 UTC*

What happened: CloudWatch alarm my-api-HighCPU fired at 09:18 UTC.
CPU reached 97.8% (threshold 85%). 14 OOMKilled events in 15 min.

Root cause: Memory leak in application heap leading to aggressive GC,
causing CPU saturation. Likely introduced in the last deployment.

Remediation: Rolling restart of deployment/my-api in namespace prod
initiated (dry-run). All pods will be replaced with fresh instances.

Follow-up:
  - Monitor CPUUtilization for next 30 min
  - Review recent commits for memory allocation changes
  - Consider setting memory limits in the Helm chart
================================================================ Running the Tests (No AWS Credentials Required) The sample ships with 12 pytest unit tests that mock boto3 entirely. You can run the full test suite in any environment, including CI, without any AWS credentials: pip install pytest pytest-mock
pytest test_sre_agent.py -v

# Expected: 12 passed pip install pytest pytest-mock
pytest test_sre_agent.py -v

# Expected: 12 passed Enabling Live Remediation Once you have validated the agent’s behaviour in dry-run mode and are satisfied with the decisions it makes, you can enable live kubectl and helm execution by setting DRY_RUN=false in your .env file: DRY_RUN=false DRY_RUN=false Conclusion In under five minutes of setup, the AWS Strands Agents SDK gives you a working multi-agent incident response loop: alarm discovery, AI-powered root cause analysis, Kubernetes remediation, and a structured incident report, all driven by a single python sre_agent.py command. The dry-run default means there is no risk in running it against a real environment while you evaluate its reasoning. From here, the natural next steps are connecting a Slack webhook for team notifications, adding a PagerDuty tool for incident tracking, or extending the RCA agent with a vector store of past postmortems. All of that is a tool definition away. I hope you found this article helpful and that it will inspire you to explore AWS Strands Agents SDK and AI agents more deeply.

This story contains new, firsthand information uncovered by the writer.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

This story contains AI-generated text. The author has used AI either for research, to generate outlines, or write the text itself. 

PagerDuty

Building an Autonomous SRE Incident Response System Using AWS Strands Agents SDK

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

PromptDesk: Simplifying Prompt Management in a Rapidly Evolving AI Landscape

'Multimodal is the most unappreciated AI breakthrough' says DoNotPay CEO Joshua Browder

From Chat-Bots to Killer-Bots?

How to Identify Your Breakthrough AI Startup Idea

The Metrics Resurrections: Action! Action! Action!

What Are Multi-Agent Systems? And Where Did They Come From?

PromptDesk: Simplifying Prompt Management in a Rapidly Evolving AI Landscape

'Multimodal is the most unappreciated AI breakthrough' says DoNotPay CEO Joshua Browder

From Chat-Bots to Killer-Bots?

How to Identify Your Breakthrough AI Startup Idea

The Metrics Resurrections: Action! Action! Action!

What Are Multi-Agent Systems? And Where Did They Come From?

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps