GuideIntermediate
Sre And Incident Response
Operates, with the discipline of SRE, everything the six prior guides in the AWS Cloud ecosystem already built on top of Andes Cargo: a system can pass the security gate and the cost gate and still fail in production, because "reliable" isn't a state — it's a measurement. This guide teaches the real math of SLI/SLO/error budget with a Python calculator that actually runs, first on fixed data and then on real metrics from Andes Cargo's Lambda read through CloudWatch, Prometheus, Grafana, and OpenTelemetry traces in Jaeger — observability treated as the input to that math, not a full instrumentation discipline. It builds alerting based on error-budget burn rate (the multi-window pattern from the real Google SRE book) instead of loose thresholds, and assembles the full incident lifecycle with roles, severity levels, and an on-call rotation designed with honesty about its human cost. At the center of the guide, it operates end to end the real Claude Code `destroy` incident that the three prior guides already used as a warning: here its timeline gets reconstructed, its severity classified, it gets declared and mitigated with the framework just built, and it closes with a blameless postmortem following the real Google SRE template and an operational runbook verified against LocalStack. The capstone runs a new, deterministic synthetic incident through the whole machine.
- 64
- lessons
- 8
- modules
- English · Spanish
- available in
- Yes
- certificate
- Free
- access
Outcomes
What you'll be able to do
- Explain what SRE is and why reliability is a measurement, not a state — the marginal cost of each additional nine of availability
- Calculate a real SLI (good events ÷ valid events), choose an SLO that means something, and get the error budget in minutes with a runnable Python calculator
- Instrument metrics, logs, and traces as direct SLI input — CloudWatch, Prometheus, Grafana, and traces with OpenTelemetry and Jaeger, running for real against Andes Cargo's inherited Lambda
- Build alerting based on error-budget burn rate (multi-window, multi-rate), with a real Alertmanager rule and a real CloudWatch alarm
- Design the full incident lifecycle: severities, roles (incident commander, communications, operations), and a deterministic on-call rotation, with honesty about its real human cost
- Operate the Claude Code `destroy` incident end to end: reconstruct its timeline, classify its severity, declare it, and trace the mitigation decision tree that led to real-world recovery
- Write a blameless postmortem following the real Google SRE template, with action items that actually get done, and build Andes Cargo's first operational runbook, verified step by step against LocalStack
- Run, as a capstone, a deterministic synthetic incident through the full machine: SLI → real data → burn-rate alert → declared incident → runbook → postmortem
Before you start
What you need to bring
It's for you if...
- Backend, DevOps, or Cloud engineers who already completed the six prior AWS Cloud ecosystem guides (Core Services, Serverless and Containers, Terraform and IaC, CI/CD and GitOps, Cloud Security, FinOps) and want to operate the reliability of what they built
- Teams with a green, secure, budgeted pipeline that never defined what "reliable" means in numbers
- Engineers preparing for on-call roles who need the real vocabulary of SLI/SLO/error budget, not just the dictionary definition
- Devs looking for a real, complete example of a blameless postmortem and an operational runbook — something almost no market curriculum covers
Requirements and materials
- The six prior AWS Cloud ecosystem guides completed (or equivalent): `andes-cargo-infra/` applied, with the GitHub Actions pipeline, the security gate, and the cost gate already working
- Docker and `docker compose` installed and working, to run Prometheus, Grafana, Alertmanager, and Jaeger
- Basic Python (the error budget calculator and the burn rate evaluator are short scripts)
- LocalStack (free Hobby plan) for CloudWatch, DynamoDB, and Lambda
Content
The syllabus, module by module
Open any of them to see its lessons.
- 1. Guide introduction: from "it works, it's secure, it fits the budget" to "it's reliable"
- 2. What SRE is (and isn't)
- 3. Reliability as a feature, not an accident
- 4. Hands-on: reading Andes Cargo like an SRE would
- 5. Two real incidents, two different questions
- 6. Hands-on: the error budget the Claude Code incident burned
- 7. The vocabulary you'll use all guide long
- 8. Project: Andes Cargo's reliability charter
- 1. Introduction: from vocabulary to math
- 2. Choosing a good SLI: the four golden signals
- 3. Hands-on: your first SLI definition
- 4. Hands-on: the error budget calculator
- 5. Choosing an SLO that means something
- 6. Hands-on: burn rate — the consumption rate, not just the balance
- 7. Hands-on: applying the calculator to a real Andes Cargo scenario
- 8. Project: Andes Cargo's `SLO.md`
- 1. Introduction: observability in service of a question, not a dashboard
- 2. The three pillars, with an SRE lens
- 3. Hands-on: real metrics from the inherited Lambda
- 4. Hands-on: real logs with `jq`
- 5. Hands-on: traces with OpenTelemetry and Jaeger
- 6. Hands-on: Prometheus and Grafana, the stack the market asks for
- 7. From raw telemetry to a measured SLI
- 8. Project: Andes Cargo's observability-to-SLI pipeline
- 1. Introduction: why a static threshold isn't enough
- 2. Multi-window, multi-burn-rate: the real Google SRE pattern
- 3. Hands-on: the burn rate evaluator
- 4. Hands-on: the real rule in Alertmanager
- 5. Hands-on: a real CloudWatch alarm on the Lambda
- 6. What AWS already automates: CloudWatch Application Signals, named
- 7. Hands-on: routing the alert
- 8. Project: Andes Cargo's alerting policy
- 1. Introduction: the framework before the case
- 2. The lifecycle of an incident
- 3. Severity levels, with real Andes Cargo examples
- 4. Roles during an incident
- 5. Hands-on: Andes Cargo's severity matrix
- 6. On-call, with honesty about its cost
- 7. Hands-on: a deterministic on-call rotation
- 8. Project: Andes Cargo's `INCIDENT-RESPONSE-PLAN.md`
- 1. Introduction: operating, not narrating again
- 2. Hands-on: reconstructing the exact timeline
- 3. Hands-on: classifying the severity
- 4. Hands-on: declaring the incident
- 5. Communication during an incident: internal vs. external
- 6. Hands-on: the mitigation decision tree
- 7. This incident's error budget, with the full vocabulary
- 8. Project: Andes Cargo's final `TIMELINE.md` for this case
- 1. Introduction: from the timeline to systemic learning
- 2. What makes a postmortem "blameless"
- 3. Hands-on: writing the Claude Code incident postmortem
- 4. Action items that actually get done
- 5. What a runbook is (and is not)
- 6. Hands-on: Andes Cargo's first real runbook
- 7. Hands-on: the honest backup/restore attempt
- 8. Project: Andes Cargo's postmortem and runbooks package
- 1. Introduction to the capstone
- 2. Architecture review: the full reliability machine
- 3. Hands-on: introducing a deterministic synthetic incident
- 4. End-to-end: the alert fires, the incident gets declared
- 5. End-to-end: the runbook mitigates, the postmortem closes
- 6. What this guide left representative, final honesty
- 7. What Andes Cargo still needs
- 8. Final project: Andes Cargo's reliability package as a deliverable
Common questions
What people usually ask
No limit. It's a free guide: come in whenever you like, as often as you like.
No. Modules run from easier to harder, but you can jump to the one you need. Progress is saved per lesson.
Whatever is needed is listed under “What you need to bring”, above. If nothing is listed there, you can start from zero.
In the Club's WhatsApp group, and every two weeks there's a live with an instructor where questions get worked through.
Yes. It's issued automatically once you finish every lesson, with a verifiable code you can share on LinkedIn.
Start whenever you like
What students say
These reviews are from enrolled students who completed at least 50% of the course. We moderate reviews only on content grounds (spam, offensive language, personal data), never for being critical or negative.
No approved reviews yet.
Be the first to share your experience!