GuideBeginner
Test Failure Diagnosis
Learn to diagnose why a test fails — the skill no "how to write tests" guide covers. A red test is information, not an enemy: this guide teaches you the scientific method of debugging (hypothesis → experiment) applied to pytest failures, starting with reading a failure report without drowning (the traceback, assert-rewriting, `--showlocals`, `--tb=short/long/line`) and isolating a failure down to its minimal reproduction. From there you move into the interactive debugger with `pytest --pdb` and the core `pdb` commands, learn to diagnose flaky tests (that sometimes pass, sometimes don't) and tests with order dependence or shared state, and close with binary search: using `git bisect` to find exactly which commit broke a test. All the hands-on material is Reservo test suites with deliberately planted failures of different kinds, which you learn to diagnose one by one. The final project hands you a broken suite with several failures of different types and asks for a complete diagnosis log — symptom, hypothesis, experiment, cause, and fix — for each one.
- 64
- lessons
- 8
- modules
- English · Spanish
- available in
- Yes
- certificate
- Free
- access
Outcomes
What you'll be able to do
- Adopt the right mindset when facing a failure: a red test is information, not a failure, and apply the scientific method (hypothesis → experiment) to debugging
- Read a pytest failure report end to end: assert-rewriting, the bottom-up traceback, `--showlocals`, `--tb=short/long/line`, and distinguish where the failure IS from where it's REPORTED
- Isolate and reproduce a failure minimally: run just the failing test (`-k`, nodeid), `--lf`/`--ff`, and strip away noise down to the minimal reproduction
- Use the interactive debugger: `pytest --pdb` (post-mortem), `breakpoint()`/`--trace`, and the core `pdb` commands (`p`, `pp`, `l`, `w`, `n`, `s`, `c`) to inspect state at the exact point of failure
- Diagnose flaky (non-deterministic) tests: their typical causes (clock, randomness, external resources, order) and how to reproduce them reliably
- Detect order dependence and shared state between tests, including using `pytest-randomly` to expose it
- Bisect a problem with binary search: use `git bisect` to find which commit broke a test, or manually bisect a large suite down to the culprit test
- Diagnose and fix a suite with several planted failures of different types, delivering a full diagnosis log for each one
Before you start
What you need to bring
It's for you if...
- Devs who already write tests but get stuck in front of a failure they don't understand, with no method to move forward
- Teams tired of flaky tests that sometimes pass and sometimes don't, with no idea where to start diagnosing them
- Devs who avoid the debugger and debug with scattered `print()` calls, wasting time on failures `pdb` would solve in minutes
- Anyone who's gotten a "it broke in some commit from last week" and needs to find which one
Requirements and materials
- Testing Fundamentals and TDD guide completed (or equivalent: writing and running tests with pytest)
- Basic git knowledge (commits, log) for the bisection module
- Python 3.14 and pytest 9.1.1 installed locally (`pdb` is part of the standard library)
Content
The syllabus, module by module
Open any of them to see its lessons.
- 1. Module introduction: a red test is information
- 2. The mindset: a failure is a clue, not a defeat
- 3. The anatomy of a pytest failure report
- 4. Where to look first
- 5. The scientific method of debugging
- 6. Hypothesis and experiment
- 7. The antipattern: changing things at random
- 8. Mini-project: diagnose three Reservo failures
- 1. Module introduction: the report is a map, not a verdict
- 2. Assert-rewriting: why pytest shows you the values
- 3. Reading the traceback bottom-up
- 4. Controlling the traceback: the `--tb` options
- 5. `--showlocals`: seeing the variables at the point of failure
- 6. The common errors and what they mean
- 7. Where the failure is vs where it is reported
- 8. Mini-project: locate the cause in three reports
- 1. Introduction: isolating is shrinking the failure's universe
- 2. Running only the failing test, by nodeid
- 3. Filtering with `-k`: isolating by what the name says
- 4. `--lf` and `--ff`: letting pytest remember what failed
- 5. Stopping early: `-x` and `--maxfail`
- 6. Cutting the noise from the run
- 7. The minimal reproduction: the smallest code that still fails
- 8. Mini-project: isolate and reproduce the failure in three commands
- 1. Module introduction: the live state the report does not show you
- 2. What `pdb` is and when to reach for it
- 3. `pytest --pdb`: landing at the point of the failure
- 4. `breakpoint()` and `pytest --trace`: enter whenever you want
- 5. Navigating the stack with `w`, `u` and `d`
- 6. Inspecting the state with `p`, `pp` and `args`
- 7. Advancing with `n`, `s` and `c`
- 8. Mini-project: why does `cancel` refund the wrong amount
- 1. Module introduction: the failure that breaks repetition
- 2. What a flaky is and why it is toxic
- 3. The clock flaky: time-dependent tests
- 4. Reproducing a flaky: run it N times
- 5. Randomness without a seed
- 6. External resources: network, disk and database
- 7. Freezing time with injection
- 8. Mini-project: make a flaky Reservo test deterministic
- 1. Module introduction: the failure that depends on who ran before
- 2. The test that passes alone and fails together
- 3. Module and global shared state
- 4. Fixture leaks by scope
- 5. Execution order dependence
- 6. Exposing it with `pytest-randomly`
- 7. Diagnose the culprit pair and isolate it
- 8. Mini-project: a suite that passes in order and fails shuffled
- 1. Module introduction: finding when it broke
- 2. The binary search: the diagnostician's superpower
- 3. `git bisect` step by step
- 4. `git bisect run`: automating the search
- 5. Bisecting a large suite to the culprit test
- 6. Bisecting data and input to the minimum
- 7. When NOT to bisect
- 8. Mini-project: hunt the culprit commit with `git bisect run`
- 1. Module introduction: the complete diagnostic process
- 2. The border assert: the 48-hour refund
- 3. The `KeyError`: a crash, not a wrong value
- 4. The clock flaky: reproduce and then freeze
- 5. The order-dependent failure and shared state
- 6. Bisecting the regression in a throwaway repo
- 7. Writing the diagnostic log
- 8. Project: diagnose and fix the broken suite
Common questions
What people usually ask
No limit. It's a free guide: come in whenever you like, as often as you like.
No. Modules run from easier to harder, but you can jump to the one you need. Progress is saved per lesson.
Whatever is needed is listed under “What you need to bring”, above. If nothing is listed there, you can start from zero.
In the Club's WhatsApp group, and every two weeks there's a live with an instructor where questions get worked through.
Yes. It's issued automatically once you finish every lesson, with a verifiable code you can share on LinkedIn.
Start whenever you like
What students say
These reviews are from enrolled students who completed at least 50% of the course. We moderate reviews only on content grounds (spam, offensive language, personal data), never for being critical or negative.
No approved reviews yet.
Be the first to share your experience!