GuideIntermediate
Reviewing AI-Generated Tests
Develop the judgment to critically review tests written by an AI (Copilot, Claude, Cursor, and similar tools). AI generates tests fast and plausibly, but they can be tautological (they always pass, they prove nothing), test the implementation instead of the behavior, forget edge and error cases, have weak assertions that produce false greens, or use unrealistic data and misleading names. This guide teaches how to spot each of these defects, apply a systematic review checklist, and decide, with judgment, which test is trustworthy, which needs strengthening, and which needs rewriting. It closes with a capstone project: review a Reservo suite that was "generated by AI" with defects of several types, mark each one with its category, fix it, and deliver a review report along with the corrected suite.
- 64
- lessons
- 8
- modules
- English · Spanish
- available in
- Yes
- certificate
- Free
- access
Outcomes
What you'll be able to do
- Recognize the tautological test: the one that can never fail because it asserts on the output of the same code that produced it
- Distinguish tests that verify behavior from tests that verify implementation (internal calls, execution order, private attributes)
- Detect the edge and error cases AI tends to forget when it only covers the happy path
- Identify weak assertions and false greens: `assert result is not None`, partial assertions, tests with no real `assert`
- Detect unrealistic test data (magic numbers, impossible values) and test names that don't describe what they actually verify
- Apply a systematic checklist to review any AI-generated suite: can it fail? does it test behavior? does it cover edges? is the assertion strong? is the data honest?
- Review and fix a complete AI-generated suite as a capstone project, delivering a review report
Before you start
What you need to bring
It's for you if...
- Devs using Copilot, Claude, Cursor, or another AI tool to generate tests who need judgment to review them
- Teams that adopted AI-generated tests and want to avoid false greens in their suite
- Devs who already know how to write tests with pytest and want to specifically build the reviewer skill
Requirements and materials
- Knowledge of pytest and TDD (or the Testing Fundamentals & TDD Guide completed)
- Basic familiarity with `unittest.mock`
- Experience using an AI code-generation tool (Copilot, Claude, Cursor, etc.)
Content
The syllabus, module by module
Open any of them to see its lessons.
- 1. Module introduction: a green test is not a good test
- 2. The green that lies
- 3. Why the AI writes these tests
- 4. The reviewer's job: not does it pass? but what does this green mean?
- 5. The five defects at a glance
- 6. Keep, strengthen, or rewrite
- 7. The cost of a bad test that passes
- 8. Mini-project: classify an AI-generated suite
- 1. Module introduction: the test that can't fail
- 2. What a tautological test is
- 3. Recomputing the expected with the same code
- 4. The mock that returns exactly what you assert
- 5. `assert True` and always-green assertions
- 6. Asserting on constants
- 7. Detecting it with by-hand mutation
- 8. Mini-project: find the tautologies by mutating the code
- 1. Module introduction: the WHAT versus the HOW
- 2. Behavior versus implementation
- 3. Asserting on internal calls
- 4. Asserting on private attributes
- 5. The test that breaks on a refactor
- 6. When the implementation DOES matter
- 7. Rewriting to verify the result
- 8. Mini-project: from implementation tests to behavior tests
- 1. Module introduction: the cases the AI forgot
- 2. The happy-path bias
- 3. The exact edge that's missing
- 4. The untested error cases and exceptions
- 5. Empty, null, and extreme inputs
- 6. Detecting the gap: equivalence classes and branches
- 7. Filling the gap
- 8. Mini-project: add the missing edges and errors to the AI suite
- 1. Module introduction: the green that almost measures
- 2. What a weak assertion is
- 3. `assert result is not None`
- 4. `assert result` and truthy assertions
- 5. The test with no `assert`
- 6. The partial assertion that omits the key field
- 7. Type instead of value, and how to strengthen
- 8. Mini-project: strengthen the assertions of an AI suite
- 1. Module introduction: the test that lies with a green face
- 2. Why realistic data matters
- 3. Magic numbers and `foo`/`bar`
- 4. Impossible domain data
- 5. Generic names that say nothing
- 6. Names that lie about what they test
- 7. Fixing data and names
- 8. Mini-project: rewrite data and names to document the domain
- 1. Module introduction: the checklist that systematizes the five defects
- 2. The five-question checklist
- 3. Applying the checklist test by test
- 4. The verdict: trust, strengthen, or rewrite
- 5. Reviewing a full AI suite
- 6. Mutation as a whole-suite measure
- 7. When the AI got it right: recognizing a good test
- 8. Mini-project: apply the checklist to a Reservo suite
Common questions
What people usually ask
No limit. It's a free guide: come in whenever you like, as often as you like.
No. Modules run from easier to harder, but you can jump to the one you need. Progress is saved per lesson.
Whatever is needed is listed under “What you need to bring”, above. If nothing is listed there, you can start from zero.
In the Club's WhatsApp group, and every two weeks there's a live with an instructor where questions get worked through.
Yes. It's issued automatically once you finish every lesson, with a verifiable code you can share on LinkedIn.
Start whenever you like
What students say
These reviews are from enrolled students who completed at least 50% of the course. We moderate reviews only on content grounds (spam, offensive language, personal data), never for being critical or negative.
No approved reviews yet.
Be the first to share your experience!