Writing Failing Tests First — The Art of the Red Test

Capsule overview

In the previous capsule you understood the red-green-refactor cycle adapted for AI: Claude Code implements, you define what must happen with tests. But there's a critical detail that separates real TDD from superficial TDD: the test must fail first.

A test that passes immediately proves nothing. A test that fails (red) and then passes (green) after the implementation is the proof that the test was testing the right thing and that the code solves it. This capsule teaches you the art of writing effective red tests: tests that fail for the right reason, that have the right size, and that give Claude Code clear information to implement.

By the end, you'll know how to write tests that document exactly what you want, in incremental cycles that overwhelm neither Claude Code nor yourself.

Why the Test MUST Fail First

A test that passes immediately proves nothing

Imagine you write this test:

# test_calculator.py
from calculator import add

def test_add_numbers():
    assert add(2, 3) == 5

Then you ask Claude Code: "Implement the add function so it passes this test." Claude generates:

# calculator.py
def add(a: float, b: float) -> float:
    return 5  # Hardcoded!

You run pytest → it passes. Do you trust the code? No. Because you never saw the test fail. You don't know whether the test is verifying real addition or verifying that it returns 5.

The golden rule: if a test passes without you having implemented anything (or with a trivial implementation), that test is useless.

The failing test PROVED that it's testing the right thing

The correct flow:

1. You write test_add_numbers, which expects add(2, 3) == 5
2. add doesn't exist → ImportError or NameError
3. Claude Code implements add(a, b): return a + b
4. pytest → passes

Now you DO trust it: the test failed because the function didn't exist or wasn't doing the right thing. The implementation fixed it. The red → green transition is the proof of progress.

The failing test DEFINES what success means

In TDD, the test doesn't verify existing code — it defines the contract. The test says: "when the function receives X, it must return Y." Until the test passes, the feature doesn't exist. The red test is the living specification.

Red → Green is the proof of progress

Each red → green transition is a verifiable increment. If you write 5 tests and they all pass on the first prompt to Claude Code, you have no evidence that those tests do anything useful. If each test fails first and then passes after the implementation, you have 5 pieces of evidence that the code does what the tests say.

Anatomy of a Good Failing Test

It imports the function that doesn't exist yet (ImportError → good!)

A test that imports something nonexistent fails immediately. That's exactly what you want.

# test_inventory.py
import pytest
from inventory import add_item, remove_item, get_stock  # Inventory doesn't exist yet


def test_add_item_increases_stock():
    add_item("widget", 10)
    assert get_stock("widget") == 10

When you run pytest, you get:

ImportError: cannot import name 'add_item' from 'inventory'

That failure is positive: it confirms that the test is trying to use the API you expect. Claude Code will read the test and create the inventory module with add_item, remove_item, get_stock.

A clear assertion about the expected behavior

# ✅ GOOD: a clear expectation
def test_register_with_valid_email_creates_user():
    user = register_user(email="alice@example.com", password="secret123")
    assert user.email == "alice@example.com"
    assert user.id is not None


# ❌ BAD: a vague assert
def test_register_works():
    user = register_user("alice@example.com", "secret")
    assert user  # What exactly?

The assertion must be specific: an exact value, a type, an exception with a message. Claude Code reads the test to know what to implement; a vague assertion gives it room for a wrong interpretation.

Small scope — ONE behavior per test

A test that validates many things is hard to interpret when it fails. Which part failed?

# ❌ BAD: multiple behaviors
def test_authentication_system():
    user = register("alice@example.com", "pass")
    assert user is not None
    token = login("alice@example.com", "pass")
    assert token is not None
    decoded = decode_token(token)
    assert decoded["user_id"] == user.id
    # If it fails, was it registration? login? decode?


# ✅ GOOD: one behavior
def test_register_with_valid_email_creates_user():
    user = register("alice@example.com", "pass")
    assert user.email == "alice@example.com"


def test_login_with_valid_credentials_returns_token():
    register("alice@example.com", "pass")
    token = login("alice@example.com", "pass")
    assert token is not None
    assert isinstance(token, str)

A descriptive name that documents the intent

The test's name is documentation. Reading test_register_with_valid_email_creates_user you know exactly what's being tested. Reading test_register_works you don't.

Rule: the name must let you deduce the scenario and the expected result without reading the body.

Tests That Are Too Big vs Too Small

TOO BIG: test_authentication_system_works

# ❌ Tests everything at once
def test_authentication_system_works():
    # Registration
    user = register("a@b.com", "pass")
    # Login
    token = login("a@b.com", "pass")
    # Validation
    decoded = decode(token)
    assert decoded["user_id"] == user.id
    # Refresh
    new_token = refresh(token)
    assert new_token != token
    # Logout
    logout(new_token)
    assert validate(new_token) is False

If it fails, you don't know where. If Claude Code implements one part badly, the feedback is confusing. Besides, it's impossible to cycle: you can't "implement registration only" because the test demands everything.

TOO SMALL: test_returns_true

# ❌ Meaningless
def test_returns_true():
    assert is_valid("x") == True

The test "tests" something, but it doesn't document what behavior is expected. Validation of what? Under what rules?

JUST RIGHT: test_register_with_valid_email_creates_user

# ✅ One scenario, one result, a descriptive name
def test_register_with_valid_email_creates_user():
    user = register_user(email="alice@example.com", password="secret123")
    assert user.email == "alice@example.com"
    assert user.id is not None

Practical rule: each test should fail for exactly ONE reason. If a test can fail because A is missing, because B is missing, or because C is missing, split it into three tests.

A quick checklist before asking Claude Code for the implementation

Before giving a red test to Claude Code, verify:

• ✅ The test currently fails (ImportError or AssertionError)
• ✅ The test's name describes the scenario and the expected result
• ✅ The assertion is specific (an exact value, a type, or an exception with a message)
• ✅ The test has a single purpose
• ✅ The imports point to the module Claude Code must create or modify

Incremental Test Writing for a Feature

Example: Building a password strength checker

Let's say you want check_strength(password: str) -> str that returns "weak", "medium" or "strong".

Cycle 1: Basic length

You write the first test:

# test_password_strength.py
import pytest
from password_strength import check_strength


def test_password_too_short_returns_weak():
    assert check_strength("abc") == "weak"

You run pytest → it fails (ImportError or AssertionError). You ask Claude Code to implement it. Claude generates:

# password_strength.py
def check_strength(password: str) -> str:
    if len(password) < 8:
        return "weak"
    return "medium"

pytest passes. Cycle 1 complete.

Cycle 2: Upper and lowercase

You add another test:

def test_password_mixed_case_returns_medium():
    assert check_strength("AbcDef12") == "medium"

You run it → it may already pass (8 chars). But you want to differentiate "medium" from "strong". You adjust the specification: "strong" requires uppercase, lowercase, numbers AND special characters.

def test_password_mixed_case_and_numbers_returns_medium():
    assert check_strength("AbcDef12") == "medium"

The current implementation already returns "medium" for that. You add:

def test_password_only_lowercase_long_returns_weak():
    assert check_strength("abcdefgh") == "weak"

Now it fails. Claude Code updates:

def check_strength(password: str) -> str:
    if len(password) < 8:
        return "weak"
    has_upper = any(c.isupper() for c in password)
    has_lower = any(c.islower() for c in password)
    has_digit = any(c.isdigit() for c in password)
    if has_upper and has_lower and has_digit:
        return "strong"
    return "medium"

Wait: the test asked for "abcdefgh" → weak. The implementation could return "medium" (8 chars, lowercase). You need to refine: "weak" = no uppercase OR no numbers. The "medium" vs "strong" logic is defined with tests. Each test adds one new requirement.

Cycle 3: Special characters

def test_password_with_special_chars_returns_strong():
    assert check_strength("AbcDef12!@") == "strong"

It will fail until the implementation requires special characters for "strong". Claude Code adjusts the logic.

A summary of the incremental approach

• ✅ Each test adds one new requirement
• ✅ Claude Code's implementation grows incrementally
• ✅ Each cycle is short: one test → implement → validate
• ✅ If you write all the tests at once, you lose the benefit of immediate feedback

The example's progression table

Each row represents an independent red-green cycle. You never move on to the next test until the previous one passes.

What Makes a Failing Test USEFUL for Claude Code

A clear function signature (which function, which arguments)

# ✅ Claude Code knows: the add_item function, args (name, quantity)
def test_add_item_increases_stock():
    add_item("widget", 10)
    assert get_stock("widget") == 10

If the test uses a made-up or inconsistent API, Claude Code may implement something that doesn't fit with the rest of the system.

A clear expected output (an exact value or an exception)

# ✅ An exact value
assert check_strength("abc") == "weak"

# ✅ An exception with a type and a message
with pytest.raises(InsufficientStockError) as exc_info:
    remove_item("widget", 100)
assert "insufficient stock" in str(exc_info.value).lower()

Context through the name

The name test_register_with_valid_email_creates_user tells Claude Code: registration is being tested, with a valid email, and the result must be a created user.

Edge cases as separate tests

Don't mix many cases into a single test. Each edge case = one test.

def test_remove_item_from_empty_inventory_raises():
    with pytest.raises(InsufficientStockError):
        remove_item("widget", 1)


def test_remove_item_more_than_stock_raises():
    add_item("widget", 5)
    with pytest.raises(InsufficientStockError):
        remove_item("widget", 10)


def test_remove_item_exact_stock_empties_item():
    add_item("widget", 5)
    remove_item("widget", 5)
    assert get_stock("widget") == 0

The "Too Much at Once" Trap

Writing 20 tests before implementing anything

If you write all the tests for a complex feature before asking for an implementation, Claude Code receives a large block with possible contradictions or unclear priorities. What should it implement first? What's most important?

❌ A problematic flow:
1. You write 20 tests for auth (register, login, tokens, refresh, logout, edge cases)
2. "Claude, implement everything so they pass"
3. Claude generates 200 lines
4. 12 tests pass, 8 fail
5. The 8 failures have different causes → Claude gets confused trying to fix everything

Better: 2-3 tests → implement → 2-3 more → implement

✅ An effective flow:
1. You write: test_register_creates_user, test_register_duplicate_email_fails
2. "Claude, implement registration so these tests pass"
3. Claude implements → 2/2 pass
4. You write: test_login_returns_token, test_login_wrong_password_fails
5. "Claude, implement login so these tests pass"
6. Claude implements → 4/4 pass
7. You repeat for tokens, refresh, etc.

The sweet spot: enough tests to define the behavior, not so many that Claude Code has contradictory constraints.

A practical heuristic

• If you have 1-3 tests → usually fine for one cycle
• If you have 4-6 tests → still manageable if they're coherent
• If you have 7+ tests for a single feature in the first cycle → consider splitting into sub-features or separate cycles

Practice: Building a Feature Test-First

Case: An inventory tracker

We're going to build an inventory module with the operations:

• add_item(name, quantity) — adds stock
• remove_item(name, quantity) — removes stock, raises if there isn't enough
• get_stock(name) — returns the quantity in stock

Step 1: A test for add_item and get_stock

Create test_inventory.py:

# test_inventory.py
import pytest
from inventory import add_item, remove_item, get_stock


def test_add_item_increases_stock():
    add_item("widget", 10)
    assert get_stock("widget") == 10

Run pytest test_inventory.py -v. It must fail (ImportError).

A prompt to Claude Code:

I have this test that fails. Implement the inventory module with the functions add_item, remove_item, and get_stock so it passes. Use an in-memory dictionary to store the stock.

Claude generates:

# inventory.py
_stock: dict[str, int] = {}


def add_item(name: str, quantity: int) -> None:
    _stock[name] = _stock.get(name, 0) + quantity


def remove_item(name: str, quantity: int) -> None:
    current = _stock.get(name, 0)
    if current < quantity:
        raise ValueError("Insufficient stock")
    _stock[name] -= quantity


def get_stock(name: str) -> int:
    return _stock.get(name, 0)

pytest → passes. Cycle 1 complete.

A note on isolation: In a real project, you'd use @pytest.fixture(autouse=True) or a setUp/tearDown to clean _stock between tests and prevent one test from affecting another. For this teaching example, the order of the tests and the data chosen avoid collisions.

Step 2: A test for remove_item

Add:

def test_remove_item_decreases_stock():
    add_item("widget", 10)
    remove_item("widget", 3)
    assert get_stock("widget") == 7

pytest → passes (the implementation already supports it).

Step 3: A test for insufficient stock

class InsufficientStockError(Exception):
    pass

You need to define the exception. Update the test:

def test_remove_item_more_than_stock_raises():
    add_item("widget", 5)
    with pytest.raises(ValueError):  # Or InsufficientStockError if you define it
        remove_item("widget", 10)

If the implementation uses ValueError, the test passes. If you want a custom exception, add InsufficientStockError to inventory and update the test to use pytest.raises(InsufficientStockError).

Step 4: A test for a nonexistent item

def test_get_stock_nonexistent_item_returns_zero():
    assert get_stock("nonexistent") == 0

pytest → passes (.get(name, 0) already returns 0).

Step 5: Run the complete flow

The project's final structure:

inventory-tracker/
├── inventory.py      # The module implemented by Claude Code
├── test_inventory.py # The tests you wrote (test-first)
└── venv/

The command to run the tests after each change:

pytest test_inventory.py -v

Expected output when everything passes:

test_inventory.py::test_add_item_increases_stock PASSED
test_inventory.py::test_remove_item_decreases_stock PASSED
test_inventory.py::test_remove_item_more_than_stock_raises PASSED
test_inventory.py::test_get_stock_nonexistent_item_returns_zero PASSED

A summary of the practical flow

1. Write a test that fails
2. Run pytest (red)
3. Give the test + context to Claude Code
4. Claude implements
5. Run pytest (green)
6. Add the next test and repeat

A suggested prompt for Claude Code

When you ask for the implementation, include:

• The complete test file (or the relevant tests)
• The instruction: "Implement module X so these tests pass"
• Constraints: "use memory instead of a database", "the API must be exactly the one the tests use"

A complete example:

I have test_inventory.py with tests that fail with an ImportError. Implement inventory.py with the functions add_item(name, quantity), remove_item(name, quantity) and get_stock(name). Use an in-memory dictionary. remove_item must raise ValueError when there isn't enough stock. The module shouldn't have global state that persists between tests (each test can start with an empty inventory, or use a pattern that allows a reset).

Exercises

Exercise 1: Identifying tests that are too big (Easy)

Review this test and split it into smaller tests. Write the names of the new tests.

def test_user_profile_system():
    user = create_user("alice", "alice@example.com")
    user.update_profile(bio="Hello")
    user.add_avatar("avatar.png")
    profile = get_profile(user.id)
    assert profile["bio"] == "Hello"
    assert profile["avatar"] == "avatar.png"
    user.delete_avatar()
    assert get_profile(user.id)["avatar"] is None

Exercise 2: Writing the first red test (Easy)

You want a validate_email(email: str) -> bool function that returns True for valid emails and False for invalid ones. Write ONE test that must fail first (the function doesn't exist yet) and that clearly defines a valid case.

# test_email_validation.py
import pytest
from email_validator import validate_email


def test_valid_email_with_at_and_domain_returns_true():
    assert validate_email("user@example.com") is True

Exercise 3: An incremental cycle for a parser (Medium)

You want parse_duration(s: str) -> int that converts strings like "5m", "1h", "30s" into seconds. Design 3 tests in incremental order (each one adds a requirement). Write the tests and briefly describe the implementation after each cycle.

def test_parse_duration_seconds():
    assert parse_duration("30s") == 30

def test_parse_duration_minutes():
    assert parse_duration("5m") == 300

def test_parse_duration_hours():
    assert parse_duration("1h") == 3600

Exercise 4: The sweet spot for the number of tests (Medium)

You're building a POST /tasks endpoint that accepts {"title": str, "priority": int}. How many tests would you write in the first cycle before asking Claude Code for the implementation? Justify it.

Exercise 5: Diagnosing why a test doesn't fail (Medium)

You wrote this test and, to your surprise, it passed before you implemented anything:

def test_calculate_total_includes_tax():
    assert calculate_total(100, 0.1) == 110

You investigate and discover that there's a calculate_total function in another module that gets imported automatically. What do you do?

Exercise 6: Writing an effective prompt for Claude Code (Medium)

You have this failing test:

def test_search_products_filters_by_category():
    add_product("Widget A", category="electronics")
    add_product("Widget B", category="clothing")
    results = search_products(category="electronics")
    assert len(results) == 1
    assert results[0]["name"] == "Widget A"

Write a short prompt (2-4 lines) that you'd give Claude Code to implement the functionality.

Anti-patterns to avoid

When writing red tests for TDD with Claude Code, avoid:

• ❌ A test that never failed: If you didn't see the test red, you don't know whether it's testing anything real
• ❌ A test that verifies the implementation: E.g. asserting that an internal function was called N times
• ❌ A monolithic test: One test that validates registration + login + tokens in a single function
• ❌ Too many tests in one cycle: More than 5-6 new tests before implementing
• ❌ Vague assertions: assert result or assert user without specifying which property

Troubleshooting

Problem 1: "My test passes but I didn't implement anything"

Cause: The test imports a function that already exists somewhere else, or the assert is trivial (assert True).

Solution: Verify that the test imports from the module you're going to create or modify. Use explicit import paths. If the test passes without an implementation, the test isn't testing what you think it is.

Problem 2: Claude Code implements something that makes the test pass incorrectly

Example: The test expects add(2, 3) == 5 and Claude generates def add(a, b): return 5.

Solution: Add a second test that forces the real behavior: assert add(1, 4) == 5. A single test can be "guessed" by hardcoding. Two tests with different inputs make it clear that the logic must be correct.

Problem 3: "I wrote 15 tests and Claude Code doesn't know where to start"

Cause: Too many tests at once generate contradictions or unclear priorities.

Solution: Hide or comment out tests. Leave only 2-3 active, ask for the implementation, and when they pass, enable the next ones. Small cycles work better.

Problem 4: The test fails with an ImportError but I don't know if that's the "right" error

Clarification: An ImportError is a valid failure. It means the test is trying to use code that doesn't exist. What matters is that, once Claude Code implements the module, the failure changes: from ImportError to (ideally) nothing, or to an AssertionError if the logic is incorrect. If you still get an ImportError after implementing, the problem is in the import paths or file names.

Problem 5: "My test is too specific and now the code is coupled to implementation details"

Example: The test verifies that an internal function is called in a specific order, instead of verifying the result.

Solution: Focus the test on the observable behavior (output, observable side effects), not on how it's implemented. If the test requires _internal_helper() to exist, you're testing the implementation, not the contract. Rewrite the test to verify the final result.

Problem 6: Tests pass separately but fail together

Cause: Shared state between tests (global variables, files, a database) that doesn't get cleaned between runs.

Solution: Use fixtures with autouse=True to reset the state, or run each test in an isolated process with pytest -x to debug the first one that fails. In the inventory, a reset_inventory() function called in a fixture solves the problem.

Project Connection

In this module's project (A complete feature with TDD), you'll build an authentication system (registration, login, tokens) using TDD with Claude Code:

• ✅ Each aspect of auth will start with failing tests: test_register_creates_user, test_login_returns_token, test_expired_token_raises_error, etc.
• ✅ You'll write 2-3 tests per cycle, ask Claude Code for the implementation, and validate before adding more.
• ✅ The red tests you write here follow the same rules: one purpose per test, a descriptive name, clear expectations.

Capsule 04 covers the implementation cycle with Claude Code: how to give context, when to iterate and when to step in.

Summary

In this capsule you learned:

• ✅ A test that passes immediately demonstrates nothing; one that fails first and then passes proves that the test and the implementation are correct
• ✅ A good red test imports what doesn't exist, has a clear assertion, a single purpose and a descriptive name
• ✅ Avoid tests that are too big (hard to debug) and too small (meaningless)
• ✅ Write tests incrementally: 2-3 tests → implement → 2-3 more → implement
• ✅ A test that's useful for Claude Code has a clear signature, an explicit expected output and context in its name
• ✅ The sweet spot: enough tests to define the behavior, without saturating Claude Code with contradictory constraints

Next capsule: The implementation cycle with Claude Code — giving context, evaluating results, iterating and deciding when to step in.

Final checklist

Before moving on to the next capsule, you should be able to:

• ✅ Explain why a test that passes immediately proves nothing
• ✅ Write a test that fails for the right reason (a nonexistent function or incorrect logic)
• ✅ Tell a test that's too big from one that's too small from one that's "just right"
• ✅ Design 2-3 tests for the first cycle of a new feature
• ✅ Follow the flow: red test → Claude implements → green pytest → next test

Additional Resources

1. Test Driven Development: By Example (Kent Beck) - The origin of TDD and the importance of the red test
2. pytest: How to write and run tests - Official pytest documentation for test structure
3. The Cycle of TDD (Red-Green-Refactor) - Robert C. Martin on the TDD cycle
4. Article: Why TDD with AI is different - Spec-driven development with LLMs (Tweag)
5. Refactoring with Tests (Martin Fowler) - Using tests as a safe foundation for refactoring
6. Writing Good Tests (Microsoft DevBlog) - Principles for clear, maintainable tests

Module 4, Capsule 03 — Testing with Claude Code Guide