Validation Loops with Claude Code

Capsule overview

You have tests that define the behavior. Claude Code implements. The tests fail. What now? In classic TDD, you read the error, identify the cause, fix, re-run, and repeat until green. It's manual, slow, and repetitive. Automatic validation loops change the game: you ask Claude Code not just to implement, but to read the errors, fix them, and re-run until every test passes. Claude Code does the fix-and-rerun cycle for you. This capsule is the differentiator of the whole guide: this is where agentic TDD becomes truly automatic.

By the end you'll master the prompt pattern for validation loops, you'll know when they work and when they fail, and you'll know your role during the cycle: writing clear specs, reviewing the result, and stepping in when Claude gets stuck.

What a Validation Loop Is

The manual cycle in traditional TDD

In classic TDD the flow is:

1. You write a red test
2. You implement the minimal code for it to pass
3. You run the tests
4. If they fail: you read the error, identify the cause, fix it manually, re-run, repeat
5. If they pass → refactor → next test

Step 4 is entirely human. You read pytest's traceback, understand what's wrong, edit the code, run it again. It can take 5 minutes or 30 depending on the complexity. And if there are 10 tests failing at once, you get lost in the sheer number of errors.

The automatic cycle with validation loops

A validation loop is the automation of that step 4:

1. You write the tests (the spec)
2. You ask Claude Code: "Implement X so these tests pass. If any test fails, read the error and fix it until they all pass."
3. Claude Code: implements → runs the tests → reads the failures → fixes → re-runs → repeats until green
4. You review the final result

Claude Code does the fix-and-rerun for you. You don't need to copy errors, explain what each traceback means, or say "fix it again". The agent iterates until convergence.

The key difference

The gain is enormous: you go from dozens of manual interactions to a single instruction that triggers the whole cycle.

How Validation Loops Work with Claude Code

The step-by-step flow

You write the tests (the spec)
        │
        ▼
You write the prompt: "Implement X. If tests fail, fix until green."
        │
        ▼
Claude Code implements
        │
        ▼
Claude Code runs: pytest test_file.py -v
        │
        ├── All pass ──► Claude finishes. You review.
        │
        └── Some fail
                │
                ▼
        Claude Code reads the output (traceback, AssertionError, values)
                │
                ▼
        Claude Code identifies the cause and fixes it
                │
                ▼
        Claude Code re-runs the tests ──► (back to the check: do they all pass?)

Claude Code has access to the terminal output. When pytest fails, it sees the AssertionError, the expected vs actual value, the traceback. With that it can infer what to change. You don't need to paste errors into the chat — it already has them.

What you need to give it

• ✅ The test file (or the path, if it has access to the repo)
• ✅ The exact command to run: pytest tests/test_X.py -v
• ✅ The explicit instruction: "If any test fails, read the error, identify the cause, fix it and re-run until they all pass"
• ✅ Optional constraints: "stdlib only", "maximum 50 lines", "no external dependencies"

What Claude Code does on its own

• Runs the command
• Reads stdout/stderr
• Interprets the failure
• Edits the code
• Runs it again
• Repeats until green (or until an iteration limit)

Your intervention during the loop is minimal. You only step in if you see Claude repeating the same error several times or going down the wrong path.

The Prompt Pattern for Validation Loops

The base template

Implement [function/module] so the tests in [test_file] pass.

After implementing, run: pytest [test_file] -v

If any test fails:
1. Read the error message
2. Identify the cause
3. Fix the implementation
4. Re-run the tests
5. Repeat until they all pass

Constraints: [list constraints]

Useful variants

With an explicit target file:

Implement the validate_password function in auth/validation.py 
so the tests in tests/test_validation.py pass.

Run: pytest tests/test_validation.py -v

If any test fails, read the error, fix it and re-run until they all pass.

Constraints:
- Stdlib only
- No external libraries
- A minimal implementation

With usage context:

Implement parse_config in config/loader.py so tests/test_config.py passes.
This function will be called from main.py to load environment variables.

pytest tests/test_config.py -v

If there are failures, iterate fixing until green.

Constraints: stdlib only, empty lines ignored

Frequent mistakes in the prompt

When Validation Loops Work Well

Validation loops converge quickly when:

• ✅ Clear, specific tests with descriptive names (test_password_too_short_raises vs test_validation)
• ✅ Well-defined signatures — explicit inputs and outputs
• ✅ A reasonable scope — one function or a small module, not a complete app
• ✅ Informative errors — asserts with messages, expected vs actual values visible

Tests that help

def test_password_too_short_raises():
    """A password with fewer than 8 characters must raise ValueError."""
    with pytest.raises(ValueError, match="at least 8"):
        validate_password("short")

The match="at least 8" and the docstring give context. If it fails, Claude sees what was expected.

def test_valid_email_returns_true():
    result = validate_email("user@example.com")
    assert result is True, f"Expected True for valid email, got {result}"

The message in the assert helps when the value is unexpected.

When Validation Loops Fail

They fail or get stuck when:

• ❌ Ambiguous tests — multiple valid implementations
• ❌ Too many tests at once — Claude gets lost in contradictions
• ❌ Tests with shared state — the execution order matters
• ❌ Very complex logic with many pieces interacting

An example: an ambiguous test

def test_format_output():
    assert format_output("hello") == "hello"

format_output could return "hello", "Hello", "HELLO", " hello ". The test doesn't discriminate. Claude can implement any of them and "pass", but it isn't the one you wanted.

The solution: Be more specific: assert format_output("hello") == "Hello" (capitalized) or document the contract.

An example: too many tests

If you give it 20 tests covering 5 different functions, Claude can fix one and break another. Reduce the scope: "Implement only validate_password. Tests 1-4 are for that function."

An example: tests with shared state

# test_bad.py
state = []

def test_a():
    state.append(1)
    assert len(state) == 1

def test_b():
    state.append(1)  # Assumes state is empty
    assert len(state) == 1  # Fails if test_a ran first

Tests that depend on the order or on global state confuse Claude. Each test must be independent.

Your Role During Validation Loops

You're not passive. Your job is:

1. Write clear tests

Garbage in, garbage out. Vague tests produce vague or incorrect implementations. Specific tests guide Claude toward the correct solution.

2. Define a reasonable scope

Don't ask for "implement the whole API". Ask for "implement the POST /users endpoint so these 5 tests pass". A small scope = shorter, converging loops.

3. Review the result

Passing tests doesn't guarantee good code. Check:

• Is the implementation correct or does it cheat?
• Is there over-engineering?
• Does it meet the constraints?

4. Step in when Claude is stuck

If Claude repeats the same error 3+ times, the loop isn't converging. Step in:

• Give more context: "The error is X. The cause is Y. You must do Z."
• Reduce the scope: split it into smaller tests
• Suggest an approach: "Use a regex for the email pattern"

A Real Example: A Validation Loop in Action

Let's follow a complete validation loop for a user registration module. The tests define the spec; Claude Code iterates until green.

Step 1: The tests

# tests/test_registration.py
import pytest
from auth.registration import register_user


def test_register_user_returns_user_id():
    """A successful registration returns a user ID."""
    user_id = register_user(email="alice@example.com", password="secure123")
    assert isinstance(user_id, str)
    assert len(user_id) >= 1


def test_register_user_rejects_short_password():
    """A password shorter than 8 characters must raise ValueError."""
    with pytest.raises(ValueError, match="at least 8"):
        register_user(email="bob@example.com", password="short")


def test_register_user_rejects_invalid_email():
    """An email without an @ must raise ValueError."""
    with pytest.raises(ValueError, match="valid email"):
        register_user(email="notanemail", password="password123")


def test_register_user_rejects_duplicate_email():
    """An already-registered email must raise ValueError."""
    register_user(email="dup@example.com", password="password123")
    with pytest.raises(ValueError, match="already registered"):
        register_user(email="dup@example.com", password="otherpass456")


def test_register_user_stores_password_hashed():
    """The password must not be stored in plain text."""
    # This test requires access to an internal store — for now
    # we leave it as integration; for a unit test we'd mock the store
    user_id = register_user(email="hash@example.com", password="secret123")
    assert user_id  # A basic check

For simplicity, in the example we work with 4 tests (we exclude the 5th, which requires a store). The auth.registration module doesn't exist yet.

Step 2: The prompt

Implement register_user in auth/registration.py so the tests 
in tests/test_registration.py pass.

Run: pytest tests/test_registration.py -v

If any test fails:
1. Read the error
2. Identify the cause
3. Fix the implementation
4. Re-run the tests
5. Repeat until they all pass

Constraints:
- Stdlib only
- For test_register_user_rejects_duplicate_email you need an in-memory store 
  (a dict) that persists between calls — use a module-level variable for now
- The password hashed with hashlib.sha256

Step 3: Claude's first implementation (2 tests fail)

Claude generates:

# auth/registration.py
import hashlib
import re

_registry: dict[str, str] = {}

def register_user(email: str, password: str) -> str:
    if len(password) < 8:
        raise ValueError("Password must be at least 8 characters")
    if "@" not in email:
        raise ValueError("Invalid email: must contain @")
    if email in _registry:
        raise ValueError("Email already registered")
    hashed = hashlib.sha256(password.encode()).hexdigest()
    user_id = hashed[:16]
    _registry[email] = hashed
    return user_id

When running pytest tests/test_registration.py -v:

• test_register_user_returns_user_id — ✅ Passes
• test_register_user_rejects_short_password — ❌ Fails: match="at least 8" doesn't match "Password must be at least 8 characters" (the message is fine, but match is a regex; "at least 8" isn't in the message)
• test_register_user_rejects_invalid_email — ❌ Fails: match="valid email" doesn't match "Invalid email: must contain @"
• test_register_user_rejects_duplicate_email — ✅ Passes

In fact, match in pytest does a substring match on the message. "at least 8" isn't in "Password must be at least 8" — "8" is. And "valid email" isn't in "Invalid email: must contain @". Let's adjust: the match "at least 8" could fail if the message says "8 characters"; "valid" is in "Invalid". It depends. Let's suppose they fail because of the wording.

Step 4: Claude reads the errors and fixes them

Claude sees pytest's output:

FAILED test_registration.py::test_register_user_rejects_short_password - 
AssertionError: Regex pattern 'at least 8' does not match 'Password must be at least 8 characters'

It adjusts the message so it matches:

# Fixes it to:
raise ValueError("Password must be at least 8 characters")  # it already had this
# The match "at least 8" should match "at least 8" in the message
# "Password must be at least 8 characters" contains "at least 8" — the regex looks for a substring
# In pytest.raises, match does re.search. "at least 8" in "Password must be at least 8 characters" → match

If it fails, it may be that the message says "8 characters" and not "at least 8". Claude could change it to:

raise ValueError("Password must be at least 8 characters")

And for the email:

raise ValueError("Email must be a valid email address")

Claude updates both messages and re-runs.

Step 5: The second iteration (1 test still fails)

After the fix, test_register_user_rejects_short_password and test_register_user_rejects_invalid_email pass. But test_register_user_rejects_duplicate_email fails because _registry gets cleared between tests (each test in pytest runs in a process; if the module is re-imported, the dict resets). Within a single process, the order would be: first register_user("dup@..."), then the second register_user("dup@...") — that's where it should fail.

The failure may be due to the test order. If test_register_user_rejects_duplicate_email runs before some other test that uses the same email, the state could be different. To isolate it, the test uses a unique email, "dup@example.com". In a typical run, the first register_user fills _registry["dup@example.com"], and the second must raise ValueError. It would work.

Let's suppose the real failure is something else: for example, that Claude generated an empty user_id in some edge case. Claude reviews the error, adjusts the logic and runs it again.

Step 6: Green — everything passes

After one or two more iterations, every test passes. Claude finishes.

Step 7: Human review

You review the code:

• Does it validate correctly?
• Is the in-memory store acceptable for this example?
• Is there over-engineering?

If everything is fine, the validation loop has done its job: from red tests to green with minimal intervention.

Optimizing Validation Loops

More useful error messages

Improving the asserts speeds up the loop:

# Not very informative
assert result == expected

# Better
assert result == expected, f"Expected {expected}, got {result}"

# Even better for structures
assert result == expected, f"Mismatch: {result!r} != {expected!r}"

Claude interprets faster when it sees the expected value and the actual one.

pytest plugins for better failure output

If pytest's errors aren't clear enough, you can use plugins that improve the output:

# pyproject.toml or pytest.ini
# [tool.pytest.ini_options]
# addopts = "-v --tb=short"

Or install pytest-clarity or pytest-verbose-parametrize to see the parametrize values in the test names. When Claude reads the output, names like test_foo[a@b.com-longpassword-False] give more context than test_foo[2]. pytest-instafail shows the failures immediately instead of at the end, which speeds up iterative debugging.

Using asserts with messages in parametrized tests

@pytest.mark.parametrize("email,password,should_raise", [
    ("a@b.com", "longpassword", False),
    ("bad", "longpassword", True),
])
def test_register_parametrized(email, password, should_raise):
    if should_raise:
        with pytest.raises(ValueError):
            register_user(email, password)
    else:
        uid = register_user(email, password)
        assert uid, f"Expected non-empty user_id for {email}"

A checklist before launching a validation loop

Before sending the prompt, verify:

• ✅ The tests have descriptive names (test_rejects_duplicate_email, not test_1)
• ✅ The asserts include messages when the expected value isn't obvious
• ✅ The scope is bounded (one function or a small module, not a complete subsystem)
• ✅ The functions' signatures are defined in the tests or in the prompt
• ✅ The constraints are written out explicitly (stdlib, no deps, etc.)
• ✅ The exact pytest command is in the prompt

If something is missing, the loop can drag on or diverge.

Practice: A Validation Loop with Price Validation

Implement a validate_price function that validates prices for an e-commerce site. Use the validation loop pattern.

The initial tests

# tests/test_prices.py
import pytest
from shop.validation import validate_price


def test_valid_price_returns_float():
    """A valid price returns the value as a float."""
    result = validate_price(19.99)
    assert result == 19.99


def test_string_price_parsed():
    """A numeric string must be parsed to a float."""
    assert validate_price("29.50") == 29.50


def test_negative_price_raises():
    """A negative price must raise ValueError."""
    with pytest.raises(ValueError, match="negative"):
        validate_price(-5.0)


def test_zero_price_raises():
    """A zero price must raise ValueError."""
    with pytest.raises(ValueError, match="zero"):
        validate_price(0)


def test_invalid_string_raises():
    """A non-numeric string must raise ValueError."""
    with pytest.raises(ValueError, match="invalid"):
        validate_price("not a number")

Create an empty shop/validation.py or one with def validate_price(price): raise NotImplementedError. Then use this prompt with Claude Code:

Implement validate_price in shop/validation.py so the tests 
in tests/test_prices.py pass.

pytest tests/test_prices.py -v

If any test fails, read the error, fix it and re-run until green.

Constraints: stdlib only

Observe how many iterations Claude needs to reach green. If it fails, check whether the tests are ambiguous or whether the error messages are clear.

Exercises

Exercise 1: Write the prompt (Easy)

You have tests/test_slugify.py with 4 tests for slugify(s: str) -> str. Write a complete prompt for a validation loop that implements slugify in utils/text.py.

Implement slugify(s: str) -> str in utils/text.py so the tests 
in tests/test_slugify.py pass.

After implementing, run: pytest tests/test_slugify.py -v

If any test fails:
1. Read the error message
2. Identify the cause
3. Fix the implementation
4. Re-run the tests
5. Repeat until they all pass

Constraints: stdlib only, a minimal implementation

Exercise 2: Improve the error messages (Medium)

This test gives little information when it fails:

def test_parse_date():
    assert parse_date("2024-01-15") == datetime(2024, 1, 15)

Rewrite it with an assert that shows the expected and actual value when it fails.

def test_parse_date():
    result = parse_date("2024-01-15")
    expected = datetime(2024, 1, 15)
    assert result == expected, f"Expected {expected!r}, got {result!r}"

def test_parse_date():
    result = parse_date("2024-01-15")
    assert result == datetime(2024, 1, 15), f"parse_date('2024-01-15') = {result!r}"

Exercise 3: Identify why a loop is failing (Medium)

Claude Code attempts a validation loop with 15 tests for an "invoice" module. After 5 iterations, 3 tests that seem to contradict each other are still failing. What would you do?

Exercise 4: Constraints to prevent over-engineering (Medium)

Claude Code often generates classes, helpers and extra validations when implementing is_valid_username(s: str) -> bool. What constraints would you add to the prompt to get a minimal implementation?

Exercise 5: A complete validation loop (Hard)

Create a validation loop from scratch for normalize_phone(phone: str) -> str that:

• Removes spaces and dashes
• Returns only digits
• Raises ValueError for empty strings or strings with no digits

Write the tests, the prompt and run (or simulate) the loop with Claude Code. How many iterations do you need to reach green?

# tests/test_phone.py
import pytest
from utils.phone import normalize_phone


def test_normalize_removes_spaces_and_dashes():
    assert normalize_phone("123-456-7890") == "1234567890"
    assert normalize_phone("123 456 7890") == "1234567890"


def test_normalize_empty_raises():
    with pytest.raises(ValueError, match="empty"):
        normalize_phone("")


def test_normalize_no_digits_raises():
    with pytest.raises(ValueError, match="no digits"):
        normalize_phone("abcdef")

Implement normalize_phone(phone: str) -> str in utils/phone.py.
It must pass the tests in tests/test_phone.py.

pytest tests/test_phone.py -v

If it fails, read the error, fix it and re-run until green.

Constraints: stdlib only

# utils/phone.py
def normalize_phone(phone: str) -> str:
    cleaned = "".join(c for c in phone if c.isdigit())
    if not phone or not cleaned:
        raise ValueError("empty or no digits")
    return cleaned

Exercise 6: When to step in (Medium)

Claude Code is on iteration 4 of the same validation loop. The same test fails with the same AssertionError as in iteration 2. What do you do?

Troubleshooting

Problem 1: Claude repeats the same error several times

Cause: The error message isn't explicit enough or the test is complex.

Solution:

• Include pytest's exact output in the prompt and a sentence: "Test X fails because it expects A but you get B. You must do C."
• Simplify the test: split it into smaller cases.
• Give an example of the expected input/output.

Problem 2: The tests pass but the code is incorrect

Cause: The tests don't cover every scenario or they're too permissive.

Solution:

• Use @pytest.mark.parametrize with several inputs to prevent hardcoding.
• Review the implementation manually; if it cheats, add tests that detect it.
• Add the constraint: "Don't hardcode. The implementation must be generic."

Problem 3: The loop takes too long (too many iterations)

Cause: The scope is too broad or the tests conflict.

Solution:

• Reduce the scope: ask it to implement a single function or a subset of the tests.
• Check that the tests don't contradict each other.
• Verify that the test file is the right one and that there are no hidden dependencies (order, global state).

Problem 4: Claude ignores the constraints

Cause: The constraints aren't very visible or there are too many.

Solution:

• Put the constraints at the end in a numbered list.
• Repeat the most important one: "IMPORTANT: stdlib only, no external dependencies."
• If it keeps ignoring them, ask in a follow-up message: "The implementation must use only the stdlib. Remove any imports of external libraries."

Problem 5: Tests that pass locally but fail in the loop

Cause: Environment differences (Python version, paths, imports) or state that gets reset.

Solution:

• Check that the prompt's command matches what you use locally.
• Avoid global state shared between tests; use fixtures for setup.
• Make sure the imports are correct (for example, from auth.registration import register_user).

Project Connection

The module's project (a Validation pipeline) requires at least one validation loop where Claude Code iterates until the tests pass.

A suggested flow:

1. You write tests for a component (for example, a data validator).
2. You use the validation loop prompt.
3. Claude implements and iterates until green.
4. You review the generated code.
5. You repeat for the other components.

The final project (Module 8) benefits from the same thing: clear specs in the form of tests and validation loops to automate the implementation of each piece.

Summary

• ✅ A validation loop automates the cycle: run the tests → read the errors → fix → re-run until green.
• ✅ In traditional TDD you do that cycle; with validation loops Claude Code does it.
• ✅ The prompt pattern includes: implement so the tests pass, the command to run, and the instruction to iterate until green.
• ✅ The loops work best with clear tests, a small scope and informative errors.
• ✅ They fail with ambiguous tests, too many tests at once or very complex logic.
• ✅ Your role: write good tests, define the scope, review the result and step in when Claude gets stuck.
• ✅ Optimize with asserts that show expected vs actual and, if needed, pytest plugins for better output.

Next capsule: Mocking external services in practice — HTTP APIs, databases, the filesystem and environment variables.

Additional Resources

1. Claude Code Documentation - Using Claude Code as an agent
2. pytest: Good Integration Practices - Well-structured tests
3. Martin Fowler: Test-Driven Development - TDD fundamentals
4. pytest: Parametrize - Parametrized tests
5. Anthropic: Prompt Engineering - How to write effective prompts
6. Claude Code Best Practices - Recommended practices for Claude Code

Module 6, Capsule 04 — Testing with Claude Code Guide