Anatomy of a Good Test-Spec

Capsule overview

You already know WHY to write tests first (capsule 02) and HOW the spec-first methodology works (capsule 03). But not just any test works as a good specification. A test that's ambiguous, dependent on other tests, or non-deterministic produces an unreliable spec — and Claude Code will generate equally unreliable implementations.

This capsule teaches you the 5 traits that make a test a good specification: deterministic, independent, focused, with a descriptive name, and with a single reason to fail. They're the design rules that separate a useful test from one that just takes up space.

By the end, you'll be able to evaluate any test and determine whether it works as spec-first — and you'll know how to fix tests that don't meet the criteria. This is a skill you'll use in all the remaining modules of the guide.

The 5 Traits of a Good Test-Spec

Trait 1: Deterministic

A deterministic test produces the same result every time it runs — regardless of order, time, machine, or how many times it's run.

# ✅ DETERMINISTIC: Always produces the same result
def test_add_two_numbers():
    assert add(2, 3) == 5

# ❌ NON-DETERMINISTIC: Depends on the time
def test_greeting_message():
    message = get_greeting()
    assert message == "Good morning"  # Fails if it's 3pm

# ❌ NON-DETERMINISTIC: Depends on external data
def test_latest_user():
    user = get_latest_user()
    assert user.name == "John"  # Fails if someone created another user

# ❌ NON-DETERMINISTIC: Depends on random
def test_random_password():
    password = generate_password()
    assert len(password) == 12  # OK
    assert password == "xK9!mP2@nQ4$"  # Fails every time

Why does it matter for spec-first? If the test isn't deterministic, Claude Code can't tell whether its implementation is correct or whether the test failed due to external factors. An ambiguous spec produces an ambiguous implementation.

How to make tests deterministic:

# Problem: Depends on the time
def test_greeting_depends_on_time():
    message = get_greeting()
    assert message == "Good morning"

# Solution: Control the input
def test_morning_greeting():
    message = get_greeting(hour=9)
    assert message == "Good morning"

def test_afternoon_greeting():
    message = get_greeting(hour=15)
    assert message == "Good afternoon"

# Problem: Depends on random
def test_random_password_bad():
    password = generate_password()
    assert password == "specific_password"

# Solution: Test properties, not exact values
def test_password_length():
    password = generate_password(length=12)
    assert len(password) == 12

def test_password_has_uppercase():
    password = generate_password(length=12, require_uppercase=True)
    assert any(c.isupper() for c in password)

Trait 2: Independent

Each test must be able to run on its own, without depending on other tests. It must not assume that another test already ran, or that certain data exists in a database.

# ❌ DEPENDENT: test_delete needs test_create to have run first
class TestUserBad:
    def test_create_user(self):
        user = create_user("john@test.com")
        assert user.id == 1
    
    def test_delete_user(self):
        delete_user(1)  # Assumes a user with id=1 exists
        assert get_user(1) is None

# ✅ INDEPENDENT: Each test creates its own setup
class TestUserGood:
    def test_create_user(self):
        user = create_user("john@test.com")
        assert user.email == "john@test.com"
    
    def test_delete_user(self):
        user = create_user("mary@test.com")  # Creates its own data
        delete_user(user.id)
        assert get_user(user.id) is None

Why does it matter for spec-first? If the tests are dependent, Claude Code needs to understand the execution order — and pytest doesn't guarantee order. An implementation that passes tests in one order can fail in another.

Pattern: Arrange-Act-Assert (AAA)

The AAA pattern guarantees independence by making each test have its own setup:

def test_discount_calculation():
    # ARRANGE: Prepare the necessary data
    price = 100.0
    discount = 20
    
    # ACT: Run the action you're testing
    result = calculate_discount(price, discount)
    
    # ASSERT: Verify the result
    assert result == 80.0

Each test has its own Arrange — it doesn't depend on the state left by another test.

Trait 3: Focused (one thing per test)

Each test must verify a single behavior. If a test verifies 5 things, when it fails you don't know which of the 5 is the problem.

# ❌ UNFOCUSED: Tests everything in one test
def test_user_registration():
    user = register("John", "john@test.com", "MyP@ss1!")
    assert user.name == "John"
    assert user.email == "john@test.com"
    assert user.is_active == True
    assert user.password != "MyP@ss1!"  # Must be hashed
    assert len(user.password) == 60  # Length of a bcrypt hash
    assert user.created_at is not None

# ✅ FOCUSED: One behavior per test
def test_register_stores_name():
    user = register("John", "john@test.com", "MyP@ss1!")
    assert user.name == "John"

def test_register_stores_email():
    user = register("John", "john@test.com", "MyP@ss1!")
    assert user.email == "john@test.com"

def test_register_activates_user():
    user = register("John", "john@test.com", "MyP@ss1!")
    assert user.is_active == True

def test_register_hashes_password():
    user = register("John", "john@test.com", "MyP@ss1!")
    assert user.password != "MyP@ss1!"

def test_register_sets_created_at():
    user = register("John", "john@test.com", "MyP@ss1!")
    assert user.created_at is not None

Why does it matter for spec-first? When you give focused tests to Claude Code and one fails, the error message says exactly which behavior doesn't work. Claude Code can fix the specific problem instead of reviewing the whole implementation.

# An unfocused test fails:
FAILED test_user_registration - AssertionError: assert 'MyP@ss1!' != 'MyP@ss1!'
→ Claude Code: "Is the problem the hashing? Or does something else in the test fail too?"

# A focused test fails:
FAILED test_register_hashes_password - AssertionError: assert 'MyP@ss1!' != 'MyP@ss1!'
→ Claude Code: "The password isn't being hashed. I need to add hashing."

Trait 4: Descriptive name

The test name must document the behavior it verifies. Without reading the test code, the name should tell you what happens if the test fails.

# ❌ BAD NAMES: They don't document behavior
def test_1():
    ...

def test_divide():
    ...

def test_edge_case():
    ...

def test_it_works():
    ...

# ✅ GOOD NAMES: They document behavior
def test_divide_positive_numbers_returns_float():
    ...

def test_divide_by_zero_raises_value_error():
    ...

def test_divide_negative_by_positive_returns_negative():
    ...

def test_divide_with_float_inputs_maintains_precision():
    ...

Naming pattern: test_[action]_[context]_[expected_result]

# Pattern: test_[what it does]_[under what condition]_[what you expect]

def test_login_with_valid_credentials_returns_token():
    ...

def test_login_with_wrong_password_returns_401():
    ...

def test_login_with_nonexistent_email_returns_404():
    ...

def test_register_with_duplicate_email_raises_conflict():
    ...

Why does it matter for spec-first? Test names are the most reliable documentation of your system. When someone (or Claude Code) reads your test names, they should understand the complete behavior without reading the code. They're your system's table of contents.

pytest -v

test_auth.py::test_login_with_valid_credentials_returns_token PASSED
test_auth.py::test_login_with_wrong_password_returns_401 PASSED
test_auth.py::test_login_with_nonexistent_email_returns_404 PASSED
test_auth.py::test_register_with_duplicate_email_raises_conflict PASSED
test_auth.py::test_token_expires_after_one_hour PASSED
test_auth.py::test_expired_token_is_rejected_with_401 PASSED

Reading just the test names, you fully understand how the authentication system works.

Trait 5: A single reason to fail

If a test can fail for multiple different reasons, it's hard to diagnose the problem. Each test must have exactly one reason it can fail.

# ❌ MULTIPLE REASONS TO FAIL:
def test_user_workflow():
    # Can fail because create_user fails
    user = create_user("John", "john@test.com")
    
    # Can fail because update_user fails
    updated = update_user(user.id, name="John Carter")
    
    # Can fail because the comparison fails
    assert updated.name == "John Carter"
    
    # Which one was the problem if it fails?

# ✅ A SINGLE REASON:
def test_create_user_stores_name():
    user = create_user("John", "john@test.com")
    assert user.name == "John"

def test_update_user_changes_name():
    user = create_user("John", "john@test.com")  # Arrange
    updated = update_user(user.id, name="John Carter")  # Act
    assert updated.name == "John Carter"  # Assert

Note: The second test still has a create_user in the Arrange. That's fine — if create_user fails, the test fails with a clear error (create_user raised Exception), not with an ambiguous assertion failure. The reason to fail of the assert is a single one: update_user didn't change the name.

Anti-Patterns: Tests That DON'T Work as Specs

Anti-pattern 1: A test that tests the implementation

# ❌ Tests HOW it's implemented (coupled to the implementation)
def test_sort_uses_quicksort():
    import unittest.mock as mock
    with mock.patch('sort_module.quicksort') as mock_qs:
        sort_list([3, 1, 2])
        mock_qs.assert_called_once()

# ✅ Tests WHAT it does (decoupled)
def test_sort_returns_ordered_list():
    assert sort_list([3, 1, 2]) == [1, 2, 3]

def test_sort_empty_list():
    assert sort_list([]) == []

def test_sort_already_sorted():
    assert sort_list([1, 2, 3]) == [1, 2, 3]

Why is it bad as a spec? If you test that it uses quicksort, Claude Code MUST use quicksort. But maybe merge sort is better for this case. The spec should define what result it produces, not how it produces it.

Anti-pattern 2: A test with complex logic

# ❌ The test has more logic than the code it tests
def test_fibonacci():
    expected = []
    a, b = 0, 1
    for _ in range(10):
        expected.append(a)
        a, b = b, a + b
    assert fibonacci(10) == expected

# ✅ Explicit values
def test_fibonacci_first_10():
    assert fibonacci(10) == [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]

Why is it bad as a spec? If the test computes the expected result with logic, and that logic has a bug, the test passes with an incorrect implementation. Expected values must be known constants, not computed.

Anti-pattern 3: A test that ignores edge cases

# ❌ Only the happy path
def test_divide():
    assert divide(10, 2) == 5
    assert divide(6, 3) == 2

# ✅ Happy path + edge cases
def test_divide_basic():
    assert divide(10, 2) == 5

def test_divide_by_zero():
    with pytest.raises(ZeroDivisionError):
        divide(10, 0)

def test_divide_zero_numerator():
    assert divide(0, 5) == 0

def test_divide_negative():
    assert divide(-10, 2) == -5

def test_divide_float_precision():
    assert divide(1, 3) == pytest.approx(0.333333, rel=1e-4)

Why is it bad as a spec? If the spec only has the happy path, Claude Code only implements the happy path. The edge cases you don't specify remain undefined behavior — the AI decides what to do, and it can decide wrong.

Checklist: Is My Test a Good Spec?

Use this checklist before giving your tests to Claude Code:

□ DETERMINISTIC
  Does it produce the same result every time it runs?
  Does it depend on time, date, external data, or random?

□ INDEPENDENT
  Can it run on its own, without other tests running first?
  Does it create its own setup (Arrange)?

□ FOCUSED
  Does it verify a single behavior?
  If it fails, do you know exactly what's wrong?

□ DESCRIPTIVE NAME
  Does the name document the behavior?
  Without reading the code, do you understand what it verifies?

□ A SINGLE REASON TO FAIL
  Does it have exactly one reason it can fail?
  Does the assert verify a single thing?

If any check fails, refactor the test before using it as a spec.

Project Connection

In the Spec-first mini-app (this module's project):

• Every test you write for the calculator will go through this checklist
• You'll see that deterministic, independent, and focused tests produce more precise implementations from Claude Code
• You'll practice naming that documents behavior: test_divide_by_zero_raises_value_error instead of test_error

These quality criteria apply throughout the guide — they're the foundation of unit tests (module 2), integration tests (module 3), and the final project (module 8).

Troubleshooting

Problem 1: "My tests feel repetitive"

Cause: Many tests that verify the same thing with minimal variations.

Solution: Use @pytest.mark.parametrize to group variations (covered in detail in module 2):

# Instead of 5 separate tests:
@pytest.mark.parametrize("input_val,expected", [
    (0, 32),
    (100, 212),
    (37, 98.6),
    (-40, -40),
])
def test_celsius_to_fahrenheit(input_val, expected):
    assert celsius_to_fahrenheit(input_val) == pytest.approx(expected)

Problem 2: "I can't make the test independent without duplicating a lot of setup"

Cause: Complex setup that every test needs.

Solution: Use pytest fixtures (covered in detail in modules 2 and 6):

import pytest

@pytest.fixture
def sample_user():
    return create_user("test@test.com", "TestPass1!")

def test_update_name(sample_user):
    updated = update_user(sample_user.id, name="New Name")
    assert updated.name == "New Name"

def test_delete_user(sample_user):
    delete_user(sample_user.id)
    assert get_user(sample_user.id) is None

Problem 3: "I don't know if my test tests behavior or implementation"

Cause: A blurry line between the two.

Solution: Ask yourself: "If I change the internal implementation but the result is the same, does my test still pass?" If the answer is yes, you're testing behavior. If not, you're testing implementation.

# Question: If I switch from quicksort to mergesort, does the test pass?
def test_sort_uses_quicksort():  # NO → tests implementation ❌
def test_sort_returns_ordered():  # YES → tests behavior ✅

Exercises

Exercise 1: Evaluate tests (Easy)

For each test, identify which trait of a good test-spec it does NOT meet:

# Test A
def test_stuff():
    result = process_data([1, 2, 3])
    assert result is not None

# Test B
import random
def test_shuffle():
    data = [1, 2, 3, 4, 5]
    result = my_shuffle(data)
    assert result == [3, 1, 5, 2, 4]

# Test C
user_id = None
def test_create():
    global user_id
    user = create_user("test@test.com")
    user_id = user.id
    assert user.id is not None

def test_get():
    user = get_user(user_id)
    assert user.email == "test@test.com"

# Test A corrected
def test_process_data_returns_sum():
    assert process_data([1, 2, 3]) == 6

# Test B corrected
def test_shuffle_contains_all_elements():
    result = my_shuffle([1, 2, 3, 4, 5])
    assert sorted(result) == [1, 2, 3, 4, 5]

def test_shuffle_same_length():
    result = my_shuffle([1, 2, 3, 4, 5])
    assert len(result) == 5

# Test C corrected
def test_create_user():
    user = create_user("test@test.com")
    assert user.id is not None

def test_get_user_by_id():
    user = create_user("test2@test.com")
    retrieved = get_user(user.id)
    assert retrieved.email == "test2@test.com"

Exercise 2: Improve names (Easy)

Rewrite these test names so they document the behavior:

def test_1():
    assert validate_age(25) == True

def test_2():
    assert validate_age(-1) == False

def test_3():
    assert validate_age(0) == True

def test_error():
    with pytest.raises(TypeError):
        validate_age("twenty")

def test_validate_age_positive_number_returns_true():
    assert validate_age(25) == True

def test_validate_age_negative_number_returns_false():
    assert validate_age(-1) == False

def test_validate_age_zero_is_valid():
    assert validate_age(0) == True

def test_validate_age_string_input_raises_type_error():
    with pytest.raises(TypeError):
        validate_age("twenty")

test_validate_age_positive_number_returns_true PASSED
test_validate_age_negative_number_returns_false PASSED
test_validate_age_zero_is_valid PASSED
test_validate_age_string_input_raises_type_error PASSED

Exercise 3: Make them independent (Medium)

These tests are dependent. Refactor them so they're independent:

items = []

def test_add_item():
    items.append({"name": "laptop", "price": 999})
    assert len(items) == 1

def test_add_second_item():
    items.append({"name": "mouse", "price": 29})
    assert len(items) == 2

def test_total_price():
    total = sum(item["price"] for item in items)
    assert total == 1028

def test_add_item_to_empty_cart():
    cart = []
    cart.append({"name": "laptop", "price": 999})
    assert len(cart) == 1

def test_add_two_items():
    cart = []
    cart.append({"name": "laptop", "price": 999})
    cart.append({"name": "mouse", "price": 29})
    assert len(cart) == 2

def test_total_price_of_two_items():
    cart = [
        {"name": "laptop", "price": 999},
        {"name": "mouse", "price": 29},
    ]
    total = sum(item["price"] for item in cart)
    assert total == 1028

Exercise 4: Write a complete spec (Medium)

You need a function slugify(text: str) -> str that converts text to URL slug format. Example: "Hello World!" → "hello-world".

Write 8 tests that meet the 5 traits of a good test-spec.

import pytest
from text_utils import slugify

def test_slugify_simple_text():
    assert slugify("Hello World") == "hello-world"

def test_slugify_converts_to_lowercase():
    assert slugify("UPPERCASE") == "uppercase"

def test_slugify_replaces_spaces_with_hyphens():
    assert slugify("hello world") == "hello-world"

def test_slugify_removes_special_characters():
    assert slugify("Hello World!") == "hello-world"

def test_slugify_multiple_spaces_become_single_hyphen():
    assert slugify("hello   world") == "hello-world"

def test_slugify_strips_leading_trailing_spaces():
    assert slugify("  hello world  ") == "hello-world"

def test_slugify_empty_string():
    assert slugify("") == ""

def test_slugify_accented_characters():
    assert slugify("café résumé") == "cafe-resume"

Summary

In this capsule you learned:

• ✅ The 5 traits of a good test-spec: deterministic, independent, focused, descriptive name, a single reason to fail
• ✅ The AAA pattern (Arrange-Act-Assert) to guarantee independence
• ✅ The naming convention test_[action]_[context]_[result] to document behavior
• ✅ 3 anti-patterns: testing implementation, complex logic in tests, ignoring edge cases
• ✅ A validation checklist to evaluate your tests before using them as a spec
• ✅ Focused tests produce more precise feedback for Claude Code when something fails

Next capsule: Your first spec→implement cycle with Claude Code — hands-on, from start to finish.

Additional Resources

1. pytest: Good Practices - Official pytest best practices
2. Arrange-Act-Assert Pattern - A detailed explanation of the AAA pattern
3. Test Desiderata (Kent Beck) - The 12 properties of good tests, by the creator of TDD
4. Writing Clean Tests - Uncle Bob on testing definitions and practices
5. pytest Naming Conventions - Naming conventions so pytest discovers your tests

Module 1, Capsule 04 — Testing with Claude Code Guide Good tests = good specs = good implementations