Spec-First Methodology: Tests as Specification

Capsule overview

In the previous capsule you understood WHY TDD matters more with AI. Now you're going to learn the HOW: the spec-first methodology. The core idea is a radical inversion of control — instead of telling Claude Code "implement this" in natural language (ambiguous), you tell it "here are the tests that define what should happen" (precise, executable, verifiable).

Spec-first isn't an invention of this guide. It's based on Tweag's work with LLM-assisted development, where they discovered that tests are the most effective language for communicating requirements to an AI agent. This capsule teaches you the concrete methodology: how to invert the control, what changes in the way you think, and how to structure the workflow with Claude Code.

By the end, you'll have a clear mental framework: you are the behavior architect (you define what should happen), Claude Code is the builder (it implements how it happens). The tests are the blueprint that connects both roles.

The Inversion of Control

Classic TDD vs Agentic TDD

In classic TDD, the developer does everything:

Classic TDD (without AI):
┌────────────┐
│ Developer  │
│            │
│ 1. Writes  │─── test ──→ [Fails] RED
│    test    │
│            │
│ 2. Writes  │─── implementation ──→ [Passes] GREEN
│    code    │
│            │
│ 3. Improves│─── refactor ──→ [Still passing] REFACTOR
│    code    │
└────────────┘

In agentic TDD, the roles split:

Agentic TDD (with Claude Code):
┌────────────┐          ┌─────────────┐
│ Developer  │          │ Claude Code │
│            │          │             │
│ 1. Defines │── test ─→│ 2. Implements ──→ [Passes] GREEN
│    spec    │          │    code       │
│    (test)  │          │             │
│            │          │ 3. Refactors  ──→ [Still passing]
│ 4. Validates│←─ pytest─│             │
│            │          │             │
│ 5. Adjusts │── test ─→│ 6. Iterates  │
│    spec    │          │             │
└────────────┘          └─────────────┘

The inversion: In classic TDD, writing the implementation is YOUR most laborious job. In agentic TDD, Claude Code does that job. Your most important job becomes defining the spec (the tests).

What does "tests as specification" mean?

A specification is a document that defines what a system should do. Traditionally, it's written in natural language:

Specification in natural language (ambiguous):
"The system must calculate discounts. Valid discounts are
between 0% and 100%. The system must handle errors appropriately."

Problems:

• Does "handle errors appropriately" mean return None? Raise an exception? Return 0?
• Does "between 0% and 100%" include 0 and 100, or exclude them?
• What happens with non-numeric inputs?

Now compare it with a specification as tests:

def test_discount_basic():
    assert calculate_discount(100, 10) == 90.0

def test_discount_zero_percent():
    assert calculate_discount(100, 0) == 100.0

def test_discount_hundred_percent():
    assert calculate_discount(100, 100) == 0.0

def test_discount_negative_percent_raises():
    with pytest.raises(ValueError, match="between 0 and 100"):
        calculate_discount(100, -5)

def test_discount_over_hundred_raises():
    with pytest.raises(ValueError, match="between 0 and 100"):
        calculate_discount(100, 150)

def test_discount_negative_price_raises():
    with pytest.raises(ValueError, match="price must be positive"):
        calculate_discount(-50, 10)

This specification:

• ✅ Is not ambiguous — each test has an exact expected result
• ✅ Is executable — pytest validates it automatically
• ✅ Is verifiable — it passes or it doesn't, no interpretation
• ✅ Documents edge cases explicitly (0%, 100%, negatives)
• ✅ Defines how errors are handled (ValueError with a specific message)

The Spec-First Framework

Step 1: Define the behavior as tests

Before touching Claude Code, think about WHAT should happen. Not HOW it's implemented.

# Think about behavior, not implementation:

# ❌ "I need a function that uses regex to validate emails"
# (this is implementation, not behavior)

# ✅ "I need a function where:"
def test_valid_email():
    assert is_valid_email("user@example.com") == True

def test_missing_at():
    assert is_valid_email("userexample.com") == False

def test_missing_domain():
    assert is_valid_email("user@") == False

def test_empty_string():
    assert is_valid_email("") == False

def test_multiple_at_signs():
    assert is_valid_email("user@@example.com") == False

Rule: If you're thinking about regex, algorithms, or data structures, you're thinking about implementation. Think about inputs and outputs.

Step 2: Give the tests to Claude Code as context

Don't tell it "implement an email validator." Give it the tests:

Prompt to Claude Code:

"I have these tests in test_email.py:

[paste tests]

Implement the is_valid_email function in email_validator.py 
so that all the tests pass."

Why does it work? Claude Code receives:

• ✅ The function name (is_valid_email)
• ✅ The function signature (email: str -> bool)
• ✅ Concrete input/output cases
• ✅ Edge cases that MUST be handled
• ✅ The expected error behavior

There's no ambiguity. Claude Code has all the information it needs to implement correctly.

Step 3: Run the tests

pytest test_email.py -v

test_email.py::test_valid_email PASSED
test_email.py::test_missing_at PASSED
test_email.py::test_missing_domain PASSED
test_email.py::test_empty_string PASSED
test_email.py::test_multiple_at_signs PASSED

========================= 5 passed in 0.01s =========================

If they all pass: the implementation meets your specification. If any fail: Claude Code iterates.

Step 4: Iterate if needed

pytest test_email.py -v

test_email.py::test_valid_email PASSED
test_email.py::test_missing_at PASSED
test_email.py::test_missing_domain FAILED  ← This one fails
test_email.py::test_empty_string PASSED
test_email.py::test_multiple_at_signs PASSED

You give the output to Claude Code:

"test_missing_domain failed. Here's the output:
AssertionError: assert True == False
The function returns True for 'user@' but it should return False.
Fix the implementation."

Claude Code reads the precise feedback from the test and fixes it. You don't need to explain the problem in natural language — pytest explains it for you.

The Architect and Builder Analogy

You are the architect. You define the blueprints: how many rooms, what dimensions, where the windows go, what materials. You don't lay bricks — you define what should exist.

Claude Code is the builder. It receives the blueprints and builds. It knows how to lay bricks, mix concrete, install plumbing. But without blueprints, it builds whatever it sees fit — and that may not be what you wanted.

The tests are the blueprints. They're precise, verifiable, and unambiguous. Each test is an instruction: "this room must measure 4x5 meters" = "this function must return 90 when it receives 100 and 10."

Without blueprints (without tests):
Architect: "I want a nice house"
Builder: [builds something]
Architect: "No, that's not what I wanted"
Builder: [rebuild from scratch]

With blueprints (with tests):
Architect: [delivers detailed blueprints]
Builder: [builds according to blueprints]
Inspector: [verifies against blueprints] ← pytest
✅ "Everything meets the specifications"

Spec-First in Practice: A Complete Example

Let's look at the full spec-first cycle for a real function. Imagine you need a password strength checker.

1. Define the spec (you write the tests)

# test_password_checker.py
import pytest
from password_checker import check_password_strength


class TestPasswordStrength:
    """Spec: Password strength checker"""
    
    def test_strong_password(self):
        """A password with 8+ chars, uppercase, lowercase, number and symbol is 'strong'"""
        result = check_password_strength("MyP@ss1!")
        assert result["strength"] == "strong"
    
    def test_medium_password(self):
        """A password with 8+ chars and 3 of 4 criteria is 'medium'"""
        result = check_password_strength("MyPass12")
        assert result["strength"] == "medium"
    
    def test_weak_password(self):
        """A password with fewer than 8 chars is always 'weak'"""
        result = check_password_strength("abc")
        assert result["strength"] == "weak"
    
    def test_empty_password(self):
        """An empty password is 'weak'"""
        result = check_password_strength("")
        assert result["strength"] == "weak"
    
    def test_returns_criteria_met(self):
        """The result includes which criteria were met"""
        result = check_password_strength("MyP@ss1!")
        assert result["has_uppercase"] == True
        assert result["has_lowercase"] == True
        assert result["has_number"] == True
        assert result["has_special"] == True
        assert result["min_length"] == True
    
    def test_only_lowercase_criteria(self):
        """A lowercase-only password reports correctly"""
        result = check_password_strength("abcdefgh")
        assert result["has_uppercase"] == False
        assert result["has_lowercase"] == True
        assert result["has_number"] == False
        assert result["has_special"] == False
    
    def test_length_boundary_7_chars(self):
        """7 characters doesn't meet the minimum of 8"""
        result = check_password_strength("Aa1!567")
        assert result["min_length"] == False
        assert result["strength"] == "weak"
    
    def test_length_boundary_8_chars(self):
        """8 characters meets the minimum"""
        result = check_password_strength("Aa1!5678")
        assert result["min_length"] == True

Notice: You don't know (or care) how it's implemented. You only define what it should return for each input.

2. Give the spec to Claude Code

"I have this spec in test_password_checker.py:

[tests above]

Create password_checker.py with the check_password_strength function
that makes all the tests pass. The function receives a string and
returns a dict with 'strength' and the individual criteria."

3. Claude Code implements

Claude Code generates something like:

# password_checker.py
import re


def check_password_strength(password: str) -> dict:
    criteria = {
        "has_uppercase": bool(re.search(r"[A-Z]", password)),
        "has_lowercase": bool(re.search(r"[a-z]", password)),
        "has_number": bool(re.search(r"\d", password)),
        "has_special": bool(re.search(r"[!@#$%^&*(),.?\":{}|<>]", password)),
        "min_length": len(password) >= 8,
    }
    
    criteria_count = sum([
        criteria["has_uppercase"],
        criteria["has_lowercase"],
        criteria["has_number"],
        criteria["has_special"],
    ])
    
    if not criteria["min_length"]:
        strength = "weak"
    elif criteria_count >= 4:
        strength = "strong"
    elif criteria_count >= 3:
        strength = "medium"
    else:
        strength = "weak"
    
    return {"strength": strength, **criteria}

4. Validate with pytest

pytest test_password_checker.py -v

test_password_checker.py::TestPasswordStrength::test_strong_password PASSED
test_password_checker.py::TestPasswordStrength::test_medium_password PASSED
test_password_checker.py::TestPasswordStrength::test_weak_password PASSED
test_password_checker.py::TestPasswordStrength::test_empty_password PASSED
test_password_checker.py::TestPasswordStrength::test_returns_criteria_met PASSED
test_password_checker.py::TestPasswordStrength::test_only_lowercase_criteria PASSED
test_password_checker.py::TestPasswordStrength::test_length_boundary_7_chars PASSED
test_password_checker.py::TestPasswordStrength::test_length_boundary_8_chars PASSED

========================= 8 passed in 0.02s =========================

8/8 tests pass. The implementation meets your specification exactly.

Comparison: Natural Language vs Tests as Spec

When to use natural language? To give general context and motivation. "I need a password checker because users pick weak passwords." This helps Claude Code understand the domain.

When to use tests? To define specific behavior. What the function returns, how it handles errors, which edge cases it covers. This is what Claude Code implements.

Best practice: Combine both. Natural language for context + tests for specification.

Project Connection

In the Spec-first mini-app (this module's project):

• You'll apply exactly this framework: you define tests for a calculator, Claude Code implements
• You'll experience the difference between giving it instructions in natural language vs giving it tests as the spec
• You'll see how Claude Code produces more precise implementations when it has tests as context

The spec-first pattern you learn here repeats in every module of the guide.

Troubleshooting

Problem 1: "I don't know which tests to write — I don't know the domain"

Cause: Confusing "I don't know the implementation" with "I don't know the behavior." You don't need to know how a password checker is implemented to know that "abc" is a weak password.

Solution: Think like a user, not like a developer. What inputs would you give? What outputs would you expect? If you're still not sure, start with the simplest case and expand:

# Start here (the most obvious case):
def test_basic():
    assert add(2, 3) == 5

# Then expand:
def test_zero():
    assert add(0, 5) == 5

# Then edge cases:
def test_negative():
    assert add(-1, 1) == 0

Problem 2: "My tests are too specific — they constrain the implementation"

Cause: Testing implementation instead of behavior.

Solution: Test WHAT it returns, not HOW it computes it:

# ❌ Too specific (tests implementation):
def test_uses_regex():
    import re
    assert re.search(r"[A-Z]", "Hello")  # You force regex

# ✅ Test behavior:
def test_detects_uppercase():
    result = check_password_strength("Hello")
    assert result["has_uppercase"] == True

Problem 3: "Claude Code doesn't understand my tests"

Cause: Tests that depend on imports or setup you didn't give as context.

Solution: Include everything Claude Code needs to understand the tests:

• The complete test file (including imports)
• The name of the file where it should implement
• Any dependency or constraint ("use only the standard library")

Exercises

Exercise 1: Convert a natural spec to tests (Easy)

Convert this natural-language specification into tests:

"I need a function celsius_to_fahrenheit that converts temperature from Celsius to Fahrenheit. The formula is F = C * 9/5 + 32. It must work with negatives and zero."

import pytest
from temperature import celsius_to_fahrenheit

def test_freezing_point():
    assert celsius_to_fahrenheit(0) == 32.0

def test_boiling_point():
    assert celsius_to_fahrenheit(100) == 212.0

def test_body_temperature():
    assert celsius_to_fahrenheit(37) == 98.6

def test_negative_temperature():
    assert celsius_to_fahrenheit(-40) == -40.0

def test_absolute_zero():
    assert celsius_to_fahrenheit(-273.15) == pytest.approx(-459.67)

Exercise 2: Identify ambiguity (Easy)

Read this specification and list 3 ambiguities that tests would resolve:

"Implement a function truncate_text that shortens long text. If the text is longer than the limit, cut it and add '...' at the end."

def test_long_text_truncated_with_ellipsis():
    assert truncate_text("Hello World!", 5) == "Hello..."

def test_text_at_limit_not_truncated():
    assert truncate_text("Hello", 5) == "Hello"

def test_short_text_unchanged():
    assert truncate_text("Hi", 5) == "Hi"

def test_empty_text():
    assert truncate_text("", 5) == ""

def test_zero_limit():
    assert truncate_text("Hello", 0) == "..."

Exercise 3: Spec-first for a real case (Medium)

You need a function parse_duration(text: str) -> int that converts a duration in text form to seconds. Examples: "2h" → 7200, "30m" → 1800, "45s" → 45.

Write 6+ tests as the spec. Don't implement the function.

import pytest
from duration_parser import parse_duration

def test_hours():
    assert parse_duration("2h") == 7200

def test_minutes():
    assert parse_duration("30m") == 1800

def test_seconds():
    assert parse_duration("45s") == 45

def test_one_hour():
    assert parse_duration("1h") == 3600

def test_zero():
    assert parse_duration("0s") == 0

def test_invalid_unit_raises():
    with pytest.raises(ValueError, match="Invalid duration format"):
        parse_duration("5x")

def test_no_number_raises():
    with pytest.raises(ValueError, match="Invalid duration format"):
        parse_duration("h")

def test_empty_string_raises():
    with pytest.raises(ValueError, match="Invalid duration format"):
        parse_duration("")

def test_negative_raises():
    with pytest.raises(ValueError, match="must be non-negative"):
        parse_duration("-5m")

Exercise 4: Detect an incomplete spec (Medium)

These tests are for a function divide(a, b). Which edge cases are missing?

def test_basic_division():
    assert divide(10, 2) == 5.0

def test_division_with_remainder():
    assert divide(7, 2) == 3.5

def test_divide_by_zero():
    with pytest.raises(ZeroDivisionError):
        divide(10, 0)

def test_divide_negative_numbers():
    assert divide(-10, 2) == -5.0

def test_divide_two_negatives():
    assert divide(-10, -2) == 5.0

def test_divide_zero_by_number():
    assert divide(0, 5) == 0.0

def test_divide_very_large_numbers():
    assert divide(1e15, 1e10) == 1e5

def test_divide_very_small_result():
    assert divide(1, 3) == pytest.approx(0.333333, rel=1e-4)

def test_divide_float_inputs():
    assert divide(1.5, 0.5) == 3.0

Summary

In this capsule you learned:

• ✅ The inversion of control in agentic TDD: you define specs (tests), Claude Code implements
• ✅ Tests as specification: precise, executable, verifiable — superior to natural language
• ✅ The spec-first framework in 4 steps: define tests → give context → run → iterate
• ✅ The architect/builder analogy: you design the blueprints, Claude Code builds
• ✅ Complete spec-first in practice: a password checker from tests to a validated implementation
• ✅ Combining natural language (context) with tests (specification) for better results

Next capsule: Anatomy of a good test-spec — what makes a test a good specification.

Additional Resources

1. Tweag Blog - Articles on LLM development and spec-first methodologies
2. pytest: Writing Tests - Official documentation on assertions in pytest
3. Specification by Example (Gojko Adzic) - The concept of "specification by example" applied to software
4. BDD vs TDD - A comparison of Behavior-Driven and Test-Driven Development
5. Test Desiderata (Kent Beck) - Properties that make a good test

Module 1, Capsule 03 — Testing with Claude Code Guide You define the what, Claude Code implements the how