Regression Tests for Refactoring

Capsule description

This capsule formalizes what the previous capsules assumed: the test cycle that makes refactoring safe. "Tests first, refactoring later" isn't a slogan — it's a methodology with concrete steps. You're going to learn to write regression tests specifically for refactoring: tests that capture the current behavior before changing anything, and verify that the behavior is preserved afterward.

The difference from normal tests is the intent. A unit test verifies that a function does the right thing. A regression test for refactoring verifies that a function keeps doing exactly the same thing as before — regardless of whether "the right thing" is debatable. If the function has a bug and everyone depends on that bug, the regression test captures the bug as the expected behavior. Fixing the bug is another separate refactoring.

This capsule connects directly with guide #7 (Testing with Claude Code) and reinforces the spec-first and TDD techniques you learned there. Here we apply them specifically to the refactoring context.

The Safe Refactoring Cycle

The 6 steps

1. IDENTIFY — What you're going to refactor and why
2. CAPTURE — Write tests that document the current behavior
3. VERIFY — Run tests and confirm that they all pass (green)
4. REFACTOR — Make the structure change
5. VERIFY — Run tests and confirm that they still pass (green)
6. CLEAN UP — Refactor the tests if necessary (rename, etc.)

The rule: if step 5 fails, the refactoring broke something. Don't publish, don't move on — investigate and fix.

The mistake everyone makes

# What developers do (DANGEROUS):
1. Refactor ←──── jump straight here
2. "Test manually"
3. "It seems to work"
4. Push

# What you should do (SAFE):
1. Write tests that capture the current behavior
2. Verify that they pass
3. Refactor
4. Verify that they still pass
5. Push with confidence

Writing Regression Tests with Claude Code

Step 1: Ask Claude Code to capture the behavior

# Prompt:
> "Before refactoring create_order() in order_service.py,
   write regression tests that capture its current
   behavior. Include:
   1. Happy path: order created successfully
   2. Each validation it does (what inputs it rejects)
   3. The exact format of the return value
   4. Side effects: what's saved in the DB? what emails are sent?
   
   The tests should pass with the CURRENT code, before
   any change."

# Claude Code generates:
# tests/test_order_service_regression.py

import pytest
from services.order_service import create_order

class TestCreateOrderRegression:
    """Regression tests: capture the current behavior
    of create_order() BEFORE the refactoring."""
    
    def test_happy_path_returns_order_with_correct_fields(self):
        """The result has id, user_id, total, status."""
        result = create_order(valid_order_data)
        assert "id" in result
        assert result["user_id"] == 123
        assert result["status"] == "pending"
        assert isinstance(result["total"], float)
    
    def test_empty_items_raises_value_error(self):
        """Empty items produces a ValueError."""
        with pytest.raises(ValueError, match="No items"):
            create_order({**valid_order_data, "items": []})
    
    def test_missing_user_id_raises_value_error(self):
        """No user_id produces a ValueError."""
        data = {k: v for k, v in valid_order_data.items() if k != "user_id"}
        with pytest.raises(ValueError, match="No user"):
            create_order(data)
    
    def test_total_includes_tax_and_shipping(self):
        """Total = subtotal + tax (16%) + shipping."""
        result = create_order(order_with_known_prices)
        expected_subtotal = 100.0  # 2 items × $50
        expected_tax = 16.0        # 16%
        expected_shipping = 0.0    # free (subtotal >= 50)
        assert result["total"] == expected_subtotal + expected_tax + expected_shipping
    
    def test_coupon_reduces_total(self):
        """A 10% coupon reduces the subtotal before tax."""
        result = create_order(order_with_coupon)
        # subtotal=100, coupon=10%, so subtotal=90, tax=14.4
        assert result["total"] == 104.4
    
    def test_order_saved_to_database(self, db_session):
        """The order is persisted in the DB."""
        result = create_order(valid_order_data)
        order = db_session.query(Order).get(result["id"])
        assert order is not None
        assert order.user_id == valid_order_data["user_id"]

Step 2: Run and confirm green

> "Run the regression tests you just wrote.
   They ALL must pass. If any fails, the test has a
   bug — not the code."

# Expected output:
# ✅ 6 tests passed, 0 failed
# The tests correctly reflect the current behavior

Step 3: Refactor with confidence

> "Now extract the calculation logic from create_order()
   into calculate_order_total(). [The refactoring you want]"

Step 4: Verify that the tests are still green

> "Run the regression tests. Do they all still pass?"

# If they pass: ✅ Successful refactoring
# If they fail: ❌ The refactoring changed behavior — investigate

Types of Regression Tests

1. Input/output tests (the most common)

They capture what the function returns for specific inputs:

def test_calculate_returns_correct_total(self):
    """Captures the exact calculation of the current code."""
    result = calculate_total(items=[
        {"price": 25.00, "quantity": 2},
        {"price": 10.00, "quantity": 1}
    ])
    # The current code returns 69.6 (subtotal=60, tax=9.6)
    assert result == 69.6

2. Side effect tests

They capture what happens besides the return value:

def test_create_order_sends_confirmation_email(self, mock_email):
    """Captures that an email is sent after creating an order."""
    create_order(valid_data)
    mock_email.send.assert_called_once_with(
        to=valid_data["email"],
        subject="Order Confirmation",
        # Note: we don't check the exact body because it can change
    )

def test_create_order_publishes_event(self, mock_events):
    """Captures that an OrderCreated event is published."""
    result = create_order(valid_data)
    mock_events.publish.assert_called_once_with(
        "order_created",
        {"order_id": result["id"]}
    )

3. Error handling tests

They capture how the function handles errors:

def test_invalid_email_raises_validation_error(self):
    """Captures that an invalid email produces a ValidationError."""
    with pytest.raises(ValidationError):
        create_user(name="Ana", email="not-an-email")

def test_payment_failure_returns_402(self, mock_stripe):
    """Captures that a failed payment returns HTTP 402."""
    mock_stripe.charge.side_effect = stripe.CardError("declined")
    response = client.post("/api/orders", json=valid_data)
    assert response.status_code == 402

4. Integration tests (for structure refactoring)

When you move files or reorganize modules, integration tests verify that the pieces still fit together:

def test_full_order_flow_still_works(self):
    """The complete order flow works after the refactoring."""
    # Create user
    user = create_user("Ana", "ana@test.com")
    
    # Create order
    order = create_order(user_id=user.id, items=test_items)
    
    # Verify state
    assert order.status == "pending"
    
    # Process payment
    payment = process_payment(order.id, "tok_test")
    
    # Verify that everything is connected
    assert payment.order_id == order.id
    assert Order.get(order.id).status == "paid"

Claude Code as a Regression Test Generator

The master prompt

> "I need to refactor [function/class/module]. Before
   making any change, generate regression tests that
   completely capture its current behavior:
   
   1. For each known valid input, capture the exact output
   2. For each invalid input, capture the exact exception
   3. For each side effect (DB, email, events), capture
      that it happens
   4. For each edge case (null, empty, extremes), capture
      the behavior
   
   The tests should:
   - Pass with the current code (no changes)
   - Use pytest
   - Have descriptive names that explain what they capture
   - Use fixtures for shared setup
   
   Do NOT optimize or fix the behavior — capture
   exactly what the code does TODAY, including known
   bugs if any exist."

When Claude Code is especially valuable

# Scenario: a 200-line function without tests
# Manually: 2-3 hours writing tests
# With Claude Code: 10-15 minutes

> "The process_invoice() function in billing_service.py has
   200 lines and 0 tests. Analyze the function, identify
   all the execution paths (happy paths, error paths,
   edge cases), and generate complete regression tests.
   Run the tests to confirm that they pass."

# Claude Code:
# 1. Reads the function
# 2. Identifies 12 execution paths
# 3. Generates 15 tests
# 4. Runs and confirms green
# Time: ~10 minutes

Common Patterns

Pattern 1: Snapshot testing for complex outputs

When the output is complex (long JSON, HTML), capture a snapshot:

def test_generate_report_matches_snapshot(self):
    """The generated report must match the snapshot."""
    result = generate_report(test_data)
    
    # First time: saves the snapshot
    # Next times: compares with the saved snapshot
    expected = load_snapshot("report_output.json")
    assert result == expected

Pattern 2: Property-based testing for invariants

When the refactoring must preserve a property, not a specific value:

def test_total_is_always_positive(self):
    """The total is always positive, regardless of the items."""
    for items in [single_item, multiple_items, discounted_items]:
        result = calculate_total(items)
        assert result > 0, f"Negative total with items: {items}"

def test_total_increases_with_quantity(self):
    """More quantity always produces a higher total."""
    result_1 = calculate_total([{"price": 10, "quantity": 1}])
    result_2 = calculate_total([{"price": 10, "quantity": 2}])
    assert result_2 > result_1

Pattern 3: Before/After comparison

For extract refactoring, verify that old and new produce the same:

def test_extracted_function_matches_original(self):
    """The extracted function produces the same result."""
    # Call the original code (before the extract)
    original_result = original_create_order(test_data)
    
    # Call the new version (after the extract)
    new_result = new_create_order(test_data)
    
    assert original_result == new_result

How Much Testing Is Enough

The 80/20 rule

You don't need 100% coverage to refactor with confidence. You need:

• ✅ Happy path — the main flow works
• ✅ Validations — invalid inputs are still rejected
• ✅ Known edge cases — null, empty, extremes
• ✅ Critical side effects — DB saves, emails, events

What you do NOT need for refactoring

• ❌ Performance tests (refactoring doesn't change performance)
• ❌ UI/visual tests (if you're not changing the UI)
• ❌ Configuration tests (if you're not changing config)
• ❌ 100% branch coverage (the branches you don't touch don't need a test)

The key question

"If this test fails after the refactoring, does that mean I broke something?"

• If the answer is YES → the test is necessary
• If the answer is NO → the test is noise

Connection with the Project

In the Module Project (capsule 06), the first step before any refactoring is to write regression tests. The project evaluates whether you:

1. Write tests BEFORE refactoring
2. The tests capture the relevant behavior
3. The tests pass before AND after the refactoring
4. If a test fails, you investigate instead of deleting the test

Troubleshooting

Problem 1: The current code has no tests and is hard to test

Cause: Hardcoded dependencies, global state, no dependency injection.

Solution: Use Michael Feathers' approach (Working Effectively with Legacy Code):

> "The process_payment() function has hardcoded
   dependencies on Stripe and the DB that make it hard
   to test. Create a test that uses monkey-patching
   or minimal mocking to capture the behavior
   without needing real connections."

Problem 2: Regression tests fail BEFORE the refactoring

Cause: The tests have a bug, or the code has non-deterministic behavior.

Solution: Regression tests should pass with the current code. If they don't pass, the test is wrong:

> "This regression test fails with the current code.
   That means the test doesn't correctly capture
   the behavior. What does the function actually return
   for this input? Adjust the test."

Problem 3: Too many tests to write

Cause: The function has too many paths.

Solution: Prioritize the most frequent paths:

> "The function has 20 branches. Write regression tests
   only for the 8 most important paths: the happy path,
   the 3 main validations, the 2 most common error handlers,
   and the 2 critical side effects."

Problem 4: A test passes before but fails after — is it a refactoring bug?

Cause: It can be a refactoring bug OR a fragile test.

Solution: Investigate what changed:

> "The test_total_calculation test fails after the
   extract. Compare the value it returned before (69.6)
   with the one it returns now (69.60000000000001).
   Is it a real change or a floating point issue?"

Exercises

Exercise 1: Identify what to test (Easy)

For this function, list which regression tests you'd write:

def register_user(name, email, password):
    if len(password) < 8:
        raise ValueError("Password too short")
    if "@" not in email:
        raise ValueError("Invalid email")
    user = User(name=name, email=email)
    user.set_password(password)
    db.session.add(user)
    db.session.commit()
    send_welcome_email(email)
    return {"id": user.id, "name": name, "email": email}

def test_happy_path_returns_user_dict(self):
    result = register_user("Ana", "ana@test.com", "secure123")
    assert "id" in result
    assert result["name"] == "Ana"
    assert result["email"] == "ana@test.com"

def test_short_password_raises_error(self):
    with pytest.raises(ValueError, match="Password too short"):
        register_user("Ana", "ana@test.com", "short")

def test_invalid_email_raises_error(self):
    with pytest.raises(ValueError, match="Invalid email"):
        register_user("Ana", "not-email", "secure123")

def test_user_saved_to_db(self, db_session):
    result = register_user("Ana", "ana@test.com", "secure123")
    user = db_session.query(User).get(result["id"])
    assert user is not None

def test_welcome_email_sent(self, mock_email):
    register_user("Ana", "ana@test.com", "secure123")
    mock_email.assert_called_once_with("ana@test.com")

Exercise 2: Write a regression test prompt (Medium)

Write the complete prompt for Claude Code that generates regression tests for a process_payment(order_id, token) function that: validates the order, charges with Stripe, updates the status, and sends an email.

> "Generate regression tests for process_payment(order_id, token)
   in src/services/payment_service.py. The function validates the order,
   charges with Stripe, updates the status, and sends a confirmation email.
   
   Necessary tests:
   1. Happy path: successful payment, returns payment_id
   2. Order doesn't exist: what error does it raise?
   3. Order already paid: what error does it raise?
   4. Stripe card declined: what error does it raise? is a rollback done?
   5. Stripe timeout: retry or error?
   6. Side effect: order.status changes to 'paid' in the DB
   7. Side effect: the confirmation email is sent
   8. Side effect: the 'payment_completed' event is published
   
   Use mocking for Stripe and email. The tests should pass
   with the CURRENT code. Run and confirm green."

Exercise 3: Diagnose a test that fails post-refactoring (Medium)

After extracting calculate_tax() from create_order(), this test fails:

def test_total_with_tax(self):
    result = create_order(items=[{"price": 100, "quantity": 1}])
    assert result["total"] == 116.0  # 100 + 16% tax
# Actual: 116.00000000000001

How do you diagnose and resolve it?

def test_total_with_tax(self):
    result = create_order(items=[{"price": 100, "quantity": 1}])
    assert result["total"] == pytest.approx(116.0, abs=0.01)

Exercise 4: Minimum coverage for refactoring (Hard)

A function has 15 branches. You're only going to refactor the first 5 (extract into a new function). Which 8 tests do you write?

# Tests for the 5 branches you're going to refactor:
1. Happy path of branch 1
2. Happy path of branch 2
3. Error path of branch 3
4. Edge case of branch 4
5. Happy path of branch 5

# "No regression" tests for the 10 you DON'T touch:
6. A test that passes through branch 6 (the most common of the 10)
7. A test that passes through branch 10 (the most edge case)
8. An integration test that verifies the complete flow
   (passes through multiple branches)

Summary

In this capsule you learned:

• The safe refactoring cycle: identify → tests → verify → refactor → verify → clean up
• Regression tests capture the current behavior — not what it should be, but what it IS
• 4 types of regression tests: input/output, side effects, error handling, integration
• Claude Code generates regression tests by analyzing the function and its paths
• The 80/20 rule: happy path + validations + edge cases + critical side effects
• If the test fails post-refactoring: investigate, don't delete the test

Next capsule: Project — Coordinated Refactoring. You're going to apply rename, extract, move, and interface changes on a real codebase with regression tests at each step.

Additional Resources

1. Working Effectively with Legacy Code - Michael Feathers - The book on introducing tests into untested code
2. pytest Documentation - Complete pytest reference
3. pytest-snapshot - A snapshot testing plugin for pytest
4. Characterization Tests - Martin Fowler - The formal concept of regression tests for refactoring
5. Approval Tests - A testing framework that captures output and compares it with approved versions
6. Coverage.py - A code coverage tool for Python

Module 4, Capsule 05 — Refactoring & Legacy Code with Claude Code Guide