Module 4: Code Review of AI Output

Module 4: Code Review of AI Output

Next →

Module 4: Code Review of AI Output

Capsule overview

So far you've built awareness (module 1), mental frameworks (module 2), and hallucination detection (module 3). You understand the problem, you have thinking tools, and you know how to detect the most subtle error. Now it's time for the next step: having a professional code review process designed specifically for AI-generated code.

This module opens Phase 2 (Professional Code Review) and marks the most important transition in the guide: you go from "I understand the problem" to "I have tools to solve it." By the end of this module you'll have a professional checklist with 15+ items, you'll know how to prioritize what to review when time is limited, you'll recognize red flags specific to AI-generated code, and you'll have completed a real code review of a PR generated by Claude Code.

The key difference: code review of human code and code review of AI code aren't the same. With human code, you trust that the developer understands the business context and you review the implementation. With AI code, you can't trust that the model understands the context — you must verify that the implementation matches your intent. That changes the priorities, the red flags, and the entire process.

Module context

Where are we?

You're in Phase 2 of the guide, which covers professional code review and debugging. Phase 1 gave you the foundation:

Phase 1 (Completed)What you gained
Module 1: Only 3% Trust ItAwareness of the problem + calibrated trust
Module 2: Mental ModelsManaging an Intern, Circuit Breaker, Trust Calibration
Module 3: HallucinationsDetection of fake imports, invented APIs, fabricated logic

Now in Phase 2 you build the practical tools:

Phase 2 (Current)What you'll gain
Module 4: Code Review of AI OutputProfessional checklist + review process
Module 5: Common Error PatternsPattern recognition for frequent errors
Module 6: Debugging with Claude CodeSystematic debugging with AI as a tool

Why is code review the central skill?

Everything else — error patterns, debugging, regenerate vs edit — are extensions of code review. If you know how to review AI code with judgment, you detect the error patterns during the review. You apply debugging when the review finds something suspicious. You make the regenerate vs edit decision after a review. Code review is the hub that connects all the skills in this guide.

Professional objective

By the end of this module you'll be able to:

  • ✅ Prioritize what to review with the pyramid: security → business logic → edge cases → performance → style
  • ✅ Use a professional checklist of 15+ items specific to AI code
  • ✅ Recognize red flags that only appear in AI-generated code (not in human code)
  • ✅ Verify business logic: that the code does exactly what the business needs
  • ✅ Complete a professional code review of a PR generated by Claude Code

Module progression

Module map

CapsuleTopicWhat you'll learn
02What to Look For FirstThe priority pyramid: if you only have 5 minutes, what do you review
03AI Code Review Checklist15-20 actionable items grouped by category
04Red Flags in AI CodeSigns specific to AI: over-engineering, obsolete APIs, phantom abstractions
05Verify Business LogicThe hardest review: confirm that the code does what the business needs
06Exercise: Code Review of a PRA complete review of a realistic PR generated by Claude Code

Learning flow

First you learn to prioritize — because you can't review everything and you need to know where to invest your limited time (capsule 02). Then you build your professional checklist with specific and verifiable items, grouped by category (capsule 03). With the checklist ready, you go deeper into the red flags that are exclusive to AI code — things that don't exist in code review of human code (capsule 04). Then you tackle the hardest part: verifying that the business logic is correct, something no linter detects (capsule 05). Finally, you apply it all in a real code review of a PR with a mix of good code and subtle problems (capsule 06).

Human Code Review vs AI Code Review

Before getting into the capsules, you need to understand why this module exists as something separate from traditional code review.

What's the same

In both human and AI code:
├── Security is priority #1
├── Edge cases need coverage
├── Error handling must be complete
├── Tests must verify real behavior
└── The code must be maintainable

What's different

In HUMAN code:
├── You trust the dev understands the business → you review the implementation
├── Red flags: excessive complexity, deep nesting, code smells
├── The errors are logic or distraction errors
├── The dev can explain their decisions if you ask
└── The project context is in the dev's mind

In AI code:
├── You do NOT trust that AI understands the business → you verify the intent
├── Red flags: over-engineering, obsolete APIs, phantom abstractions
├── The errors are "confidence without comprehension" errors
├── AI can't explain why it chose an approach
└── The context is lost between prompts

A concrete example

You ask a human developer: "Implement a volume discount: 10% if they buy more than 10 units."

The developer understands that "more than 10" means quantity > 10 and that the discount applies to the total. If they get it wrong, it's probably an off-by-one (>= vs >).

You ask Claude Code the same thing. The code might:

  • Apply the discount to the unit price instead of the total
  • Use >= instead of > (or vice versa)
  • Create a complete discount table with 5 tiers that nobody asked for
  • Apply the discount only to the extra units (after the first 10)
  • Work perfectly for the base case but fail when quantity is 0

Each of those variations looks professional. The code compiles, the trivial tests pass, the linter is happy. Only a human who understands the business detects that the discount applies to the total, not the unit price.

The consequences of not doing AI code review

Consider this real scenario:

from fastapi import FastAPI, HTTPException
from decimal import Decimal

app = FastAPI()

SUBSCRIPTION_PRICES = {
    "basic": Decimal("9.99"),
    "pro": Decimal("29.99"),
    "enterprise": Decimal("99.99"),
}


@app.post("/subscriptions/upgrade")
async def upgrade_subscription(
    user_id: str,
    new_plan: str,
):
    user = get_user(user_id)
    current_plan = user["plan"]
    
    current_price = SUBSCRIPTION_PRICES[current_plan]
    new_price = SUBSCRIPTION_PRICES[new_plan]
    
    prorate = new_price - current_price
    
    charge_user(user_id, prorate)
    update_plan(user_id, new_plan)
    
    return {"status": "upgraded", "charged": float(prorate)}

At first glance, this code looks correct. It calculates the price difference and charges the proration. But:

What happens if new_plan == current_plan? → Charges $0 and "upgrades" to the same plan.
What happens if it's a downgrade? (enterprise → basic) → prorate is negative → does it charge -$90?
What happens if new_plan isn't in SUBSCRIPTION_PRICES? → Unhandled KeyError.
Does the proration consider the days remaining in the cycle? → No, it charges the full difference.
Is there auth? → No. Anyone can change any user's plan.

Five problems in 20 lines of code that pass the linter, pass trivial tests, and look professional. Without code review, this reaches production. With code review, you find them in 5 minutes.

What You're Going to Build in This Module

The 4 artifacts

Over the 5 following capsules, you'll build 4 artifacts you'll use for the rest of your career:

MODULE 4 ARTIFACTS:

1. PRIORITY PYRAMID (Capsule 02)
   └── What to review first when time is limited
   └── Security → Logic → Edge Cases → Performance → Style
   └── Time distribution by availability (5/15/30 min)

2. PROFESSIONAL CHECKLIST (Capsule 03)
   └── 20 specific and verifiable items
   └── Grouped by: Security, Logic, Edge Cases, AI-Specific, Quality
   └── Each item with: what to verify, how to verify, example of a failure

3. RED FLAGS CATALOG (Capsule 04)
   └── 8 red flags exclusive to AI code
   └── Over-engineering, earlier APIs, phantom abstractions
   └── Confidence without correctness, mixing frameworks
   └── A quick detection rule for each red flag

4. LOGIC VERIFICATION PROCESS (Capsule 05)
   └── 4 steps: requirements → trace → verify → edge cases
   └── The hardest and most important part of the review
   └── No automatic tool does it for you

How they connect

Priority Pyramid
      ↓ (tells you WHAT to review first)
Professional Checklist
      ↓ (tells you HOW to review each category)
Red Flags Catalog
      ↓ (tells you WHAT TO LOOK FOR specifically in AI code)
Logic Verification
      ↓ (tells you HOW TO VERIFY the hardest part)
Exercise: PR Review
      ↓ (you apply IT ALL together in a real scenario)

The Reality of AI Code Review in the Industry

What most developers do

The average developer with AI:
1. Asks Claude Code for code
2. Sees that it "looks good"
3. Runs it and it works → merge
4. Problems appear in production days/weeks later

What professionals do

The professional developer with AI:
1. Asks Claude Code for code
2. Applies the pyramid: security first
3. Runs the checklist: specific items by category
4. Looks for AI red flags: over-engineering, old APIs
5. Verifies business logic: does it do what the business needs?
6. Documents findings: severity, category, fix
7. Decision: approve, edit, regenerate, or reject

The difference isn't talent — it's process. The professional developer isn't smarter. They have a system that lets them find problems consistently. That's what you build in this module.

How much time does it add to the workflow?

Without review: 0 minutes (but you pay later in bugs and hotfixes)
With basic review (pyramid + security): 5-10 minutes
With complete review (checklist + red flags): 15-25 minutes
With exhaustive review (+ business logic): 25-40 minutes

Estimated ROI:
- 15 minutes of review now
- vs 2-4 hours of debugging a bug in production
- vs days of incident response for a security breach

Connection to the Project

This module's exercise

You're going to do a complete code review of a PR generated by Claude Code. The PR has ~100 lines of FastAPI code with a mix of good code and 6-8 problems distributed across all categories: security, business logic, edge cases, and AI red flags.

Connection to the capstone project (Module 8)

The checklist you build in this module is the most important artifact for the capstone project. In module 8, you'll receive a complete FastAPI codebase with 15-20 planted problems. Your checklist is your main tool for finding them. If your checklist is good, you'll find the problems. If it's incomplete, you'll miss them.

This moduleCapstone project (M8)
1 PR with 6-8 issuesComplete codebase with 15-20 issues
~100 lines of code~500-800 lines in 8-12 files
Checklist as a practice toolChecklist as an evaluation tool
30-45 minutes90-120 minutes

Limits: What this module does NOT cover

  • ❌ Step-by-step debugging — That's module 6. Here you identify problems; there you solve them.
  • ❌ Specific error patterns in detail — That's module 5. Here you have the general checklist; there you go deeper into each type of error.
  • ❌ When to regenerate vs edit — That's module 7. Here you do the review; there you decide the correct action.
  • ❌ Generic code review — This module focuses on what's specific to AI-generated code. General code review best practices are assumed known.

The Tone of This Module

This module has a professional and methodical tone. It's not improvisation — it's process. This is how seniors work: with a checklist, with clear priorities, and with the judgment to know where to invest their limited time.

You're not going to review every line with the same intensity. You're going to learn to be strategic: dedicate 80% of your attention to the 20% of the code that carries the most risk. That's not laziness — it's professionalism. A doctor doesn't do a complete exam every time a patient shows up with a headache. They evaluate symptoms, prioritize, and go deeper where it matters. You'll do the same with AI code.

Signs of success

By the end of this module, you'll know you succeeded if:

  • ✅ You can recite the priority pyramid without consulting it
  • ✅ Your checklist has 15+ items that are specific and verifiable (not generic)
  • ✅ You can identify at least 5 red flags exclusive to AI code
  • ✅ You can explain why verifying business logic is the hardest part of the review
  • ✅ You completed the PR code review with at least 5 of the 6-8 issues documented
  • ✅ You documented each finding with severity, category, and recommended action

Troubleshooting

Problem 1: "I already do code review — why do I need a different process for AI?"

Cause: Traditional code review assumes the author understands the context. With AI, that assumption doesn't apply. Solution: You don't replace your current process — you expand it. The security and edge case items are similar. What you add is: intent verification (does it solve MY problem?), AI red flags (over-engineering, old APIs), and business logic verification (does it do what the business needs, not what AI thinks it needs?).

Problem 2: "My team doesn't do formal code review — how do I apply this?"

Cause: Many teams have informal or nonexistent review processes. Solution: Start with yourself. Apply the checklist to your own AI code before committing. You don't need a formal team process to benefit. When your code has fewer bugs, the team will notice and ask how.

Problem 3: "The checklist feels bureaucratic — isn't it faster to 'just review'?"

Cause: Ad-hoc review feels faster but is less effective. Solution: "Just review" works for developers with 10+ years of experience who have internalized the checks. For everyone else, the checklist compensates for the lack of experience with a process. Over time, you internalize the checklist and no longer need to consult it — but it keeps running in your head.

Exercises

Exercise 1: Self-assessment (Easy)

Answer honestly:

  1. When Claude Code generates code, how much time do you spend reviewing it before using it?

    • a) < 1 minute
    • b) 1-5 minutes
    • c) 5-15 minutes
    • d) > 15 minutes

  2. Do you have a defined process or do you review "by feeling"?

  3. Have you found a bug in production that came from AI-generated code?

See reflection

If you answered (a) to the first question, you're at the "I accept without verifying" extreme. If you answered (d), you might be over-reviewing. The optimal point depends on the type of code:

  • Boilerplate/CRUD: 1-5 minutes (answer b)
  • Business logic: 5-15 minutes (answer c)
  • Security/finance: 15+ minutes (answer d)

If you don't have a defined process (question 2), this module gives you one. If you found AI bugs in production (question 3), this module teaches you to find them in review, not in production.

Exercise 2: Classify reviews (Easy)

For each type of code, define how much time you would spend on the review and why:

  1. A GET endpoint that returns the API version
  2. A service that processes credit card payments
  3. A function that formats dates for display
  4. An endpoint that lets admins delete user accounts
  5. A logging configuration file
See solution

  1. API version → 1 minute. Zero risk. Visual review: does it return the correct version? Does it not expose sensitive info?

  2. Card payments → 30-45 minutes. Maximum risk. Security (PCI compliance, secrets), logic (correct calculations, atomicity), edge cases (negative amount, API timeout). Consider: should a second pair of eyes review it?

  3. Date formatting → 2-3 minutes. Low risk. Verify: does it handle time zones? Invalid dates? Is the format what the frontend needs?

  4. Delete accounts → 15-20 minutes. High risk. Security (admins only?), logic (soft delete or hard delete?), edge cases (what happens to the user's data?), confirmation (is there a double confirmation?).

  5. Logging config → 3-5 minutes. Low risk but important. Does it log sensitive data? Is the level correct for production? Do the logs go to the right place?

Exercise 3: Identify the difference (Medium)

Read these two snippets. One was written by a human developer and the other by AI. Which is which? How do you know?

Snippet X:

@app.post("/users")
async def create_user(user: UserCreate, db: Session = Depends(get_db)):
    existing = db.query(User).filter(User.email == user.email).first()
    if existing:
        raise HTTPException(status_code=409, detail="Email taken")
    
    new_user = User(**user.model_dump())
    new_user.password_hash = hash_password(user.password)
    db.add(new_user)
    db.commit()
    db.refresh(new_user)
    return new_user

Snippet Y:

@app.post("/users", response_model=UserResponse, status_code=201)
async def create_user(
    user: UserCreate,
    db: AsyncSession = Depends(get_async_session),
    background_tasks: BackgroundTasks = BackgroundTasks(),
):
    """Create a new user account with email verification."""
    existing = await db.execute(
        select(User).where(User.email == user.email)
    )
    if existing.scalar_one_or_none():
        raise HTTPException(
            status_code=status.HTTP_409_CONFLICT,
            detail="An account with this email already exists",
        )
    
    new_user = User(
        email=user.email,
        name=user.name,
        password_hash=hash_password(user.password),
        is_verified=False,
        created_at=datetime.now(timezone.utc),
    )
    db.add(new_user)
    await db.commit()
    await db.refresh(new_user)
    
    background_tasks.add_task(
        send_verification_email,
        email=new_user.email,
        user_id=str(new_user.id),
    )
    
    return new_user
See solution

Snippet X is more likely to be human. It's direct, with no extras. It does exactly what was asked and nothing more. It uses simple patterns (db.query instead of select), doesn't add extra functionality.

Snippet Y is more likely to be AI. Signs:

  • Adds unrequested functionality (email verification, background tasks)
  • More verbose and "presentable" (docstring, status constants, detailed messages)
  • Uses async + SQLAlchemy 2.0 patterns (can be correct, but AI tends to use the most modern)
  • Adds is_verified and created_at that nobody explicitly asked for

Both snippets are functional. But Y has clear AI signs: over-completeness, excessive professional presentation, and functionality that anticipates future needs without being asked.

Note: this distinction isn't 100% reliable. A senior developer could write like Y, and AI could generate something like X. The point is to develop the intuition to know when to go deeper in the review.

Summary

  • This module opens Phase 2 and marks the transition from awareness to practical tools
  • AI code review ≠ human code review: different priorities, different red flags, different process
  • With human code you trust the developer's context; with AI code you verify the intent
  • The priority pyramid tells you what to review when time is limited: security > logic > edge cases > performance > style
  • The professional checklist is the artifact you take from this module and use for the rest of your career
  • The AI red flags are specialized knowledge that sets you apart from developers who only review human code
  • Verifying business logic is the hardest part: the linter doesn't detect it, the tests don't cover it
  • The final exercise is a real code review of a PR with problems distributed across all categories

Additional resources

  1. Google — Code Review Developer Guide - The industry standard for code review, adaptable to AI code
  2. Anthropic — Claude Code Best Practices - Official documentation with validation recommendations
  3. OWASP — Code Review Guide - A code review framework with a security focus
  4. Microsoft — Code Review Best Practices - Microsoft's perspective on the code review process
  5. Stack Overflow — AI-Generated Code Survey 2024 - Data on how developers handle AI code

Next capsule: What to Look For First — the priority pyramid that defines where to invest your limited time.

Debugging & Code Review with Claude Code — Module 4, Capsule 01 Claude Code Agentic Development Path — Guide #6 of 11

← PreviousExercise: Detecting Hallucinations in CodeNext →What to Look For First: The Priority Pyramid