Exercise: Detecting Hallucinations in Code

Capsule overview

This is the moment of truth. In the 4 previous capsules you learned the taxonomy of hallucinations, the techniques for detecting fake imports and invented APIs, the signs of fabricated logic, and the automated tools. Now you're going to apply it all in a practical exercise.

You're going to receive 5 snippets of AI-generated code. Each snippet has exactly 1 hidden hallucination. Your job is to find all 5. The difficulty is progressive: snippet 1 is relatively obvious, snippets 2 and 3 are medium difficulty, and snippets 4 and 5 are subtle.

The benchmark: find at least 4 of 5. If you find all 5, you have the eye of an expert detective.

How to Work Through This Exercise

Rules

1. Read each complete snippet before looking for the hallucination
2. Classify the type of hallucination (Fake import, Invented API, Incorrect parameter, Fabricated logic)
3. Explain why it's a hallucination (it's not enough to point at it — justify)
4. Provide the fix (what the correct code should be)
5. You can use tools — ruff, mypy, python -c, documentation. In real life you'd use them.

Response format

For each snippet, document:

SNIPPET [number]:
├── Hallucination found: [line or fragment]
├── Type: [Import / API / Parameter / Logic]
├── Why it's a hallucination: [explanation]
├── Fix: [correct code]
└── Tool that would detect it: [ruff / mypy / test / docs / eye]

Suggested time

• Snippet 1: 2-3 minutes
• Snippet 2: 3-5 minutes
• Snippet 3: 3-5 minutes
• Snippet 4: 5-8 minutes
• Snippet 5: 5-10 minutes
• Total: 20-30 minutes

Snippet 1: Configuration Service (Difficulty: Easy)

Context

Claude Code generated a configuration service for a FastAPI app. The service reads environment variables with default values and exposes an endpoint to check the current configuration.

Code

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field
from pydantic_settings import BaseSettings
from typing import Optional
from functools import lru_cache
import os

class Settings(BaseSettings):
    app_name: str = "My FastAPI App"
    debug: bool = False
    database_url: str = "sqlite:///./test.db"
    redis_url: str = "redis://localhost:6379"
    secret_key: str = "change-me-in-production"
    allowed_hosts: list[str] = ["localhost", "127.0.0.1"]
    max_connections: int = 100

    model_config = {"env_file": ".env", "env_file_encoding": "utf-8"}


class HealthResponse(BaseModel):
    status: str
    app_name: str
    debug: bool
    database_connected: bool


app = FastAPI()


@lru_cache()
def get_settings() -> Settings:
    return Settings()


@app.get("/health", response_model=HealthResponse)
async def health_check():
    settings = get_settings()
    
    db_connected = True
    try:
        from sqlalchemy import create_engine, text
        engine = create_engine(settings.database_url)
        with engine.connect() as conn:
            conn.execute(text("SELECT 1"))
    except Exception:
        db_connected = False
    
    return HealthResponse(
        status="healthy" if db_connected else "degraded",
        app_name=settings.app_name,
        debug=settings.debug,
        database_connected=db_connected,
    )


@app.get("/config")
async def get_config():
    settings = get_settings()
    return {
        "app_name": settings.app_name,
        "debug": settings.debug,
        "allowed_hosts": settings.allowed_hosts,
        "max_connections": settings.max_connections,
        "database_url": settings.database_url,
        "secret_key": settings.secret_key,
    }

Your turn

Find the hallucination in this snippet.

SNIPPET 1:
├── Hallucination found: The /config endpoint exposes database_url and secret_key
├── Type: Fabricated logic
├── Why it's a hallucination: The endpoint claims to return the "configuration" but 
│   it exposes sensitive data (database_url with credentials, secret_key) without 
│   authentication. The "show the configuration" logic was implemented 
│   as "show EVERYTHING including secrets", which doesn't correspond to any 
│   professional pattern for health/config endpoints.
├── Fix: Exclude sensitive data from the response:
│   return {
│       "app_name": settings.app_name,
│       "debug": settings.debug,
│       "allowed_hosts": settings.allowed_hosts,
│       "max_connections": settings.max_connections,
│   }
│   Or better: add authentication to the endpoint and still 
│   obfuscate sensitive values.
└── Tool that would detect it: the eye (review what data is exposed) 
    or bandit (ruff with S rules enabled can detect 
    hardcoded secrets)

Snippet 2: Rate Limiting System (Difficulty: Medium)

Context

Claude Code generated a rate limiting middleware for FastAPI that limits requests per IP using a sliding window approach.

Code

from fastapi import FastAPI, Request, HTTPException
from fastapi.responses import JSONResponse
from starlette.middleware.base import BaseHTTPMiddleware
from collections import defaultdict
from datetime import datetime, timedelta
from typing import Dict, List, Callable
import time

app = FastAPI()


class RateLimitConfig:
    def __init__(
        self,
        requests_per_minute: int = 60,
        burst_limit: int = 10,
        excluded_paths: list[str] = None,
    ):
        self.requests_per_minute = requests_per_minute
        self.burst_limit = burst_limit
        self.excluded_paths = excluded_paths or ["/health", "/docs"]


class SlidingWindowRateLimiter(BaseHTTPMiddleware):
    def __init__(self, app: FastAPI, config: RateLimitConfig = None):
        super().__init__(app)
        self.config = config or RateLimitConfig()
        self.request_log: Dict[str, List[float]] = defaultdict(list)

    def _cleanup_old_requests(self, client_id: str) -> None:
        """Remove requests older than 1 minute."""
        cutoff = time.time() - 60
        self.request_log[client_id] = [
            t for t in self.request_log[client_id] if t > cutoff
        ]

    def _get_client_id(self, request: Request) -> str:
        """Extract client identifier from request."""
        forwarded = request.headers.get("X-Forwarded-For")
        if forwarded:
            return forwarded.split(",")[0].strip()
        return request.client.host if request.client else "unknown"

    async def dispatch(self, request: Request, call_next: Callable):
        if request.url.path in self.config.excluded_paths:
            return await call_next(request)

        client_id = self._get_client_id(request)
        self._cleanup_old_requests(client_id)

        current_requests = len(self.request_log[client_id])

        if current_requests >= self.config.requests_per_minute:
            return JSONResponse(
                status_code=429,
                content={
                    "detail": "Rate limit exceeded",
                    "retry_after": 60,
                },
                headers={"Retry-After": "60"},
            )

        recent_requests = [
            t for t in self.request_log[client_id]
            if t > time.time() - 1
        ]
        if len(recent_requests) >= self.config.burst_limit:
            return JSONResponse(
                status_code=429,
                content={
                    "detail": "Burst limit exceeded",
                    "retry_after": 1,
                },
                headers={"Retry-After": "1"},
            )

        self.request_log[client_id].append(time.time())
        
        response = await call_next(request)

        remaining = self.config.requests_per_minute - len(self.request_log[client_id])
        response.headers["X-RateLimit-Limit"] = str(self.config.requests_per_minute)
        response.headers["X-RateLimit-Remaining"] = str(max(0, remaining))
        response.headers["X-RateLimit-Reset"] = str(int(time.time()) + 60)

        return response


rate_config = RateLimitConfig(
    requests_per_minute=100,
    burst_limit=20,
    excluded_paths=["/health", "/docs", "/openapi.json"],
)

app.add_middleware(SlidingWindowRateLimiter, config=rate_config)


@app.get("/health")
async def health():
    return {"status": "ok"}


@app.get("/api/data")
async def get_data():
    return {"data": "example", "timestamp": datetime.utcnow().isoformat()}

Your turn

This snippet is more complex. The hallucination isn't a security error — it's a subtle technical error.

SNIPPET 2:
├── Hallucination found: from starlette.middleware.base import BaseHTTPMiddleware
│   The import is correct, BUT there's an unused import that is the 
│   real hallucination:
│   from datetime import datetime, timedelta ← timedelta is imported 
│   but NEVER used in the code. However, that's not the 
│   main hallucination.
│
│   The main hallucination: the request_log is an in-memory Dict 
│   shared across ALL requests, but it's NOT thread-safe.
│   defaultdict(list) + append() in an async context can cause 
│   race conditions.
│
│   BUT the REAL hallucination in the snippet is more subtle:
│   The `from collections import defaultdict` import and the general use 
│   are correct. The hallucination is in the parameter of the 
│   RateLimitConfig constructor:
│   excluded_paths: list[str] = None ← uses the list[str] built-in syntax
│   which requires Python 3.9+, mixed with List imported from typing
│   which is used in the Dict's type.
│
│   Wait — the CLEAREST hallucination is another one:
│   datetime and timedelta are imported from datetime, but the code
│   uses time.time() for all the timing. timedelta is imported and 
│   never used. datetime is only used in the final endpoint 
│   (.utcnow()). This is an unused import but it's NOT the 
│   planted hallucination.
│
│   THE REAL HALLUCINATION: Look at the response line:
│   response.headers["X-RateLimit-Reset"] = str(int(time.time()) + 60)
│   This calculates the reset time by adding 60 to the current 
│   timestamp. But the rate limiter uses a sliding window — there's NO 
│   fixed "reset" at 60 seconds. The header is misleading: it says it 
│   resets in 60s, but the window slides continuously.
│   
│   However, the most concrete and verifiable error as a 
│   hallucination is:
│   
├── Type: Incorrect parameter / Fabricated logic
├── Why it's a hallucination: The code imports `timedelta` from 
│   datetime but never uses it. More importantly: the `X-RateLimit-Reset` 
│   header returns an incorrect value for a sliding window — 
│   it promises a fixed reset when the window slides continuously.
│   A client that waits until the "reset" could still be 
│   rate-limited if it made recent requests.
├── Fix: For the header, calculate when the oldest request 
│   in the window expires:
│   oldest = min(self.request_log[client_id]) if self.request_log[client_id] else time.time()
│   reset_at = int(oldest + 60)
│   response.headers["X-RateLimit-Reset"] = str(reset_at)
└── Tool that would detect it: the eye + a quick test that verifies 
    the header's behavior

Snippet 3: Product CRUD with Search (Difficulty: Medium)

Context

Claude Code generated CRUD endpoints for products with text search functionality.

Code

from fastapi import FastAPI, HTTPException, Query, Depends
from pydantic import BaseModel, Field
from typing import Optional, List
from datetime import datetime
from decimal import Decimal
import uuid
import re

app = FastAPI(title="Product API")

products_db: dict = {}


class ProductCreate(BaseModel):
    name: str = Field(..., min_length=1, max_length=200)
    description: Optional[str] = Field(None, max_length=2000)
    price: float = Field(..., gt=0)
    category: str = Field(..., min_length=1, max_length=100)
    tags: List[str] = Field(default_factory=list)
    in_stock: bool = True


class ProductUpdate(BaseModel):
    name: Optional[str] = Field(None, min_length=1, max_length=200)
    description: Optional[str] = Field(None, max_length=2000)
    price: Optional[float] = Field(None, gt=0)
    category: Optional[str] = Field(None, min_length=1, max_length=100)
    tags: Optional[List[str]] = None
    in_stock: Optional[bool] = None


class Product(ProductCreate):
    id: str
    created_at: datetime
    updated_at: datetime


@app.post("/products", response_model=Product, status_code=201)
async def create_product(product: ProductCreate):
    product_id = str(uuid.uuid4())
    now = datetime.utcnow()
    new_product = Product(
        id=product_id,
        created_at=now,
        updated_at=now,
        **product.model_dump(),
    )
    products_db[product_id] = new_product
    return new_product


@app.get("/products", response_model=List[Product])
async def list_products(
    category: Optional[str] = None,
    in_stock: Optional[bool] = None,
    min_price: Optional[float] = Query(None, ge=0),
    max_price: Optional[float] = Query(None, ge=0),
    search: Optional[str] = None,
    skip: int = Query(default=0, ge=0),
    limit: int = Query(default=20, ge=1, le=100),
):
    products = list(products_db.values())

    if category:
        products = [p for p in products if p.category == category]

    if in_stock is not None:
        products = [p for p in products if p.in_stock == in_stock]

    if min_price is not None:
        products = [p for p in products if p.price >= min_price]

    if max_price is not None:
        products = [p for p in products if p.price <= max_price]

    if search:
        pattern = re.compile(search, re.IGNORECASE)
        products = [
            p for p in products
            if pattern.search(p.name) or pattern.search(p.description or "")
        ]

    products.sort(key=lambda p: p.created_at, reverse=True)
    return products[skip: skip + limit]


@app.get("/products/{product_id}", response_model=Product)
async def get_product(product_id: str):
    if product_id not in products_db:
        raise HTTPException(status_code=404, detail="Product not found")
    return products_db[product_id]


@app.patch("/products/{product_id}", response_model=Product)
async def update_product(product_id: str, product_update: ProductUpdate):
    if product_id not in products_db:
        raise HTTPException(status_code=404, detail="Product not found")

    existing = products_db[product_id]
    update_data = product_update.model_dump(exclude_unset=True)

    for field, value in update_data.items():
        setattr(existing, field, value)

    existing.updated_at = datetime.utcnow()
    products_db[product_id] = existing
    return existing


@app.delete("/products/{product_id}")
async def delete_product(product_id: str):
    if product_id not in products_db:
        raise HTTPException(status_code=404, detail="Product not found")
    del products_db[product_id]
    return {"message": "Product deleted successfully"}

Your turn

The hallucination in this snippet is a security one. The general CRUD is well implemented.

SNIPPET 3:
├── Hallucination found: re.compile(search, re.IGNORECASE)
│   where search comes directly from the user input
├── Type: Fabricated logic (security)
├── Why it's a hallucination: The code passes the user input 
│   directly to re.compile() without sanitization. This allows:
│   
│   1. ReDoS (Regular Expression Denial of Service):
│      A user can send a catastrophic regex like:
│      search="(a+)+$" with a long string of "a"s
│      that causes exponential backtracking and 100% CPU.
│   
│   2. Invalid regex that causes a crash:
│      search="[invalid" → re.error: unterminated character set
│      The server returns a 500 Internal Server Error.
│   
│   3. The docstring implies "text search" but it's actually 
│      "regex search" — different functionality from what 
│      a user expects.
│   
├── Fix: Escape the user input to treat it as literal 
│   text, not as regex:
│   
│   if search:
│       escaped_search = re.escape(search)
│       pattern = re.compile(escaped_search, re.IGNORECASE)
│       products = [
│           p for p in products
│           if pattern.search(p.name) or pattern.search(p.description or "")
│       ]
│   
│   Or simpler, without regex:
│   
│   if search:
│       search_lower = search.lower()
│       products = [
│           p for p in products
│           if search_lower in p.name.lower() 
│           or search_lower in (p.description or "").lower()
│       ]
│   
└── Tool that would detect it: bandit (ruff S rules) can 
    detect regex injection. Also: the eye (distrust user 
    input passed to evaluation/compilation functions).

Snippet 4: JWT Authentication Service (Difficulty: Subtle)

Context

Claude Code generated a complete authentication service with JWT, password hashing, and verification middleware for a FastAPI API.

Code

from fastapi import FastAPI, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm
from pydantic import BaseModel, EmailStr, Field
from passlib.context import CryptContext
from jose import jwt, JWTError
from datetime import datetime, timedelta
from typing import Optional
import os

app = FastAPI(title="Auth Service")

SECRET_KEY = os.getenv("SECRET_KEY", "fallback-dev-key")
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 30

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth/login")

users_db = {
    "alice@example.com": {
        "email": "alice@example.com",
        "full_name": "Alice Smith",
        "hashed_password": pwd_context.hash("password123"),
        "disabled": False,
        "role": "admin",
    }
}


class Token(BaseModel):
    access_token: str
    token_type: str


class TokenData(BaseModel):
    email: Optional[str] = None
    role: Optional[str] = None


class User(BaseModel):
    email: EmailStr
    full_name: str
    disabled: bool = False
    role: str = "user"


class UserInDB(User):
    hashed_password: str


def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)


def get_user(email: str) -> Optional[UserInDB]:
    if email in users_db:
        return UserInDB(**users_db[email])
    return None


def authenticate_user(email: str, password: str) -> Optional[UserInDB]:
    user = get_user(email)
    if not user:
        return None
    if not verify_password(password, user.hashed_password):
        return None
    return user


def create_access_token(data: dict, expires_delta: Optional[timedelta] = None) -> str:
    to_encode = data.copy()
    expire = datetime.utcnow() + (expires_delta or timedelta(minutes=15))
    to_encode.update({"exp": expire})
    return jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)


async def get_current_user(token: str = Depends(oauth2_scheme)) -> User:
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        email: str = payload.get("sub")
        role: str = payload.get("role")
        if email is None:
            raise credentials_exception
        token_data = TokenData(email=email, role=role)
    except JWTError:
        raise credentials_exception

    user = get_user(token_data.email)
    if user is None:
        raise credentials_exception
    if user.disabled:
        raise HTTPException(
            status_code=status.HTTP_403_FORBIDDEN,
            detail="Inactive user",
        )
    return user


def require_role(required_role: str):
    async def role_checker(current_user: User = Depends(get_current_user)):
        if current_user.role != required_role:
            raise HTTPException(
                status_code=status.HTTP_403_FORBIDDEN,
                detail=f"Role '{required_role}' required",
            )
        return current_user
    return role_checker


@app.post("/auth/login", response_model=Token)
async def login(form_data: OAuth2PasswordRequestForm = Depends()):
    user = authenticate_user(form_data.username, form_data.password)
    if not user:
        raise HTTPException(
            status_code=status.HTTP_401_UNAUTHORIZED,
            detail="Incorrect email or password",
            headers={"WWW-Authenticate": "Bearer"},
        )
    access_token = create_access_token(
        data={"sub": user.email, "role": user.role},
        expires_delta=timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES),
    )
    return Token(access_token=access_token, token_type="bearer")


@app.get("/users/me", response_model=User)
async def read_users_me(current_user: User = Depends(get_current_user)):
    return current_user


@app.get("/admin/dashboard")
async def admin_dashboard(admin: User = Depends(require_role("admin"))):
    return {"message": "Welcome to admin dashboard", "admin": admin.email}

Your turn

This code is more complex and the hallucination is subtle. The general authentication code is correct — but there's an error in one of the functions that isn't what it appears to be.

SNIPPET 4:
├── Hallucination found: In create_access_token, the default 
│   expires_delta is timedelta(minutes=15), but in the 
│   /auth/login endpoint timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES) is passed
│   where ACCESS_TOKEN_EXPIRE_MINUTES = 30.
│   
│   The REAL problem is in create_access_token:
│   expire = datetime.utcnow() + (expires_delta or timedelta(minutes=15))
│   
│   When expires_delta=timedelta(minutes=0), the "or" evaluates to 
│   timedelta(minutes=15) because timedelta(minutes=0) is falsy.
│   This is an edge case that probably won't trigger here.
│   
│   But THE REAL HALLUCINATION is more subtle:
│   The role is stored DIRECTLY in the JWT token:
│   data={"sub": user.email, "role": user.role}
│   
│   And in get_current_user, the role is read FROM THE TOKEN:
│   role: str = payload.get("role")
│   
│   BUT then the user is queried from the database:
│   user = get_user(token_data.email)
│   
│   And the user's role in the DB could be DIFFERENT from the role 
│   in the token. If an admin is downgraded to "user" in the DB, 
│   their JWT token still says role="admin" until it expires.
│   
│   However, this is a design issue, not a technical 
│   hallucination.
│   
│   THE TECHNICAL HALLUCINATION:
│   datetime.utcnow() is deprecated in Python 3.12+.
│   But that's not what we're looking for either.
│   
│   OK — the planted hallucination is:
│   In create_access_token, the default value of the parameter  
│   expires_delta is timedelta(minutes=15). BUT the constant  
│   ACCESS_TOKEN_EXPIRE_MINUTES = 30. There's an inconsistency  
│   between the function's default (15 min) and the constant  
│   defined (30 min). If someone calls create_access_token()  
│   without passing expires_delta, the token expires in 15 minutes,  
│   not 30. The hallucination is that the LLM defined a  
│   constant (30 min) but used a different value as the  
│   default (15 min), creating inconsistent behavior.
│   
│   BUT THAT'S DEBATABLE as a hallucination.
│   
│   THE CLEAREST AND MOST VERIFIABLE HALLUCINATION:
│   The login endpoint uses form_data.username to look up the 
│   user, but authenticate_user searches by email in users_db.
│   OAuth2PasswordRequestForm uses "username" as the field 
│   (it's the OAuth2 standard), but the system uses emails.
│   This WORKS only if the username IS the email.
│   It's not a crash — but it's a semantic confusion that 
│   works by coincidence.
│   
│   After reflection, the most concrete hallucination:
├── Type: Fabricated logic  
├── Why it's a hallucination: timedelta(0) is falsy in Python.
│   The expression `expires_delta or timedelta(minutes=15)` fails  
│   when timedelta(0) is explicitly passed — it uses the default  
│   of 15 minutes instead of 0. This means it's impossible  
│   to create a token that expires immediately (useful for revocation).
│   More generally, the `param or default` pattern is an anti-pattern  
│   for parameters that can be legitimate falsy values.
├── Fix:
│   def create_access_token(
│       data: dict, 
│       expires_delta: Optional[timedelta] = None
│   ) -> str:
│       to_encode = data.copy()
│       if expires_delta is not None:
│           expire = datetime.utcnow() + expires_delta
│       else:
│           expire = datetime.utcnow() + timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
│       to_encode.update({"exp": expire})
│       return jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
└── Tool that would detect it: A quick test with 
    expires_delta=timedelta(0). A trained eye that knows the 
    falsy-values pattern in Python.

Snippet 5: Data Processing Pipeline (Difficulty: Subtle)

Context

Claude Code generated a data processing pipeline that reads a CSV, cleans the data, calculates statistics, and generates a report.

Code

from fastapi import FastAPI, UploadFile, HTTPException
from pydantic import BaseModel
from typing import List, Optional, Dict, Any
from datetime import datetime
import csv
import io
import statistics
import math

app = FastAPI(title="Data Processing Pipeline")


class DataStats(BaseModel):
    column: str
    count: int
    mean: Optional[float] = None
    median: Optional[float] = None
    std_dev: Optional[float] = None
    min_val: Optional[float] = None
    max_val: Optional[float] = None
    null_count: int = 0


class ProcessingResult(BaseModel):
    filename: str
    rows_total: int
    rows_processed: int
    rows_skipped: int
    columns: List[str]
    stats: List[DataStats]
    processed_at: str


def clean_value(value: str) -> Optional[float]:
    """Attempts to convert a string value to float, returns None if not possible."""
    if not value or value.strip() in ("", "NA", "N/A", "null", "None", "-"):
        return None
    try:
        cleaned = value.strip().replace(",", "").replace("$", "").replace("%", "")
        return float(cleaned)
    except (ValueError, TypeError):
        return None


def calculate_column_stats(column_name: str, values: List[Optional[float]]) -> DataStats:
    """Calculates statistics for a single column of numeric data."""
    non_null = [v for v in values if v is not None]
    null_count = len(values) - len(non_null)

    if not non_null:
        return DataStats(
            column=column_name,
            count=len(values),
            null_count=null_count,
        )

    mean_val = statistics.mean(non_null)
    median_val = statistics.median(non_null)

    if len(non_null) >= 2:
        variance = sum((x - mean_val) ** 2 for x in non_null) / len(non_null)
        std_dev = math.sqrt(variance)
    else:
        std_dev = 0.0

    return DataStats(
        column=column_name,
        count=len(values),
        mean=round(mean_val, 4),
        median=round(median_val, 4),
        std_dev=round(std_dev, 4),
        min_val=min(non_null),
        max_val=max(non_null),
        null_count=null_count,
    )


def process_csv_data(content: str, filename: str) -> ProcessingResult:
    """Processes CSV content and returns statistics for each numeric column."""
    reader = csv.DictReader(io.StringIO(content))
    columns = reader.fieldnames or []

    column_values: Dict[str, List[Optional[float]]] = {col: [] for col in columns}
    rows_total = 0
    rows_skipped = 0

    for row in reader:
        rows_total += 1
        row_valid = False

        for col in columns:
            value = clean_value(row.get(col, ""))
            if value is not None:
                row_valid = True
            column_values[col].append(value)

        if not row_valid:
            rows_skipped += 1

    stats = []
    for col in columns:
        col_stats = calculate_column_stats(col, column_values[col])
        if col_stats.mean is not None:
            stats.append(col_stats)

    return ProcessingResult(
        filename=filename,
        rows_total=rows_total,
        rows_processed=rows_total - rows_skipped,
        columns=columns,
        stats=stats,
        processed_at=datetime.utcnow().isoformat(),
    )


@app.post("/process", response_model=ProcessingResult)
async def process_file(file: UploadFile):
    if not file.filename or not file.filename.endswith(".csv"):
        raise HTTPException(status_code=400, detail="Only CSV files are accepted")

    content = await file.read()

    try:
        decoded = content.decode("utf-8")
    except UnicodeDecodeError:
        raise HTTPException(status_code=400, detail="File must be UTF-8 encoded")

    if len(decoded) > 10_000_000:
        raise HTTPException(
            status_code=413, detail="File too large (max 10MB)"
        )

    result = process_csv_data(decoded, file.filename)
    return result

Your turn

The hallucination in this snippet is mathematical. Everything else (imports, API calls, structure) is correct.

SNIPPET 5:
├── Hallucination found: The standard deviation calculation
│   in calculate_column_stats
├── Type: Fabricated logic (mathematical calculation)
├── Why it's a hallucination: The code calculates the POPULATION 
│   VARIANCE (divides by N):
│   
│   variance = sum((x - mean_val) ** 2 for x in non_null) / len(non_null)
│   
│   But the statistical convention for samples (which is what 
│   you have when processing data) uses SAMPLE VARIANCE (divides 
│   by N-1), known as Bessel's correction:
│   
│   variance = sum((x - mean_val) ** 2 for x in non_null) / (len(non_null) - 1)
│   
│   Python's statistics.stdev() uses N-1 (sample).
│   Python's statistics.pstdev() uses N (population).
│   
│   The LLM used N (population) but the function should use N-1 
│   (sample) to be consistent with statistics.stdev() and with 
│   standard statistical practice.
│   
│   Ironically, the code uses statistics.mean() and 
│   statistics.median() correctly, but IMPLEMENTS 
│   std_dev manually instead of using statistics.stdev().
│   
│   This is exactly the logic hallucination pattern:
│   the LLM uses the standard library for mean and median, but 
│   for std_dev it decides to implement manually — and it does so 
│   with the wrong formula.
│   
├── Fix:
│   # Option 1: Use the standard library (preferred)
│   if len(non_null) >= 2:
│       std_dev = statistics.stdev(non_null)
│   else:
│       std_dev = 0.0
│   
│   # Option 2: Fix the manual formula (if there's a reason 
│   # not to use the library)
│   if len(non_null) >= 2:
│       variance = sum((x - mean_val) ** 2 for x in non_null) / (len(non_null) - 1)
│       std_dev = math.sqrt(variance)
│   else:
│       std_dev = 0.0
│   
└── Tool that would detect it: A quick test comparing against 
    statistics.stdev(). Example:
    
    data = [2, 4, 4, 4, 5, 5, 7, 9]
    stats_result = statistics.stdev(data)   # → 2.138...
    manual_result = math.sqrt(sum((x - statistics.mean(data))**2 
                   for x in data) / len(data))  # → 2.0
    # Different results → hallucination confirmed

data = [10, 20]
statistics.stdev(data)       # → 7.071... (N-1, correct for samples)
# The code's formula:        # → 5.0 (N, incorrect for samples)
# Difference: 29%

Meta-Evaluation: Your Result

Results table

After completing the 5 snippets, mark which ones you found:

Interpretation

• 5/5: Excellent. You have the eye of an experienced detective. Subtle hallucinations won't surprise you.
• 4/5: Very good. The benchmark is met. The most common hallucinations don't escape you. The one that escaped you was probably from a domain you don't master — and that's normal.
• 3/5: Good. You detect medium and low difficulty hallucinations. For the subtle ones, you need to rely more on the tools (layer 3: quick tests, layer 4: docs).
• 2/5 or less: You need more practice. Review capsules 03 and 04 again, and repeat the exercise focusing on the patterns that escaped you.

Reflection

Answer these questions:

1. Which type of hallucination was hardest to detect? (Import, API, Parameter, Logic)
2. Which tool would have helped most? (ruff, mypy, quick test, docs)
3. How much time did you invest in total? (compare with the suggested 20-30 minutes)
4. What would you do differently in the capstone project?

Connection to the Project

From the exercise to the capstone project

This exercise worked with isolated snippets. The capstone project (module 8) has a complete codebase where:

The key difference: in the project, a hallucination in models.py can cause unexpected behavior in routes.py that shows up in tests.py. Your ability to trace the root cause across files is what the project evaluates.

What you take from module 3

After this complete module, your toolkit is:

• ✅ A taxonomy of 4 types of hallucinations
• ✅ Detection techniques for imports and APIs
• ✅ Warning signs for fabricated logic
• ✅ 4 verification layers (ruff → mypy → tests → docs)
• ✅ Practice with 5 snippets of increasing difficulty
• ✅ The instinct of "if I haven't used it before, I verify"

Troubleshooting

Problem 1: "I didn't find the hallucination and I feel like I failed"

Cause: The subtle hallucinations are designed to be hard. That's the point.

Solution: Don't judge yourself by whether you found it "at a glance." Judge yourself by whether you found it with your complete toolkit (tools + eyes + tests). In real life, you have access to all the tools — the exercise trains you to use them.

Problem 2: "I found other issues besides the planted hallucination"

Cause: The snippets may have minor issues besides the main hallucination.

Solution: Finding additional issues is excellent — it shows that your code review eye is active. In a real code review, you'd document all the issues, not just the hallucinations. The difference: the additional issues are improvements (design, performance), the hallucinations are factual errors.

Problem 3: "It took me much longer than 30 minutes"

Cause: Logic hallucinations require deep thinking.

Solution: The time decreases with practice. The first time it can take 45-60 minutes. The second time you see a similar pattern, you'll detect it in minutes. The key: building a repertoire of hallucination patterns you recognize quickly.

Summary

In this capsule:

• You applied everything you learned in the module to 5 real snippets with hidden hallucinations
• You worked with progressive difficulty: 1 easy, 2 medium, 2 subtle
• The 5 snippets covered: security (exposed secrets), header logic (sliding window vs fixed), regex injection (user input in re.compile), falsy values (timedelta(0) as False), and mathematical calculation (population vs sample variance)
• Each hallucination required a different combination of tools and knowledge
• The 4/5 benchmark confirms that you can detect hallucinations in AI code in real life

Next module: Code Review of AI Output — from detecting hallucinations to a complete professional code review process.

Additional resources

1. OWASP — Regular Expression DoS - ReDoS attacks and prevention
2. Python statistics module - Reference for correct statistical functions
3. Python secrets module - Secure generation of tokens and random values
4. JWT Best Practices (RFC 8725) - Recommended practices for JWT
5. FastAPI Security Documentation - Correct authentication in FastAPI
6. Real Python — Python Pitfalls - Common Python gotchas that LLMs reproduce

Debugging & Code Review with Claude Code — Module 3, Capsule 06 Claude Code Agentic Development Path — Guide #6 of 11