Module 3: Detecting Hallucinations in Code

Hallucinations in Logic

Hallucinations in Logic

Capsule overview

A fake import fails at import. An incorrect parameter fails (sometimes) at runtime. But a hallucination in logic — code that compiles, runs, produces a result, and that result is incorrect — is the most dangerous type because it can reach production without anyone noticing.

In the previous capsule you worked with imports and APIs: errors that automated tools can catch. In this capsule you enter the territory where your human eye and your domain knowledge are the only defense. There's no linter that detects that a calculate_percentile() function implements the wrong formula. There's no type checker that knows validate_email() should do more than check for @.

This is the level of detection that separates the developer who uses AI from the developer who masters AI.


What a Logic Hallucination Is

Precise definition

A logic hallucination is code that:

  1. ✅ Compiles without errors
  2. ✅ Runs without exceptions
  3. ✅ Produces a result (it's not void nor does it crash)
  4. ❌ The result doesn't correspond to what the function name, the docstring, or the declared intent promise

The key difference from a bug: a bug is an implementation error where the developer (human or AI) tried to do something and got it wrong. A logic hallucination is an implementation the LLM fabricated — it was never correct, it was never verified against a reference.

Why they're the most dangerous

Fake import:
  Detection time: immediate (at import)
  Impact: none (doesn't reach production)
  Cost: 0

Incorrect parameter:
  Detection time: minutes to hours
  Impact: low-medium (unexpected behavior)
  Cost: hours of debugging

Fabricated logic:
  Detection time: days to weeks to NEVER
  Impact: high (incorrect results in production)
  Cost: from hours of debugging to business damage

Fabricated logic can be in production for weeks producing incorrect results without anyone noticing — because the code "works" and produces results that look reasonable.


The 6 Patterns of Fabricated Logic

Pattern 1: Superficial validation

The LLM generates a validation function that checks the minimum and presents it as complete validation.

import re

def validate_email(email: str) -> bool:
    """
    Validates an email address according to RFC 5322.
    Returns True if the email is valid, False otherwise.
    """
    if not email or not isinstance(email, str):
        return False
    
    parts = email.split("@")
    if len(parts) != 2:
        return False
    
    local, domain = parts
    if not local or not domain:
        return False
    
    if "." not in domain:
        return False
    
    return True

What it claims to do: Validate an email according to RFC 5322. What it really does: Checks that it has @, something before, something after, and a . in the domain.

What it accepts and shouldn't:

validate_email("a@b.c")              # True ← 1-letter TLD
validate_email("us er@domain.com")   # True ← Space in the local part
validate_email("user@dom ain.com")   # True ← Space in the domain  
validate_email(".user@domain.com")   # True ← Dot at the start
validate_email("user@.domain.com")   # True ← Dot after @
validate_email("a" * 500 + "@b.com") # True ← 500-char local part

The correct solution:

from email_validator import validate_email as real_validate, EmailNotValidError

def validate_email(email: str) -> bool:
    """Validates an email address using the email-validator library."""
    try:
        real_validate(email, check_deliverability=False)
        return True
    except EmailNotValidError:
        return False

Lesson: When the validation is complex (email, URL, credit card, phone), distrust manual implementations. Specialized libraries exist because correct validation is hard.

Pattern 2: Incorrect algorithm with a correct name

The LLM generates a mathematical or algorithmic function with the correct name but the implementation is an incorrect approximation.

from typing import List

def calculate_median(data: List[float]) -> float:
    """
    Calculates the median of a list of numbers.
    For even-length lists, returns the average of the two middle values.
    """
    if not data:
        raise ValueError("Cannot calculate median of empty list")
    
    sorted_data = sorted(data)
    n = len(sorted_data)
    mid = n // 2
    
    if n % 2 == 0:
        return (sorted_data[mid] + sorted_data[mid + 1]) / 2
    else:
        return sorted_data[mid]

The error: For even-length lists, the code uses sorted_data[mid] and sorted_data[mid + 1], but it should be sorted_data[mid - 1] and sorted_data[mid].

data = [1, 2, 3, 4]  # n=4, mid=2
# The code calculates: (sorted_data[2] + sorted_data[3]) / 2 = (3 + 4) / 2 = 3.5
# The correct thing:   (sorted_data[1] + sorted_data[2]) / 2 = (2 + 3) / 2 = 2.5

Why it's subtle: For odd-length lists, it works perfectly. For even-length lists, the error is off by one index — the result is close but not correct. With large datasets, the difference can be so small that nobody notices.

How to verify:

import statistics

def test_median():
    assert calculate_median([1, 2, 3]) == statistics.median([1, 2, 3])         # OK
    assert calculate_median([1, 2, 3, 4]) == statistics.median([1, 2, 3, 4])   # FAIL
    assert calculate_median([1]) == statistics.median([1])                       # OK

Pattern 3: Security that gives false confidence

The LLM generates security code that looks robust but has fundamental vulnerabilities.

import hashlib
import os

def hash_password(password: str) -> str:
    """
    Securely hashes a password using SHA-256 with a random salt.
    Returns the salt and hash concatenated.
    """
    salt = os.urandom(16).hex()
    salted_password = salt + password
    password_hash = hashlib.sha256(salted_password.encode()).hexdigest()
    return f"{salt}:{password_hash}"

def verify_password(password: str, stored_hash: str) -> bool:
    """Verifies a password against a stored hash."""
    salt, password_hash = stored_hash.split(":")
    salted_password = salt + password
    return hashlib.sha256(salted_password.encode()).hexdigest() == password_hash

What it seems to be: Secure hashing with a random salt. What it really is: SHA-256 is a general-purpose hash, not a password hash.

The problems:

  1. SHA-256 is too fast — it allows ~10 billion hashes/second on a modern GPU
  2. It has no key stretching (bcrypt does 2^12 iterations by default)
  3. It has no protection against timing attacks in the comparison (== vs hmac.compare_digest)
  4. An attacker with access to the database can brute-force passwords efficiently

The correct solution:

from passlib.hash import bcrypt

def hash_password(password: str) -> str:
    """Securely hashes a password using bcrypt."""
    return bcrypt.hash(password)

def verify_password(password: str, stored_hash: str) -> bool:
    """Verifies a password against its bcrypt hash."""
    return bcrypt.verify(password, stored_hash)

Lesson: For security functions (hashing, encryption, token generation, sanitization), always use specialized libraries. Never trust manual implementations, even if they seem reasonable.

Pattern 4: Incomplete sanitization

The LLM generates a sanitization function that covers the obvious cases but leaves attack vectors open.

import re

def sanitize_html(text: str) -> str:
    """
    Sanitizes HTML input to prevent XSS attacks.
    Removes all script tags and event handlers.
    """
    sanitized = re.sub(r"<script[^>]*>.*?</script>", "", text, flags=re.DOTALL | re.IGNORECASE)
    
    sanitized = re.sub(r'\s+on\w+\s*=\s*"[^"]*"', "", sanitized, flags=re.IGNORECASE)
    sanitized = re.sub(r"\s+on\w+\s*=\s*'[^']*'", "", sanitized, flags=re.IGNORECASE)
    
    sanitized = re.sub(r"javascript:", "", sanitized, flags=re.IGNORECASE)
    
    return sanitized

What it removes:

<script>alert('xss')</script>                    <!-- ✅ Removed -->
<div onclick="alert('xss')">Click</div>          <!-- ✅ Removed -->
<a href="javascript:alert('xss')">Link</a>       <!-- ✅ Removed -->

What it does NOT remove:

<img src=x onerror=alert('xss')>                  <!-- ❌ No quotes in the attribute -->
<svg onload=alert('xss')>                          <!-- ❌ No quotes in the attribute -->
<div style="background:url(javascript:alert(1))">  <!-- ❌ CSS injection -->
<iframe src="data:text/html,<script>alert(1)</script>"> <!-- ❌ data: URI -->
<a href="&#106;avascript:alert(1)">Link</a>        <!-- ❌ HTML entities -->

The correct solution:

import bleach

def sanitize_html(text: str) -> str:
    """Sanitizes HTML using the bleach library."""
    return bleach.clean(
        text,
        tags=["p", "br", "b", "i", "u", "a", "ul", "ol", "li"],
        attributes={"a": ["href"]},
        protocols=["https"],
        strip=True,
    )

Pattern 5: Conversion with loss of precision

The LLM generates conversion code that loses precision in non-obvious ways.

from datetime import datetime, timezone

def unix_timestamp_to_datetime(timestamp: float) -> datetime:
    """
    Converts a Unix timestamp to a timezone-aware datetime object.
    Handles both seconds and milliseconds timestamps.
    """
    if timestamp > 1e12:
        timestamp = timestamp / 1000
    
    return datetime.fromtimestamp(timestamp, tz=timezone.utc)

def calculate_duration_hours(start: datetime, end: datetime) -> float:
    """
    Calculates the duration between two datetimes in hours.
    Returns a float with the fractional hours.
    """
    delta = end - start
    return delta.seconds / 3600

The error in calculate_duration_hours: It uses delta.seconds instead of delta.total_seconds().

from datetime import timedelta

delta = timedelta(days=1, hours=2, minutes=30)

delta.seconds       # → 9000 (only the seconds part, ignores days!)
delta.total_seconds() # → 95400.0 (correct total including days)

delta.seconds only returns the seconds within the current day, ignoring full days. If start and end differ by more than 24 hours, the result is completely incorrect.

The fix:

def calculate_duration_hours(start: datetime, end: datetime) -> float:
    """Calculates the duration between two datetimes in hours."""
    delta = end - start
    return delta.total_seconds() / 3600

Lesson: The timedelta methods are confusing. delta.seconds isn't "the total seconds" — it's a detail of Python's API that LLMs and many developers confuse.

Pattern 6: Incorrect concurrency

The LLM generates async code that looks correct but has race conditions or potential deadlocks.

import asyncio
from typing import Dict, Any

class AsyncCache:
    """Thread-safe async cache with TTL support."""
    
    def __init__(self):
        self._cache: Dict[str, Any] = {}
        self._lock = asyncio.Lock()
    
    async def get(self, key: str) -> Any:
        """Gets a value from cache. Returns None if not found."""
        return self._cache.get(key)
    
    async def set(self, key: str, value: Any) -> None:
        """Sets a value in the cache."""
        async with self._lock:
            self._cache[key] = value
    
    async def get_or_set(self, key: str, factory) -> Any:
        """Gets a value from cache, or creates it using factory if not found."""
        value = await self.get(key)
        if value is None:
            value = await factory()
            await self.set(key, value)
        return value

The errors:

  1. get() doesn't use the lock: If another task is modifying _cache while get() reads, there can be inconsistencies. It should use the lock too.

  2. get_or_set() has a race condition: Between get() and set(), another task may have done set() with the same key. The factory runs multiple times unnecessarily. The correct pattern is check-lock-check (double-checked locking).

The fix:

async def get(self, key: str) -> Any:
    async with self._lock:
        return self._cache.get(key)

async def get_or_set(self, key: str, factory) -> Any:
    async with self._lock:
        value = self._cache.get(key)
        if value is None:
            value = await factory()
            self._cache[key] = value
        return value

How to Detect Logic Hallucinations

The fundamental principle: distrust manual implementations

The pattern that connects all the previous examples:

Email validation     → Library: email-validator
Password hashing     → Library: passlib / bcrypt
HTML sanitization    → Library: bleach
SQL sanitization     → Parameterized queries (no sanitization)
Statistical calcs    → Library: statistics / numpy
Time calculations    → datetime methods (total_seconds, not seconds)
Concurrency          → Established patterns with complete locks

Rule: When an LLM implements something from scratch that a library already does, be suspicious. Libraries exist because the correct implementation is hard. If the LLM avoided the library and implemented it manually, ask yourself why — and then verify the implementation.

The 5 detection questions

When you review an AI-generated function, ask yourself these questions:

1. Is there a standard library for this?
   If yes → why doesn't the code use it?
   
2. Does the docstring promise more than the implementation does?
   If yes → The implementation is probably incomplete
   
3. Does the function handle edge cases?
   Test: None, empty, negative, very large, special characters
   
4. Is the result verifiable against a reference?
   For calculations: use numpy/statistics as a reference
   For validation: use the standard library as a reference
   
5. Does the function touch security?
   If yes → NEVER trust manual implementations

Technique: the 3-minute quick test

For any suspicious function, write 3 tests in 3 minutes:

def verify_function(func, test_cases):
    """Quick verification of a function against test cases."""
    for inputs, expected in test_cases:
        result = func(*inputs) if isinstance(inputs, tuple) else func(inputs)
        status = "✅" if result == expected else "❌"
        print(f"{status} func({inputs}) = {result}, expected {expected}")
# For validate_email:
verify_function(validate_email, [
    ("user@domain.com", True),     # normal case
    ("@domain.com", False),        # no local part
    ("user@.com", False),          # invalid domain
    ("us er@domain.com", False),   # space
    ("", False),                   # empty
])

# For calculate_median:
import statistics
data_sets = [[1,2,3], [1,2,3,4], [5], [1,1,1,1]]
for data in data_sets:
    expected = statistics.median(data)
    result = calculate_median(data)
    status = "✅" if result == expected else "❌"
    print(f"{status} median({data}) = {result}, expected {expected}")

If any test fails, the function has a logic hallucination. If they all pass, it doesn't mean it's correct — but the most common edge cases are covered.


High-Risk Domains

Where to look first

You can't review every function in detail. Prioritize the logic review in these domains:

Critical Priority:

  • ✅ Authentication and authorization (login, tokens, permissions)
  • ✅ Hashing and encryption (passwords, secrets, PII)
  • ✅ Input sanitization (SQL, HTML, command injection)
  • ✅ Financial data validation (calculations, conversions)

High Priority:

  • ✅ Data validation (email, phone, credit card)
  • ✅ Mathematical/statistical calculations (percentiles, averages, aggregations)
  • ✅ Date and time handling (time zones, durations, conversions)
  • ✅ Concurrency and async operations (locks, race conditions)

Medium Priority:

  • ⚠️ Data formatting and serialization
  • ⚠️ Pagination and filtering
  • ⚠️ Data transformations

Low Priority:

  • ❌ CRUD boilerplate (generally correct)
  • ❌ Configuration and setup
  • ❌ Imports and model definitions

The security rule: never manual implementations

For security functions, the rule is absolute:

NEVER accept manual implementations of:
├── Password hashing → use bcrypt / argon2
├── Token generation → use the secrets module
├── Encryption → use the cryptography library  
├── SQL queries → use parameterized queries
├── HTML sanitization → use bleach
├── CSRF tokens → use the framework (FastAPI/Django)
├── Session management → use the framework
└── Input validation → use specialized libraries

If Claude Code generates a manual implementation of 
any of these → REJECT automatically.
It doesn't matter if the code "looks good."

Logic Hallucinations in FastAPI

Framework-specific examples

Since the capstone project uses FastAPI, these are the most common logic hallucination patterns in this framework:

Pagination with off-by-one:

@app.get("/items")
async def list_items(page: int = 1, size: int = 20):
    """Returns paginated items."""
    # ❌ Hallucination: off-by-one in the calculation
    start = page * size
    end = start + size
    items = all_items[start:end]
    # For page=1, size=20: returns items[20:40] — skips the first 20
    
    # ✅ Correct:
    start = (page - 1) * size
    end = start + size
    items = all_items[start:end]
    # For page=1, size=20: returns items[0:20] — correct
    
    return {"items": items, "total": len(all_items), "page": page}

Inverted filter:

@app.get("/tasks")
async def get_active_tasks(include_completed: bool = False):
    """Returns active tasks. Optionally includes completed ones."""
    tasks = get_all_tasks()
    
    if include_completed:
        # ❌ Hallucination: filters the opposite
        return [t for t in tasks if t.status != "completed"]
    
    # ✅ Correct:
    if not include_completed:
        return [t for t in tasks if t.status != "completed"]
    
    return tasks

Dependency injection with the wrong scope:

from fastapi import FastAPI, Depends

app = FastAPI()

# ❌ Hallucination: the connection is created once and reused
# instead of being created per request
db_connection = create_db_connection()  

async def get_db():
    """Provides a database connection."""
    return db_connection  # ← Same object for all requests

# ✅ Correct: create and close the connection per request
async def get_db():
    """Provides a database session per request."""
    db = create_db_session()
    try:
        yield db
    finally:
        db.close()

Connection to the Project

What to look for in the capstone project

In the codebase of the capstone project (module 8), there's at least 1 logic hallucination planted. The type of hallucination you might find:

  • A validation function that doesn't validate correctly
  • A calculation that uses the wrong operator or method
  • A filter that includes instead of excludes (or vice versa)
  • A security function that isn't really secure

Your process for finding it:

  1. Identify functions with names that promise something specific (validate, calculate, sanitize, hash)
  2. Read the implementation — does it really do what it promises?
  3. Write quick tests with edge cases
  4. If there's a standard library for that function, compare against it

From analysis to action

When you find a logic hallucination in the project:

  1. Document: what it claims to do vs what it really does
  2. Classify: the severity of the impact (Critical if it's security, High if it's business logic)
  3. Fix: ideally using the standard library, not patching the manual implementation

Troubleshooting

Problem 1: "I can't distinguish between a logic hallucination and a bug"

Cause: The distinction is subtle and sometimes academic. What matters is detecting the error, not classifying it perfectly.

Solution: If the name/docstring says one thing and the implementation does another, treat it as a hallucination. If the implementation tries to do the right thing but has a bug, treat it as a bug. In both cases, the action is the same: fix it. The classification matters for understanding the pattern (LLMs tend toward certain types of hallucinations) but doesn't change the action.

Problem 2: "I don't have enough domain knowledge to know if the implementation is correct"

Cause: You can't be an expert in every domain. That's normal.

Solution: Use these strategies:

  1. Look for a library: If there's a library for that, compare against it
  2. Look at the official documentation: For standard Python functions, the documentation has examples
  3. Quick test with known values: For calculations, use a calculator or Wolfram Alpha
  4. Ask Claude Code: "Is this implementation of X correct?" — but verify the answer with tests

Problem 3: "The quick test passes but I'm not sure the function is correct"

Cause: Your tests cover the happy paths but not the edge cases.

Solution: Add these edge cases to your quick test:

edge_cases = [
    None,              # null
    "",                # empty string
    [],                # empty list
    0,                 # zero
    -1,                # negative
    float('inf'),      # infinity
    float('nan'),      # NaN
    " ",               # only spaces
    "a" * 10000,       # very long string
    [1],               # single-element list
    [1, 1, 1, 1],      # all equal
]

If the function passes all the edge cases relevant to its domain, you can have greater confidence.

Problem 4: "Reviewing the logic of every function takes too long"

Cause: You don't need to review every function. Prioritize by risk domain.

Solution: Follow the prioritization in the "High-Risk Domains" section. In a typical code review:

  • Security functions: always review (5-10 minutes per function)
  • Validation/calculation functions: review if they don't use a standard library (3-5 minutes)
  • CRUD functions: review the logic quickly (1-2 minutes)
  • Boilerplate: skip (0 minutes)

Problem 5: "Claude Code generated a manual implementation of something that has a library. Is it always bad?"

Cause: Sometimes there are valid reasons not to use a library (minimizing dependencies, specific requirements).

Solution: If there's a valid reason for the manual implementation, verify it exhaustively. If there's no clear reason, replace it with the library. The rule: a manual implementation needs justification. If there's no justification, it's probably a hallucination where the LLM didn't "remember" that the library exists.


Exercises

Exercise 1: Detect superficial validation (Easy)

This function claims to validate URLs. Is it correct?

from urllib.parse import urlparse

def validate_url(url: str) -> bool:
    """
    Validates that a URL is properly formatted and uses 
    a safe protocol (http or https).
    """
    try:
        result = urlparse(url)
        return result.scheme in ("http", "https") and bool(result.netloc)
    except Exception:
        return False
See solution

This implementation is reasonably correct for a basic validation. Unlike validate_email() which only checks for @, this function:

  • ✅ Uses urlparse from the standard library (doesn't implement manual parsing)
  • ✅ Verifies that the scheme is http or https
  • ✅ Verifies that there's a netloc (domain)
  • ✅ Handles exceptions

However, it has limitations:

validate_url("http://localhost")     # True — is this desired?
validate_url("http://192.168.1.1")   # True — is a private IP valid?
validate_url("https://example..com") # True — double dot in the domain

Verdict: It's not a hallucination — it's a correct implementation for the general case. The limitations are edge cases that depend on the usage context. For most applications, this validation is enough.

Lesson: Not everything AI generates is incorrect. Knowing when code is "good enough" is also an important skill.

Exercise 2: Find the calculation error (Medium)

This function calculates a progressive discount. Is the calculation correct?

def calculate_discount(subtotal: float, coupon_percent: float = 0) -> dict:
    """
    Calculates the final price with progressive discount:
    - Orders >= $100: 5% discount
    - Orders >= $500: 10% discount  
    - Orders >= $1000: 15% discount
    Plus any coupon discount applied AFTER the progressive discount.
    """
    if subtotal < 100:
        progressive_discount = 0
    elif subtotal < 500:
        progressive_discount = 5
    elif subtotal < 1000:
        progressive_discount = 10
    else:
        progressive_discount = 15
    
    after_progressive = subtotal * (1 - progressive_discount / 100)
    
    coupon_discount_amount = subtotal * (coupon_percent / 100)
    
    final_price = after_progressive - coupon_discount_amount
    
    return {
        "subtotal": subtotal,
        "progressive_discount_percent": progressive_discount,
        "coupon_discount_percent": coupon_percent,
        "final_price": max(final_price, 0),
    }
See solution

There's a logic hallucination in how the coupon is applied:

The docstring says: "coupon discount applied AFTER the progressive discount." But the implementation calculates the coupon on the original subtotal, not on the price after the progressive discount.

# What it claims to do (coupon on the discounted price):
after_progressive = 1000 * (1 - 15/100) = 850
coupon_10 = 850 * (10/100) = 85
final = 850 - 85 = 765

# What it really does (coupon on the original subtotal):
after_progressive = 1000 * (1 - 15/100) = 850
coupon_10 = 1000 * (10/100) = 100  # ← On the original subtotal!
final = 850 - 100 = 750            # ← $15 less than the correct amount

The fix:

coupon_discount_amount = after_progressive * (coupon_percent / 100)

Impact: For a business, this means giving more discount than intended. With thousands of transactions, the difference accumulates.

Exercise 3: Detect fake security (Medium)

Is this token-generation code secure?

import random
import string
import time

def generate_reset_token(user_id: str) -> str:
    """
    Generates a secure password reset token.
    Token is unique per user and time-based for expiration.
    """
    timestamp = str(int(time.time()))
    random_part = ''.join(
        random.choices(string.ascii_letters + string.digits, k=32)
    )
    token = f"{user_id}_{timestamp}_{random_part}"
    return token

def verify_reset_token(token: str, max_age_seconds: int = 3600) -> str:
    """
    Verifies a password reset token and returns the user_id.
    Returns None if token is expired.
    """
    parts = token.split("_")
    if len(parts) != 3:
        return None
    
    user_id, timestamp, random_part = parts
    
    token_age = int(time.time()) - int(timestamp)
    if token_age > max_age_seconds:
        return None
    
    return user_id
See solution

This code has multiple security logic hallucinations:

  1. random.choices isn't cryptographically secure. The random module uses Mersenne Twister, which is predictable if the state is known. For security tokens, you should use secrets:
import secrets
random_part = secrets.token_urlsafe(32)
  1. The user_id is in the token in plaintext. An attacker can see the user_id and generate tokens for any user if they know the pattern.

  2. There's no signature or integrity verification. Anyone who knows the format can forge a valid token. They only need: a valid user_id, a recent timestamp, and 32 random characters. The verify_reset_token function doesn't verify that the token was generated by the system — it only verifies the format and expiration.

  3. The token can be tampered with. An attacker can take the user_id from a stolen token and change the timestamp to "renew" the token.

The correct solution:

import secrets
import hmac
import hashlib
import time

SECRET_KEY = "your-secret-key-from-env"

def generate_reset_token(user_id: str) -> str:
    timestamp = str(int(time.time()))
    random_part = secrets.token_urlsafe(32)
    payload = f"{user_id}.{timestamp}.{random_part}"
    signature = hmac.new(
        SECRET_KEY.encode(), payload.encode(), hashlib.sha256
    ).hexdigest()
    return f"{payload}.{signature}"

# Or better yet, use itsdangerous or PyJWT:
from itsdangerous import URLSafeTimedSerializer
serializer = URLSafeTimedSerializer(SECRET_KEY)
token = serializer.dumps(user_id, salt="password-reset")

Exercise 4: Verify with a quick test (Medium-Hard)

Write 5 test cases that would expose the hallucination in this function:

def is_palindrome(text: str) -> bool:
    """
    Checks if a string is a palindrome.
    Ignores case and non-alphanumeric characters.
    """
    cleaned = ''.join(c.lower() for c in text if c.isalnum())
    return cleaned == cleaned[::-1]
See solution

Surprise: this implementation is correct. The 5 test cases confirm it:

assert is_palindrome("racecar") == True         # ✅ Simple palindrome
assert is_palindrome("A man, a plan, a canal: Panama") == True  # ✅ With punctuation
assert is_palindrome("hello") == False           # ✅ Not a palindrome
assert is_palindrome("") == True                 # ✅ Empty string (convention)
assert is_palindrome("Aa") == True               # ✅ Case insensitive

The point of this exercise: The quick test doesn't only detect hallucinations — it also confirms when the code is correct. Knowing to stop and accept that the code is good is as important as detecting errors. If you spend 20 minutes looking for an error that doesn't exist, it's wasted time.

Exercise 5: Analyze complete logic (Hard)

This code implements rate limiting. Find all the logic hallucinations:

import time
from collections import defaultdict
from typing import Optional

class RateLimiter:
    """
    Token bucket rate limiter.
    Allows 'rate' requests per 'period' seconds.
    """
    
    def __init__(self, rate: int = 10, period: float = 60.0):
        self.rate = rate
        self.period = period
        self.tokens = defaultdict(lambda: rate)
        self.last_update = defaultdict(float)
    
    def is_allowed(self, client_id: str) -> bool:
        """
        Check if a request is allowed for the given client.
        Uses token bucket algorithm.
        """
        now = time.time()
        elapsed = now - self.last_update[client_id]
        
        self.tokens[client_id] += elapsed * (self.rate / self.period)
        
        if self.tokens[client_id] > self.rate:
            self.tokens[client_id] = self.rate
        
        self.last_update[client_id] = now
        
        if self.tokens[client_id] >= 1:
            self.tokens[client_id] -= 1
            return True
        
        return False
    
    def get_wait_time(self, client_id: str) -> Optional[float]:
        """Returns seconds to wait before next allowed request."""
        if self.tokens[client_id] >= 1:
            return 0.0
        
        tokens_needed = 1 - self.tokens[client_id]
        return tokens_needed / (self.rate / self.period)
See solution

This is an interesting case: the token bucket implementation is fundamentally correct. The algorithm:

  1. ✅ Calculates tokens earned since the last update
  2. ✅ Caps the tokens at the maximum (rate)
  3. ✅ Consumes a token if available
  4. ✅ get_wait_time calculates the correct wait time

However, there are practical issues (not logic hallucinations, but design ones):

  1. ⚠️ Not thread-safe. If multiple requests from the same client arrive simultaneously, there's a race condition between reading and writing tokens. It needs a lock.

  2. ⚠️ Memory leak. tokens and last_update grow indefinitely — entries for inactive clients are never cleaned up.

  3. ⚠️ Not distributed. In a system with multiple workers/pods, each instance has its own rate limiter. It should use Redis for shared state.

Verdict: The algorithm's logic is correct. The issues are design ones for production, not hallucinations. This is a good example of code that "works" but needs refinement to be production-ready.

Lesson: Not every issue is a hallucination. Distinguishing between "the logic is wrong" (hallucination) and "the design isn't production-ready" (design improvement) is important for prioritizing your review.


Summary

In this capsule you learned:

  • Logic hallucinations are the most dangerous because the code compiles, runs, and produces results — just incorrect ones
  • The 6 main patterns: superficial validation, incorrect algorithm, fake security, incomplete sanitization, loss of precision, incorrect concurrency
  • The manual implementations rule: if there's a library for that, distrust the manual code
  • The 5 detection questions: standard library, docstring vs implementation, edge cases, verifiable reference, touches security
  • The 3-minute quick test: 5 test cases with edge cases can expose most hallucinations
  • The high-risk domains: security, finance, validation, calculations, dates, concurrency
  • Not everything is a hallucination: knowing when code is correct is just as important as detecting errors

Next capsule: Detection Tools — type checkers, linters, quick tests, and official documentation as a safety net.


Additional resources

  1. OWASP — Password Storage Cheat Sheet - Why SHA-256 isn't for passwords
  2. OWASP — Input Validation Cheat Sheet - Correct validation vs incorrect sanitization
  3. Python secrets module - Cryptographically secure token generation
  4. email-validator library - Correct email validation in Python
  5. bleach library - Correct HTML sanitization
  6. Real Python — Common Python Gotchas - timedelta.seconds vs total_seconds and other gotchas

Debugging & Code Review with Claude Code — Module 3, Capsule 04 Claude Code Agentic Development Path — Guide #6 of 11