Trust Calibration by Type of Task

Capsule overview

You already know how to supervise (Managing an Intern) and when to pause (Circuit Breaker). Now you need the most actionable tool of the three: how much to trust. Trust Calibration turns "I think it's fine" into "for this type of task, my trust is 40%, so I review logic and edge cases." It's the difference between intuition and process.

In module 1 you saw a preliminary table of trust by type of task. This capsule expands it significantly: you're going to build a detailed table with 15 types of task, understand the factors that raise and lower your trust, and learn to update your calibration with experience. Trust Calibration is the model you can apply literally tomorrow at work.

The key: Trust Calibration isn't an exact number. It's a range that guides your behavior. "70% trust" doesn't mean that 7 out of 10 times the code is correct. It means your level of review corresponds to a moderate-risk task where a focused review is enough.

---

Trust Calibration as a Structured Process

It's not intuition — it's process

Most developers calibrate by intuition: "it looks good" or "something's off." Intuition has two problems:

Problem 1: Intuition is inconsistent
├── Monday morning (rested): "I'm going to review everything"
├── Friday at 6 PM (tired): "Looks good, I accept"
└── The code's risk didn't change. Your calibration did.

Problem 2: Intuition isn't communicable
├── "Why did you accept this code without review?"
├── "I don't know, it looked good"
└── You can't teach or defend "it looked good"

Trust Calibration replaces intuition with process:

The Trust Calibration process:
1. Identify the type of task
2. Consult your base trust table
3. Adjust for contextual factors
4. Define review depth based on the result
5. Document and update with experience

The mental model

Think of Trust Calibration as a trust thermometer:

100% ─── Total trust (doesn't exist for AI code)
 90% ─── Visual review (boilerplate, formatting)
 80% ─── Quick review (config, docs)
 70% ─── Focused review (CRUD, utilities)
 60% ─── Detailed review (medium logic)
 50% ─── Careful review (tests, queries)
 40% ─── Exhaustive review (business logic)
 30% ─── Line-by-line review (data pipelines)
 20% ─── Active distrust (regex, concurrency)
 10% ─── Verification against documentation (auth, crypto)
  0% ─── Don't trust it, write it yourself (doesn't exist either)

Your position on the thermometer determines your behavior: how much time you spend, how deeply you review, and what tools you use to verify.

---

The Trust Calibration Table

Complete table: 15 types of task

#	Type of task	Base trust	Review depth	What to verify
1	README / documentation	85-90%	Quick visual	That the information is correct and not misleading
2	Boilerplate (setup, initial config)	80-90%	Quick visual	Dependency versions, correct paths
3	Dockerfiles / CI config	75-85%	Quick	Base image, secrets not hardcoded, stages
4	Data models (Pydantic, ORM)	70-80%	Focused	Correct fields, appropriate types, validations
5	CRUD endpoints	60-70%	Focused	Validations, error handling, status codes
6	Utility functions	60-70%	Focused	Edge cases, naming, return types
7	Unit tests	50-60%	Detailed	That they test the right thing, not just that they pass
8	SQL / ORM queries	40-55%	Detailed	SQL injection, N+1, query logic, indexes
9	Error handling / logging	50-60%	Detailed	That it doesn't expose sensitive info, that it captures what's needed
10	Business logic	25-40%	Exhaustive	That it does what the business needs, correct values
11	External integrations (APIs)	30-45%	Exhaustive	Correct API, error handling, retry logic, timeouts
12	Data pipelines / ETL	25-40%	Exhaustive	Correct transformations, handling of null/invalid data
13	Regex patterns	15-25%	Line by line	Test with edge-case inputs, false positives/negatives
14	Auth / Security / Crypto	10-20%	Line by line + docs	Against official documentation, every security decision
15	Financial processing	10-20%	Line by line + tests	Every calculation, precision (Decimal), compliance

How to read the table

80-90% trust: "It's probably fine"
→ Visual review of 1-3 minutes
→ If something jumps out, go deeper
→ If not, accept

60-70% trust: "It's fine if the key parts are fine"
→ A 5-10 minute review focused on logic and edge cases
→ Not every line, but the main functions

40-55% trust: "I need to verify"
→ A 10-20 minute review
→ Verify logic, test with edge-case inputs
→ Compare against documentation if applicable

10-30% trust: "I assume there are problems until proven otherwise"
→ A 20-40 minute review
→ Read every line, verify against official docs
→ Write tests for each scenario
→ Ask yourself "how could this fail?"

---

Factors That Adjust Your Calibration

Factors that INCREASE trust

Your base trust rises when these conditions are present:

+10-15%: Existing tests that cover the scenario
  → If there are tests that verify the logic, your review
    can be less exhaustive. The tests do part of the
    verification work.

+5-10%: A domain you know well
  → If you know the domain, you detect errors faster.
    Your quick review is more effective than the exhaustive
    review of someone who doesn't know the domain.

+5-10%: A well-documented standard pattern
  → If the code follows a pattern you've seen 100 times
    (e.g., standard FastAPI CRUD), the probability of an error
    is lower.

+5%: Short and simple code (< 30 lines, linear flow)
  → Fewer lines = less surface area for errors.
    But don't confuse short with safe (3 lines of
    SQL injection are worse than 300 of CRUD).

+5-10%: Claude Code has a good track record on this task
  → If the last 5 times you asked for CRUD endpoints
    the result was correct, your calibration rises.

Factors that DECREASE trust

Your base trust drops when these conditions are present:

-10-20%: There are no tests
  → Without tests, you're the only line of defense.
    Your review must be more exhaustive.

-10-15%: A domain you don't know
  → If you don't know the domain, subtle errors can
    go unnoticed. A tax calculation function can
    "look good" but have the wrong percentages.

-10-15%: Code with mutable state or side effects
  → Side effects (modifies the DB, sends emails, calls APIs)
    are hard to verify visually. You need to
    run it or write tests.

-5-10%: The prompt was vague or ambiguous
  → If your prompt wasn't specific, Claude Code had
    to "guess" your intent. Higher probability that
    it guessed wrong.

-10-20%: Security or compliance involved
  → Anything that touches auth, encryption, PII,
    or regulations requires active distrust.

-5-10%: Complex code (multiple branches, async, concurrency)
  → High complexity = more surface area for errors.
    AI is especially unpredictable with concurrency.

-10-15%: Integration with an API that changed recently
  → Claude Code can generate code for previous
    versions of an API. If the API changed, the code
    can use endpoints that no longer exist.

Example: Adjusting calibration

Task: Generate a CRUD endpoint for products

Base trust (table): 65%

Adjustments:
  +10%: I know FastAPI well (familiar domain)
  +5%:  It's a standard pattern (simple CRUD)
  -10%: There are no tests yet
  -5%:  My prompt was generic ("generate CRUD for products")

Adjusted trust: 65 + 10 + 5 - 10 - 5 = 65%

→ Stayed similar. Focused review of 5-8 minutes.
→ Verify: validations, error handling, status codes.

Task: Generate rate limiting middleware

Base trust (table): 35% (business logic + security)

Adjustments:
  -15%: I don't know rate limiting algorithms well
  -10%: There are no tests
  -10%: It has security implications (DoS protection)
  +5%:  Relatively short code

Adjusted trust: 35 - 15 - 10 - 10 + 5 = 5%

→ Very low. Review every line.
→ Verify against rate limiting documentation.
→ Write tests before accepting.
→ Consider using a proven library instead of custom.

---

Trust Debt: When You Don't Calibrate

What Trust Debt is

Trust Debt is the concept that when you under-calibrate (trust too much), problems accumulate silently, like technical debt. You don't see the cost immediately, but it shows up eventually.

Trust Debt in action:

Week 1: You accept a search endpoint without reviewing the queries
  → Trust: "70% — it's CRUD"
  → Reality: unparameterized queries (SQL injection)

Week 2: You accept a filters endpoint without reviewing
  → It's based on the week 1 endpoint
  → Same vulnerability, now in 2 endpoints

Week 3: You accept an "advanced search" feature
  → It's based on the 2 previous endpoints
  → Now there are 3 endpoints with SQL injection

Week 6: Security audit
  → They find SQL injection in 3 endpoints
  → The fix requires changing 3 files + 2 of tests
  → 3 days of work + review + QA
  → The cost of "not reviewing queries" accumulated

How to avoid Trust Debt

1. Calibrate in the moment, not "later"
   → "I'll review it tomorrow" = Trust Debt
   → Review at the checkpoint or document that review is pending

2. Don't inherit calibration from others
   → If a coworker says "this is fine," verify it yourself
   → Your calibration is personal

3. Update your table with experience
   → If you discover that Claude Code consistently generates
     unparameterized queries, lower your trust in
     "SQL queries" from 45% to 30%

4. Pay off Trust Debt periodically
   → Each week, spend 30 minutes reviewing AI code
     you accepted quickly during the week
   → Look for the patterns your calibration didn't cover

---

Updating Your Calibration

Track record: how to improve your table

Your calibration table isn't static. It updates based on your experience:

Update process:

1. Generate code with Claude Code
2. Apply your current calibration
3. Review according to the trust level
4. Did you find problems your calibration didn't predict?

If you DID find unexpected problems:
  → Lower your base trust for that type of task
  → Add the type of problem to "what to verify"
  → Example: "CRUD always came out fine for me, but today I found
    that it didn't handle soft delete. I lower from 70% to 65% and add
    'verify delete behavior' to my checklist"

If you did NOT find problems:
  → Keep your calibration (don't raise it prematurely)
  → After 5-10 experiences with no problems, raise 5%
  → Example: "The last 8 times I generated Pydantic models,
    everything was correct. I raise from 75% to 80%"

Calibration log

Keep a simple log to track your calibration:

Date  | Type of task    | Trust used | Problems found       | Adjustment
------|-----------------|------------|----------------------|-------
03/10 | CRUD endpoint   | 70%        | Missing email validation | -5%
03/11 | Auth JWT        | 15%        | Token didn't expire  | Keep 15%
03/12 | Pydantic model  | 80%        | None                 | Keep 80%
03/13 | SQL query       | 45%        | N+1 in a join        | -5%
03/14 | Dockerfile      | 85%        | None                 | Keep 85%

After 20-30 entries, your calibration table reflects your real experience with Claude Code, not theoretical values.

---

Trust Calibration in Action

Complete example: Calibrating a new module

You ask Claude Code to generate a complete subscription-management module. It generates 4 files. You calibrate each one:

File 1: models.py

from pydantic import BaseModel, Field
from typing import Optional
from datetime import datetime
from enum import Enum
from decimal import Decimal

class SubscriptionPlan(str, Enum):
    FREE = "free"
    BASIC = "basic"
    PRO = "pro"
    ENTERPRISE = "enterprise"

class PlanPricing(BaseModel):
    plan: SubscriptionPlan
    monthly_price: Decimal
    annual_price: Decimal
    max_users: int
    features: list[str]

class SubscriptionCreate(BaseModel):
    plan: SubscriptionPlan
    billing_cycle: str = Field(..., pattern="^(monthly|annual)$")
    payment_method_id: str

class Subscription(BaseModel):
    id: str
    user_id: str
    plan: SubscriptionPlan
    billing_cycle: str
    status: str = "active"
    current_period_start: datetime
    current_period_end: datetime
    created_at: datetime

Calibration of models.py:
├── Type: Data models → Base trust: 75%
├── Adjustments:
│   ├── +5%: I know Pydantic well
│   ├── +5%: Standard pattern
│   └── No negative adjustments
├── Final trust: 85%
├── Depth: Quick review (2-3 min)
└── What I verify:
    ├── ✅ Uses Enum for plan → Correct
    ├── ✅ Uses Decimal for prices → Correct
    ├── ✅ billing_cycle validated with regex → Correct
    ├── ✅ Reasonable fields for a subscription
    └── → PASS

File 2: pricing_service.py

from decimal import Decimal
from models import SubscriptionPlan, PlanPricing

PRICING = {
    SubscriptionPlan.FREE: PlanPricing(
        plan=SubscriptionPlan.FREE,
        monthly_price=Decimal("0.00"),
        annual_price=Decimal("0.00"),
        max_users=1,
        features=["basic_access"],
    ),
    SubscriptionPlan.BASIC: PlanPricing(
        plan=SubscriptionPlan.BASIC,
        monthly_price=Decimal("9.99"),
        annual_price=Decimal("99.00"),
        max_users=5,
        features=["basic_access", "email_support", "api_access"],
    ),
    SubscriptionPlan.PRO: PlanPricing(
        plan=SubscriptionPlan.PRO,
        monthly_price=Decimal("29.99"),
        annual_price=Decimal("299.00"),
        max_users=25,
        features=["basic_access", "email_support", "api_access",
                   "priority_support", "advanced_analytics"],
    ),
    SubscriptionPlan.ENTERPRISE: PlanPricing(
        plan=SubscriptionPlan.ENTERPRISE,
        monthly_price=Decimal("99.99"),
        annual_price=Decimal("999.00"),
        max_users=100,
        features=["basic_access", "email_support", "api_access",
                   "priority_support", "advanced_analytics",
                   "custom_integrations", "sla"],
    ),
}

def get_price(plan: SubscriptionPlan, billing_cycle: str) -> Decimal:
    pricing = PRICING[plan]
    if billing_cycle == "annual":
        return pricing.annual_price
    return pricing.monthly_price

def calculate_proration(
    old_plan: SubscriptionPlan,
    new_plan: SubscriptionPlan,
    days_remaining: int,
    billing_cycle: str,
) -> Decimal:
    old_daily = get_price(old_plan, billing_cycle) / Decimal("30")
    new_daily = get_price(new_plan, billing_cycle) / Decimal("30")
    difference = new_daily - old_daily
    return (difference * Decimal(str(days_remaining))).quantize(Decimal("0.01"))

Calibration of pricing_service.py:
├── Type: Financial business logic → Base trust: 25%
├── Adjustments:
│   ├── -10%: The prices are made up (not MY business's)
│   ├── -5%:  Proration calculations are complex
│   ├── +5%:  Uses Decimal correctly
│   └── -10%: There are no tests
├── Final trust: 5%
├── Depth: Line-by-line review (15-20 min)
└── What I verify:
    ├── ⚠️ Prices are placeholders — I MUST replace them with the real ones
    ├── ⚠️ Proration assumes 30 days per month (not correct for all months)
    ├── ⚠️ What happens with a downgrade? difference would be negative → refund?
    ├── ⚠️ features are strings — they should be an Enum for validation
    ├── ⚠️ Doesn't handle annual billing with proration
    │   (does it divide by 30 or by 365?)
    └── → NEEDS EDIT — the financial calculations need correction

File 3: subscription_service.py

from datetime import datetime, timedelta
from typing import Optional
from models import Subscription, SubscriptionCreate, SubscriptionPlan
from pricing_service import get_price, calculate_proration
import uuid

subscriptions_db: dict = {}

class SubscriptionService:
    def create(self, user_id: str, data: SubscriptionCreate) -> Subscription:
        if any(
            s.user_id == user_id and s.status == "active"
            for s in subscriptions_db.values()
        ):
            raise ValueError("User already has an active subscription")

        now = datetime.utcnow()
        period_days = 365 if data.billing_cycle == "annual" else 30

        sub = Subscription(
            id=str(uuid.uuid4()),
            user_id=user_id,
            plan=data.plan,
            billing_cycle=data.billing_cycle,
            current_period_start=now,
            current_period_end=now + timedelta(days=period_days),
            created_at=now,
        )
        subscriptions_db[sub.id] = sub
        return sub

    def cancel(self, subscription_id: str) -> Subscription:
        sub = subscriptions_db.get(subscription_id)
        if not sub:
            raise ValueError("Subscription not found")
        sub.status = "cancelled"
        return sub

    def upgrade(
        self, subscription_id: str, new_plan: SubscriptionPlan
    ) -> dict:
        sub = subscriptions_db.get(subscription_id)
        if not sub:
            raise ValueError("Subscription not found")

        days_remaining = (sub.current_period_end - datetime.utcnow()).days
        proration = calculate_proration(
            sub.plan, new_plan, days_remaining, sub.billing_cycle,
        )

        sub.plan = new_plan
        return {
            "subscription": sub,
            "proration_charge": float(proration),
        }

Calibration of subscription_service.py:
├── Type: Business logic → Base trust: 30%
├── Adjustments:
│   ├── -10%: Depends on pricing_service (which has issues)
│   ├── -5%:  Upgrade/cancel logic has financial implications
│   ├── -10%: There are no tests
│   └── +5%:  Clear and readable structure
├── Final trust: 10%
├── Depth: Line by line (20+ min)
└── What I verify:
    ├── ⚠️ cancel() doesn't refund or calculate the remaining period
    ├── ⚠️ upgrade() charges proration but doesn't record the charge
    ├── ⚠️ There's no downgrade (only upgrade)
    ├── ⚠️ Doesn't verify that new_plan is different from the current plan
    ├── ⚠️ In-memory storage — needs a real DB
    ├── ⚠️ proration_charge is converted to float (loses precision)
    └── → NEEDS SIGNIFICANT EDIT

Module calibration summary:

File                   | Trust | Decision
-----------------------|-------|------------------
models.py              |  85%  | Accept
pricing_service.py     |   5%  | Edit significantly
subscription_service.py|  10%  | Edit significantly
routes.py (not shown)  |  45%  | Edit after fixes

The calibration tells you exactly where to invest your time: almost none on models.py, almost all on pricing and subscription.

---

The Difference Between Correct and Incorrect Calibration

Case 1: Over-calibration (you trust too much)

Task: Generate a password validation function
Your calibration: 70% ("it's a simple function")

What happened:
- The function only validated minimum length
- It didn't check for special characters
- It didn't prevent common passwords (password123)
- It had no protection against timing attacks

Correct calibration: 20-30%
- It's a security function
- It has compliance implications
- It requires specific knowledge of best practices

Error: You confused "simple to read" with "low risk"

Case 2: Under-calibration (you distrust too much)

Task: Generate Pydantic models for a blog
Your calibration: 25% ("I don't trust AI")
Time invested: 30 minutes reviewing 40 lines of models

What you found:
- Everything was correct
- Reasonable fields
- Correct types
- Validations present

Correct calibration: 75-85%
- Simple data models
- Standard pattern
- Low risk
- 3 minutes of review would have been enough

Error: You spent an extra 27 minutes finding nothing

Case 3: Correct calibration

Task: Generate a webhook endpoint to process Stripe payments
Your calibration: 15%
Time invested: 25 minutes

What you found:
- It didn't verify the webhook signature (critical)
- It didn't handle duplicate events (idempotency)
- It hardcoded the webhook secret
- It had no retry logic for failed processing

Result: You found 4 critical issues in 25 minutes.
Without your exhaustive review, any of these issues
could have caused a production incident.

Correct calibration: Yes. 15% for payment webhooks.

---

Connection to the Project

How you'll use Trust Calibration in Module 8

In the capstone project you'll receive a codebase with multiple modules. Your calibration table guides you:

1. Before starting: Classify each file of the codebase by type of task and assign base trust.
2. During the review: Adjust trust for contextual factors. Are there tests? Do you know the domain?
3. Prioritization: Start with the files with the lowest calibration (highest risk). If you have 60 minutes for the review, spend 40 on the files at 10-20% trust and 20 on the rest.
4. Documentation: For each module reviewed, document your calibration, what you found, and whether your calibration was correct.

---

Troubleshooting

Problem 1: "My table doesn't cover all my cases"

Cause: The table of 15 types is a starting point, not exhaustive. Solution: Add rows to your table according to your context. If you work with WebSockets frequently, add "WebSocket handlers" with the trust you consider appropriate. If you work with ML pipelines, add that category. The table is yours — customize it.

Problem 2: "I don't know if a factor raises or lowers my trust"

Cause: Lack of experience evaluating factors. Solution: Simple rule: if the factor reduces your ability to detect errors (you don't know the domain, there are no tests, the code is complex) → lower trust. If the factor increases your ability (you know the domain, there are tests, standard pattern) → raise trust.

Problem 3: "My initial calibration is always wrong"

Cause: Your base table may not reflect your experience with Claude Code specifically. Solution: Start conservative (low trust) and raise it gradually with experience. It's better to invest extra time at the start than to discover bugs later. After 20-30 experiences, your table will be accurate for your context.

Problem 4: "My team has different calibrations"

Cause: Each person has different experience and context. Solution: That's normal and expected. What matters is that each person has a calibration that's explicit, not that everyone has the same one. In shared code (auth, payments), the team should agree on a minimum calibration: "security code always has trust < 20%, regardless of who reviews it."

Problem 5: "I don't have time to keep a calibration log"

Cause: The log feels like extra work. Solution: You don't need a formal log. It's enough to update your table mentally: "the last time I generated SQL with Claude Code, I found an N+1. My trust in queries drops to 35%." If you want to be rigorous, a text file with 5 columns is enough. It takes 30 seconds per entry.

---

Exercises

Exercise 1: Calibrate everyday tasks (Easy)

For each task, assign base trust, apply adjustment factors, and define review depth:

1. Generate a docker-compose.yml for PostgreSQL + Redis + your app
2. Create a function that parses dates in 5 different formats
3. Implement an endpoint that deletes a user's account and all their data
4. Generate type hints for 20 existing functions

See solution

1. docker-compose.yml

• Base trust: 80% (Dockerfiles/CI config)
• Adjustment: +5% if you know Docker well, -5% if there are secrets (DB passwords)
• Final trust: ~80%
• Review: Quick (2-3 min). Verify: correct ports, passwords not hardcoded (use env vars), image versions.

2. Date parsing (5 formats)

• Base trust: 60% (utility function)
• Adjustment: -10% for complexity (multiple formats = edge cases), -5% for implicit regex
• Final trust: ~45%
• Review: Detailed (10-15 min). Verify: that it covers the 5 formats, that it handles invalid formats, that it handles timezones if applicable, test with edge-case inputs.

3. Account and data deletion

• Base trust: 25% (destructive operation + compliance)
• Adjustment: -10% if there are regulations (GDPR), -10% no tests, -5% for side effects (deleting across multiple tables)
• Final trust: ~0-5%
• Review: Line by line (25-35 min). Verify: that it deletes EVERYTHING (leaves no orphaned data), that it's transactional, that it requires confirmation, that it has auth, that it logs, GDPR compliance if applicable.

4. Type hints for existing functions

• Base trust: 85% (documentation/boilerplate)
• Adjustment: +5% if the functions are simple, -5% if the functions have complex logic Claude Code could misinterpret
• Final trust: ~85%
• Review: Quick visual (3-5 min). Verify: that the types are correct (especially return types and Optional), that it doesn't introduce mypy errors.

Exercise 2: Find the incorrect calibration (Medium)

A developer has this calibration table. Find the errors:

My calibration table:
1. REST endpoints     → 80%  (they're always the same)
2. SQL queries        → 70%  (SQL is simple)
3. Auth middleware    → 60%  (I've done it before)
4. README.md          → 30%  (I don't trust the info)
5. Regex validation   → 75%  (short expressions)
6. Payment processing → 50%  (it's just math)

See solution

Errors in the calibration:

1. REST endpoints (80%): Too high as a generalization. A CRUD endpoint can be 70%, but an endpoint that handles permissions or sensitive data should be 30-40%. The justification "they're always the same" ignores that endpoints vary enormously in risk.

2. SQL queries (70%): Dangerously high. SQL has the risk of injection, N+1, and incorrect logic. It should be 40-55%. The justification "SQL is simple" is a trap — SQL looks simple but has subtle pitfalls.

3. Auth middleware (60%): Too high for security. Auth should be at 10-20% always. "I've done it before" raises your familiarity (+5-10%) but not enough to compensate for the fact that it's security code.

4. README.md (30%): Too low. Documentation carries low risk. 85-90% is appropriate. The "incorrect info" concern is valid but is resolved with a 2-minute review, not 15.

5. Regex validation (75%): Too high. Regex is one of the areas where AI is most unpredictable. 15-25% is more appropriate. "Short expressions" doesn't reduce the risk — a 10-character regex can have catastrophic backtracking.

6. Payment processing (50%): Too high. Financial processing should be 10-20%. "It's just math" ignores precision (float vs Decimal), compliance, currency edge cases, rounding, and the impact of an error (direct financial loss).

Pattern: This developer calibrates based on perceived difficulty ("it's simple," "they're short") instead of the impact of an error. Result: over-trusting high-risk code and under-trusting low-risk code.

Exercise 3: Calibrate a real output (Medium)

Claude Code generated this function. Calibrate: type of task, base trust, adjustment factors, final trust, and review depth. Then, review according to your calibration.

from datetime import datetime, timedelta
from typing import Optional
import jwt
import os

SECRET = os.getenv("JWT_SECRET")
ALGORITHM = "HS256"

def create_reset_token(user_email: str, expires_in: int = 3600) -> str:
    payload = {
        "sub": user_email,
        "type": "password_reset",
        "exp": datetime.utcnow() + timedelta(seconds=expires_in),
        "iat": datetime.utcnow(),
    }
    return jwt.encode(payload, SECRET, algorithm=ALGORITHM)

def verify_reset_token(token: str) -> Optional[str]:
    try:
        payload = jwt.decode(token, SECRET, algorithms=[ALGORITHM])
        if payload.get("type") != "password_reset":
            return None
        return payload.get("sub")
    except jwt.ExpiredSignatureError:
        return None
    except jwt.InvalidTokenError:
        return None

def reset_password(token: str, new_password: str) -> bool:
    email = verify_reset_token(token)
    if not email:
        return False

    # Update password in database
    from database import get_user_by_email, update_user_password
    user = get_user_by_email(email)
    if not user:
        return False

    update_user_password(user.id, new_password)
    return True

See solution

Calibration:

• Type of task: Auth/Security (password reset) → Base trust: 15%
• Factors:
  • -10%: Token-based auth with security implications
  • -5%: There are no tests
  • +5%: Relatively short and readable code
  • -5%: Circular import in reset_password (from database import...)
• Final trust: 0% (practical minimum: treat it as total distrust)
• Depth: Line by line + verify against docs

Line-by-line review:

1. ✅ SECRET = os.getenv("JWT_SECRET") — No fallback, good. But if JWT_SECRET isn't set, SECRET is None → jwt.encode will fail with a cryptic error. It should validate that SECRET exists at startup.

2. ⚠️ expires_in: int = 3600 — 1 hour for a reset token. Is it appropriate? Most services use 15-30 minutes. 1 hour is generous for an attacker.

3. ⚠️ verify_reset_token — Returns None for any error. It doesn't distinguish between an expired token (you tell the user "your link expired, request a new one") and an invalid token (a possible attack). The UX and security would benefit from distinguishing these cases.

4. ⚠️ reset_password — It doesn't hash new_password. It calls update_user_password(user.id, new_password) with the password in plaintext. If update_user_password doesn't hash internally, the password is saved in plaintext in the database. CRITICAL issue.

5. ⚠️ reset_password — It doesn't invalidate the token after using it. A reset token can be used multiple times until it expires. This lets an attacker who intercepts the token change the password multiple times.

6. ⚠️ Circular import — from database import... inside the function. It works but it's a code smell that can cause problems.

7. ⚠️ There's no rate limiting — An attacker can try thousands of tokens by brute force.

8. ⚠️ There's no complexity validation for new_password — The new password can be "1".

Decision: REJECT AND REGENERATE. Issues 4 (doesn't hash the password) and 5 (doesn't invalidate the token) are security deal-breakers. The function has the right shape but fails at the details that matter.

Exercise 4: Build your custom table (Hard)

Create your own Trust Calibration table with at least 10 types of task relevant to your work. For each one:

1. Type of task
2. Base trust (%)
3. Top 3 things to verify
4. A factor that raises your trust
5. A factor that lowers your trust

See solution

There's no single answer. Your table should meet these criteria:

• ✅ It has at least 3 distinct trust levels (not everything at 50%)
• ✅ Security code is at 10-25%
• ✅ Boilerplate and docs are at 75-90%
• ✅ Business logic is at 25-40%
• ✅ Each entry has specific "what to verify" (not just "review")
• ✅ The adjustment factors are concrete, not vague

Example for a backend developer in SaaS:

Type of task	Trust	Top 3 checks	Raise if...	Lower if...
Data models	80%	Fields, types, validations	Familiar domain	Complex relationships
CRUD endpoints	65%	Validations, auth, status codes	Tests exist	Sensitive data
Auth/JWT	15%	Every line, against docs, tests	Mature library	Custom implementation
SQL queries	45%	Injection, N+1, logic	ORM used well	Raw SQL
Background jobs	40%	Retry logic, idempotency, errors	Standard pattern	Complex state
WebSocket handlers	35%	Connection management, auth, memory leaks	Familiar domain	Concurrency
Email templates	80%	Correct content, variables	Simple text	Complex HTML
API integrations	35%	Error handling, retry, correct API	Well-documented API	API changed recently
Config files	85%	Secrets not hardcoded, values	Standard setup	Multi-environment
Migrations	30%	Reversibility, data loss, locks	Simple schema	Existing data

---

Summary

In this capsule you learned:

• Trust Calibration turns "I think it's fine" into a structured process with trust quantified by type of task
• The table of 15 types of task gives you a concrete starting point for calibrating your trust
• Adjustment factors raise or lower your base trust: tests, familiarity, complexity, security, track record
• Trust Debt accumulates when you trust too much — the problems don't disappear, they accumulate silently
• Your calibration table updates with experience — start conservative and adjust with real data
• Correct calibration invests time proportional to the risk: 30 minutes on auth, 2 minutes on models
• Over-calibration (trusting too much) and under-calibration (distrusting too much) are both costly

Next capsule: Integrative exercise — applying the 3 mental models together to real scenarios with code.

---

Additional resources

1. Thinking in Bets — Annie Duke - Trust calibration and decision-making under uncertainty
2. Superforecasting — Philip Tetlock - How the best forecasters calibrate their confidence
3. Google — Code Review Developer Guide - How Google prioritizes by risk in code review
4. OWASP — Risk Rating Methodology - A risk evaluation framework for security
5. Anthropic — Claude Code Documentation - Official Claude Code documentation
6. Risk-Based Testing — ISTQB - Principles of risk-based testing applicable to trust calibration

---

Debugging & Code Review with Claude Code — Module 2, Capsule 04 Claude Code Agentic Development Path — Guide #6 of 11