Hallucinations in Imports and APIs

Capsule overview

In the previous capsule you classified hallucinations into 4 types. Now you're going to go deeper into the first two: fake imports and APIs with invented signatures. They're the most common types — they represent approximately 60% of hallucinations in AI-generated code — and the easiest to detect if you know where to look.

This capsule gives you a systematic process for verifying imports and API calls. It's not about memorizing every function of every library — that's impossible. It's about developing the instinct for when to verify and the techniques for how to do it fast.

Fake Imports: The Most Common Case

Why imports are the #1 target for hallucinations

LLMs see thousands of imports during training. For popular libraries like FastAPI, pandas, sklearn, or requests, the model has seen tens of thousands of import combinations. The problem: not all those combinations correspond to the same version of the library, and many combinations the LLM generates are extrapolations of what it has seen, not copies of real imports.

Anatomy of a fake import

from fastapi.security import OAuth2TokenValidator
#    ^^^^^^^^^^^^^^^^^^^    ^^^^^^^^^^^^^^^^^^^^
#    real module            invented class
#    (fastapi.security      (OAuth2PasswordBearer does exist,
#     does exist)           OAuth2TokenValidator doesn't)

The most common pattern: the module is real but the class/function is invented. The LLM knows that fastapi.security exists because it has seen it hundreds of times. But it invents a class name that fits the module's pattern.

The 5 most common patterns of fake imports

Pattern 1: Combining existing concepts

The LLM combines two real concepts into a name that doesn't exist:

# They exist separately:
from collections import OrderedDict
from collections import defaultdict

# ❌ The LLM combines both:
from collections import OrderedDefaultDict
# Doesn't exist — it was never implemented in standard Python

# They exist separately:
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import MultiLabelBinarizer

# ❌ The LLM combines concepts:
from sklearn.metrics import roc_auc_multiclass
# Doesn't exist — for multiclass you use roc_auc_score with multi_class="ovr"

Pattern 2: Extrapolated naming convention

The LLM sees a naming pattern and extends it:

# They exist in FastAPI:
from fastapi.security import OAuth2PasswordBearer
from fastapi.security import OAuth2AuthorizationCodeBearer

# ❌ The LLM extends the pattern:
from fastapi.security import OAuth2TokenValidator
from fastapi.security import OAuth2RefreshTokenBearer
# The OAuth2[X]Bearer / OAuth2[X]Validator pattern seems logical,
# but these classes don't exist

Pattern 3: A submodule that should exist but doesn't

# json is a standard module
import json

# ❌ The LLM assumes submodules that follow other libraries' conventions:
from json.exceptions import JSONDecodeError
# "json.exceptions" doesn't exist as a submodule
# ✅ The correct thing: from json import JSONDecodeError

# logging has handlers
import logging

# ❌ The LLM invents specific handlers:
from logging.handlers import JSONHandler
# Doesn't exist — there's RotatingFileHandler, TimedRotatingFileHandler, etc.
# For JSON logging you use python-json-logger (a separate package)

Pattern 4: A popular alias that isn't the real name

# ❌ "Router" seems more intuitive than "APIRouter"
from fastapi import Router
# ✅ The correct thing:
from fastapi import APIRouter

# ❌ "JsonResponse" with that capitalization
from fastapi.responses import JsonResponse
# ✅ The correct thing:
from fastapi.responses import JSONResponse
# The difference: "Json" vs "JSON"

Pattern 5: An import from an earlier version

# ❌ Pydantic v1 style (deprecated in v2)
from pydantic import validator

# ✅ Pydantic v2 style:
from pydantic import field_validator

# ❌ An earlier httpx version:
from httpx import TestClient

# ✅ For testing FastAPI with httpx:
from httpx import ASGITransport, AsyncClient
# TestClient is in starlette: from starlette.testclient import TestClient

Verification process for imports

When you see an import you don't recognize, follow this process:

STEP 1: Do you recognize the import?
├── Yes → Probably correct (but verify if you haven't used the library in a while)
└── No → Continue to step 2

STEP 2: Is the base module real?
├── python -c "import fastapi.security"
├── If it fails → The whole import is fake
└── If it works → The module is real, verify the class/function

STEP 3: Does the class/function exist in the module?
├── python -c "from fastapi.security import OAuth2TokenValidator"
├── If it works → Valid import
└── If it gives an ImportError → Hallucination confirmed

STEP 4: What DOES exist in that module?
├── python -c "import fastapi.security; print(dir(fastapi.security))"
└── Look for the correct alternative in the output

Verification in the terminal

# Verify a specific import
python -c "from fastapi.security import OAuth2TokenValidator"
# Output: ImportError: cannot import name 'OAuth2TokenValidator'

# See what exists in a module
python -c "import fastapi.security; print([x for x in dir(fastapi.security) if not x.startswith('_')])"
# Output: ['HTTPAuthorizationCredentials', 'HTTPBasic', 'HTTPBasicCredentials',
#  'HTTPBearer', 'OAuth2', 'OAuth2AuthorizationCodeBearer', 
#  'OAuth2PasswordBearer', 'OAuth2PasswordRequestForm', 
#  'OAuth2PasswordRequestFormStrict', 'OpenIdConnect', 'SecurityScopes']

# Verify whether a package is installed
pip show pyjwt
# Output: Name: PyJWT, Version: 2.8.0, etc.

# Look up a function in the package's documentation
python -c "import jwt; help(jwt.decode)"

APIs with Invented Signatures: The Most Subtle Case

What makes them different from fake imports

With fake imports, the function/class doesn't exist. Python gives you an immediate ImportError. With invented APIs, the function does exist but it's used with arguments that don't exist or with values that aren't valid.

This makes them more dangerous because:

1. The import doesn't fail
2. In some cases, the extra argument is silently ignored (with **kwargs)
3. The error can appear only at runtime, when the function is called with certain inputs

Anatomy of an invented API

# The function exists
decoded = jwt.decode(token, secret, algorithms=["HS256"])  # ✅ Correct

# The function exists but with an invented parameter
decoded = jwt.decode(token, secret, algorithms=["HS256"], verify=True)  # ❌
#                                                         ^^^^^^^^^^^
#                                     invented parameter — "verify" doesn't exist
#                                     the real thing: options={"verify_signature": True}

The 5 most common patterns of invented APIs

Pattern 1: An invented argument value

The function and the parameter exist, but the value isn't valid:

import pandas as pd

df = pd.DataFrame({"a": [1, 2, 3]})

# ❌ orient="dict" isn't a valid value
result = df.to_json(orient="dict")

# ✅ Valid values:
# "split", "records", "index", "columns", "values", "table"
result = df.to_json(orient="records")

# Note: df.to_dict() exists as a separate method
# The LLM mixed to_json(orient=...) with to_dict()

Pattern 2: A chained method that doesn't exist

The LLM generates a method that fits the library's API but doesn't exist:

from sqlalchemy import select
from sqlalchemy.orm import Session

# ❌ .eager_load() doesn't exist as a method of Select
stmt = select(User).where(User.active == True).eager_load(User.posts)

# ✅ The correct thing:
from sqlalchemy.orm import joinedload
stmt = select(User).where(User.active == True).options(joinedload(User.posts))

# ❌ .filter_by_date() doesn't exist in QuerySet
users = db.query(User).filter_by_date(created_at__gte=start_date)

# ✅ The correct thing (SQLAlchemy):
users = db.query(User).filter(User.created_at >= start_date)

Pattern 3: A constructor with parameters from another domain

from pydantic import BaseModel, Field

class Product(BaseModel):
    # ❌ unique=True and index=True are database concepts, not Pydantic
    sku: str = Field(..., unique=True, index=True)
    name: str = Field(..., min_length=1, max_length=200)
    price: float = Field(..., gt=0)

# ✅ Pydantic Field() accepts: default, alias, title, description,
#    gt, ge, lt, le, min_length, max_length, pattern, etc.
#    For DB constraints: use SQLAlchemy Column(unique=True, index=True)

Pattern 4: A function with a signature from a different version

# ❌ In Pydantic v2, the API changed
from pydantic import BaseModel

class User(BaseModel):
    name: str
    
    class Config:  # ❌ v1 style
        orm_mode = True

# ✅ In Pydantic v2:
class User(BaseModel):
    name: str
    
    model_config = {"from_attributes": True}  # v2 style

# ❌ pytest.raises with match as a positional argument
import pytest

with pytest.raises(ValueError, "invalid input"):
    process_data(bad_input)

# ✅ The correct thing:
with pytest.raises(ValueError, match="invalid input"):
    process_data(bad_input)

Pattern 5: A real function with an incorrectly assumed return type

import os

# ❌ The LLM assumes os.getenv always returns str
port: int = int(os.getenv("PORT"))
# If PORT isn't defined, os.getenv returns None
# int(None) → TypeError

# ✅ The correct thing:
port: int = int(os.getenv("PORT", "8000"))
# Or with validation:
port_str = os.getenv("PORT")
if port_str is None:
    raise ValueError("PORT environment variable is required")
port = int(port_str)

Case study: The hallucination that cost 4 hours

Imagine this real scenario:

from fastapi import FastAPI, HTTPException
from fastapi.middleware.cors import CORSMiddleware

app = FastAPI()

app.add_middleware(
    CORSMiddleware,
    allowed_origins=["http://localhost:3000"],  # ❌ HALLUCINATION
    allowed_methods=["GET", "POST", "PUT", "DELETE"],
    allowed_headers=["*"],
    allow_credentials=True,
)

The developer copies this code. Runs it. The app starts with no errors. But CORS doesn't work — the frontend at localhost:3000 keeps getting CORS errors.

Why? Because the real parameter is allow_origins (without a "d"), not allowed_origins. The extra parameter is silently ignored by **kwargs. CORS is configured with the defaults (which don't include localhost:3000).

The developer spends 4 hours:

1. Reviewing the browser headers (1 hour)
2. Looking for the error in the frontend (1 hour)
3. Trying different CORS configurations (1 hour)
4. Finally, reading the official documentation and comparing parameter by parameter (30 min)
5. Finds allow_origins vs allowed_origins (30 min)

4 hours for an extra "d". That's the cost of a Type 2/3 hallucination that passes silently.

Verification process for APIs

STEP 1: Does the function exist?
├── python -c "import jwt; print(type(jwt.decode))"
├── If it gives an AttributeError → The function doesn't exist (Type 1)
└── If it works → The function is real, verify the arguments

STEP 2: Are the arguments correct?
├── python -c "import inspect; import jwt; print(inspect.signature(jwt.decode))"
├── Compare the parameters you see with the ones the code uses
└── Any parameter that doesn't appear in the signature → suspicious

STEP 3: Are the values valid?
├── Look up the accepted values in the documentation
├── For enums/options: the docs list the valid values
└── For types: verify that the value's type matches

STEP 4: Verify with a quick test
├── Run the function with the arguments from the code
├── If it gives a TypeError → Argument not accepted
├── If it works but the result is unexpected → Incorrect value
└── If it works and the result is correct → The API is valid

Verification in the terminal

# See a function's signature
python -c "import inspect; import jwt; print(inspect.signature(jwt.decode))"
# Output: (jwt, key='', algorithms=None, options=None, ...)

# See the full documentation
python -c "import jwt; help(jwt.decode)"

# Quick test of a parameter value
python -c "
import pandas as pd
df = pd.DataFrame({'a': [1]})
try:
    print(df.to_json(orient='dict'))
except ValueError as e:
    print(f'Error: {e}')
"
# Output: Error: Invalid value 'dict' for option 'orient'

# Verify official documentation in the terminal
python -c "import pandas; help(pandas.DataFrame.to_json)"

Strategy: When to Verify and When to Trust

The 80/20 rule for imports and APIs

You can't verify every import and every API call in every file AI generates — productivity would drop to zero. You need a strategy for when to verify:

ALWAYS verify:
├── Imports you haven't personally used before
├── Specific submodules (from X.Y.Z import ...)
├── API calls with parameters you don't recognize
├── Security functions (auth, encryption, hashing)
└── Functions from libraries that were recently updated

VERIFY IF IN DOUBT:
├── Imports from libraries you know but from uncommon modules
├── API calls with specific parameter values
├── Chained methods (.method1().method2().method3())
└── Testing functions with non-standard arguments

GENERALLY TRUST:
├── Imports from Python standard modules (os, json, datetime)
├── Imports of a framework's main class (FastAPI, BaseModel)
├── API calls you've used hundreds of times
└── Functions with simple signatures (1-2 known arguments)

Practical rule: "If I haven't written it before, I verify"

The simplest and most effective rule: if it's an import or API call you haven't personally written at least once, verify it. It doesn't matter if it sounds correct. It doesn't matter if Claude Code generated it with confidence. If it's the first time you see it, verify it.

This rule is conservative at first (you verify a lot) but it relaxes naturally with experience: every verified import becomes one you know for next time.

Import Hallucinations by Library

Top 5 libraries with the most import hallucinations

Based on common patterns of hallucinations in AI-generated Python code:

1. FastAPI / Starlette

# Common hallucinations:
from fastapi.security import OAuth2TokenValidator    # ❌
from fastapi import Router                           # ❌ (it's APIRouter)
from fastapi.responses import JsonResponse           # ❌ (it's JSONResponse)
from fastapi import QueryParam                       # ❌ (it's Query)
from starlette.middleware import SessionMiddleware    # ❌ (it's from starlette.middleware.sessions)

2. SQLAlchemy

# Common hallucinations:
from sqlalchemy.orm import relationship, backref, lazy_load  # ❌ lazy_load doesn't exist
from sqlalchemy import Column, Integer, UniqueConstraint     # ⚠️ UniqueConstraint is real but imported differently
from sqlalchemy.ext.asyncio import AsyncSession, async_session  # ❌ async_session as a function doesn't exist like this

3. Pydantic

# Common hallucinations:
from pydantic import validator          # ⚠️ v1 — deprecated in v2, it's field_validator
from pydantic import Schema             # ❌ doesn't exist
from pydantic.types import EmailStr     # ❌ it's from pydantic import EmailStr
from pydantic import ConfigDict         # ⚠️ Depends on the version

4. sklearn / scikit-learn

# Common hallucinations:
from sklearn.metrics import roc_auc_multiclass           # ❌
from sklearn.preprocessing import TextVectorizer         # ❌ (it's CountVectorizer or TfidfVectorizer)
from sklearn.model_selection import StratifiedKFoldCV    # ❌ (it's StratifiedKFold)
from sklearn.ensemble import XGBoostClassifier           # ❌ (XGBoost is a separate package)

5. pytest

# Common hallucinations:
from pytest import mock                     # ❌ (it's from unittest.mock or from pytest_mock)
from pytest import parametrize              # ❌ (it's @pytest.mark.parametrize as a decorator)
from pytest.fixtures import fixture         # ❌ (it's @pytest.fixture as a decorator)

Connection to the Project

How it shows up in the capstone project

In the codebase of the capstone project (module 8), there are 1-2 fake imports and 1 invented API planted. Examples of the type of hallucination you might find:

# In the project's authentication file:
from fastapi.security import OAuth2PasswordBearer  # ✅ Correct
from fastapi.security import SecurityScopes         # ✅ Correct
# But in another file:
from fastapi.security import TokenValidator          # ❌ Doesn't exist

The technique: when you review a codebase, verify all the imports you don't recognize before moving on to the logic. It's the fastest verification (1-2 seconds per import with python -c) and it eliminates the most obvious hallucinations first.

Process for the project

1. List all the unique imports in the codebase
2. Mark the ones you recognize vs the ones you don't
3. Verify the ones you don't recognize with python -c
4. Note the hallucinations found with their type and location
5. Move on to API verification (the next level)

Troubleshooting

Problem 1: "I verify an import but it gives an error because I don't have the package installed"

Cause: You don't have the library installed locally.

Solution: First verify that the package exists by searching at https://pypi.org/project/[name]/ or running pip index versions [name]. If the package exists, install it: pip install [name]. Then verify the import. If the package doesn't exist on PyPI, the whole package is a hallucination.

Problem 2: "The import works but I'm not sure it's the correct way"

Cause: There may be multiple ways to import the same thing.

Solution: Verify the official documentation. Many libraries have deprecated imports that still work but aren't recommended. Example: from pydantic import validator works in Pydantic v2 but is deprecated. Working doesn't mean it's correct for your version.

Problem 3: "How do I verify APIs if the function accepts **kwargs?"

Cause: Functions with **kwargs accept any argument without an error, making verification harder.

Solution: For functions with **kwargs:

1. Read the documentation to know which parameters are actually processed
2. Look in the function's source code for which kwargs are extracted
3. Do a quick test: pass the parameter and check whether it has an effect
4. Example: requests.get(url, verify_ssl=False) doesn't give an error but verify_ssl has no effect — verify that SSL is still active

Problem 4: "There are too many imports to verify in a large file"

Cause: A file with 30+ imports is overwhelming to verify one by one.

Solution: Prioritize:

1. Filter out imports from Python standard modules (generally correct)
2. Filter out imports of each library's main class (generally correct)
3. Focus on imports of submodules and specific functions
4. Use a linter like flake8 or ruff that detects imports that don't exist or aren't used

Problem 5: "The code uses a different version of the library than I have"

Cause: The hallucination could be valid code from another version.

Solution: Verify which version of the library the project uses (pip show [library] or check requirements.txt). Then verify against that specific version's documentation. What's correct in v1 can be incorrect in v2.

Exercises

Exercise 1: Verify imports (Easy)

For each import, determine whether it's correct or a hallucination. Verify with python -c if in doubt:

# 1
from fastapi import FastAPI, HTTPException, Depends

# 2
from pydantic import BaseModel, Field, EmailStr

# 3
from sqlalchemy.orm import joinedload, selectinload

# 4
from fastapi.security import OAuth2PasswordBearer, OAuth2TokenValidator

# 5
from collections import OrderedDict, namedtuple, ChainMap

Exercise 2: Detect the invented API (Medium)

In each snippet, there's one incorrect API call. Find it:

Snippet A:

import pandas as pd

df = pd.read_csv("data.csv")
summary = df.describe()
json_output = df.to_json(orient="records")
filtered = df.query("age > 30")
sorted_df = df.sort_values(by="name", ascending=True)
unique_names = df["name"].unique_values()

Snippet B:

from pathlib import Path

config_dir = Path.home() / ".config" / "myapp"
config_dir.mkdir(parents=True, exist_ok=True)

config_file = config_dir / "settings.json"
content = config_file.read_text(encoding="utf-8")
config_file.write_text('{"key": "value"}', encoding="utf-8")
files = list(config_dir.iterdir())
size = config_file.file_size()

Exercise 3: Verify API parameters (Medium)

These API calls use real functions. Are the parameters correct?

# Call 1
import jwt
token = jwt.encode({"user": "alice"}, "secret", algorithm="HS256")

# Call 2
import requests
response = requests.get("https://api.example.com", verify_ssl=False)

# Call 3
from sqlalchemy import create_engine
engine = create_engine("sqlite:///db.sqlite", echo=True, pool_size=5)

# Call 4
import logging
logging.basicConfig(level=logging.DEBUG, format="%(message)s")

Exercise 4: Hallucination investigation (Hard)

Claude Code generated this authentication code. Without running it, identify all the hallucinations:

from fastapi import FastAPI, Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer, OAuth2PasswordRequestForm
from fastapi.authentication import AuthenticationMiddleware
from jose import jwt, JWTError
from passlib.context import CryptContext
from pydantic import BaseModel, EmailStr
from datetime import datetime, timedelta
from typing import Optional

app = FastAPI()

pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="auth/token")

SECRET_KEY = "your-secret-key"
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE = 30

class Token(BaseModel):
    access_token: str
    token_type: str

class TokenData(BaseModel):
    username: Optional[str] = None

def create_access_token(data: dict, expires_delta: Optional[timedelta] = None):
    to_encode = data.copy()
    expire = datetime.utcnow() + (expires_delta or timedelta(minutes=15))
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
    return encoded_jwt

async def get_current_user(token: str = Depends(oauth2_scheme)):
    credentials_exception = HTTPException(
        status_code=status.HTTP_401_UNAUTHORIZED,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        username: str = payload.get("sub")
        if username is None:
            raise credentials_exception
        token_data = TokenData(username=username)
    except JWTError:
        raise credentials_exception
    return token_data

from fastapi.authentication import AuthenticationMiddleware  # ❌ HALLUCINATION

Exercise 5: Build a personal checklist (Hard)

Based on what you've learned, build an import and API verification checklist for a library you use in your work. The checklist should include:

1. Top 5 imports you would always verify
2. Top 3 API calls with easily confused parameters
3. A 3-step verification process

Summary

In this capsule you learned:

• Fake imports follow 5 patterns: combining concepts, extrapolated naming, assumed submodules, popular aliases, imports from another version
• Invented APIs follow 5 patterns: invented argument values, fake chained methods, parameters from another domain, signatures from another version, assumed return types
• The verification process for imports: verify the module → verify the class → see what exists in the module
• The verification process for APIs: verify the function → inspect the signature → verify the values → quick test
• The 80/20 rule: always verify unknown imports and submodules; trust standard imports and main classes
• The simple rule: "if I haven't written it before personally, I verify"
• API hallucinations with **kwargs are the most dangerous because they fail silently

Next capsule: Hallucinations in Logic — code that looks correct but implements something different from what it says.

Additional resources

1. Python inspect module - How to inspect function signatures programmatically
2. FastAPI Security Documentation - Official reference for security classes
3. Pydantic v1 → v2 Migration Guide - Guide to the changes between versions
4. SQLAlchemy Documentation - Reference for verifying ORM APIs
5. PyPI Search - Verify the existence of Python packages
6. ruff — Python Linter - A fast linter that detects nonexistent imports

Debugging & Code Review with Claude Code — Module 3, Capsule 03 Claude Code Agentic Development Path — Guide #6 of 11