Claude hallucinates framework APIs: writes method calls that do not exist in the framework version used

Problem

Claude writes calls to framework APIs, methods, and library functions that do not exist in the version being used. Unlike hallucinating column names (where the column is just named wrong), this is hallucinating entire API surfaces -- calling methods that have never existed on that class, using function signatures from different framework versions, or constructing API calls that are plausible but fictional.

These hallucinated API calls pass syntax checking, linting, and even import-time checks. They only crash at runtime on the specific code path, meaning they can ship to production and sit dormant until first real use.

---

The Core Behavior

Claude generates code that calls methods on the correct class, but methods that do not exist on that class in the framework version being used. The hallucination is plausible -- the method name sounds right, the signature looks reasonable, the usage pattern matches what similar methods would do -- but the method simply does not exist.

This differs from identifier hallucination (wrong column name) because:

• Column name bugs can often be caught by simple grep or static analysis
• API hallucination requires knowledge of the framework version's actual class interface
• The code looks syntactically and semantically correct -- only runtime execution reveals the error
• py_compile, ruff, mypy (without stubs), and even careful human review may miss it

---

Incident Timeline

Incident 1: KB-084 -- AsyncSession.execution_options() does not exist (2026-04-27, CRITICAL)

What happened: Claude wrote session.execution_options(schema_translate_map=...) in 4 different files as the core mechanism for multi-tenant schema routing. This is the foundation of the entire multi-tenant architecture -- every tenant query goes through this code path.

The hallucination:

# What Claude wrote (in 4 files):
session = session.execution_options(schema_translate_map={None: f"co_{company_id}"})

# Reality: execution_options() does NOT exist on AsyncSession or Session
# in SQLAlchemy 2.0.x (verified 2.0.43). It only exists on AsyncConnection.

# Correct code:
conn = await session.connection()
await conn.execution_options(schema_translate_map={None: f"co_{company_id}"})

Files affected:

• backend/app/database.py -- the core tenant session dependency (every tenant endpoint uses this)
• backend/app/tasks/tally_sync.py -- background sync task
• backend/app/tasks/export_tasks.py -- report export task
• backend/app/tasks/intelligence_build.py -- analytics build task

Impact: Every tenant-scoped database operation would crash with AttributeError: 'AsyncSession' object has no attribute 'execution_options'. This affects 100% of business functionality (ledgers, vouchers, sync, exports, analytics).

Why it wasn't caught earlier:

• py_compile passes (syntax is valid Python)
• ruff passes (no linting violation)
• Import-time works fine (the attribute isn't accessed until the function body runs)
• Dev testing may not have exercised tenant-scoped paths
• The code looks completely reasonable -- execution_options IS a real SQLAlchemy concept, just not available on AsyncSession

From KB-084: "execution_options() does NOT exist on AsyncSession or Session in SQLAlchemy 2.0.x (verified 2.0.43). It only exists on AsyncConnection. The call session.execution_options(...) was always invalid -- it just hadn't been exercised in production."

Incident 2: KB-069 -- Missing imports pass py_compile but crash at runtime (2026-04-26)

What happened: Claude wrote code in admin.py that referenced settings.APP_URL inside a function body but never imported settings at the top of the file.

# What Claude wrote:
async def create_invitation(...):
    invite_url = f"{settings.APP_URL}/accept-invite/{token}"  # NameError at runtime
    # ... settings was never imported

Why it wasn't caught:

• Python does not check name resolution at import time for names used inside function bodies
• py_compile only checks syntax, not name binding
• ruff doesn't catch cross-module name resolution
• The crash only happens when POST /admin/invitations is actually called with valid data

Impact: Invitation emails crash the endpoint. The feature appears to exist (route is mounted, UI has the button) but is non-functional.

Incident 3: KB-071 -- Case-sensitive class import works on Windows, crashes on Linux (2026-04-26)

What happened: Claude wrote an import statement with wrong casing that works on Windows (case-insensitive filesystem) but crashes on Linux (case-sensitive).

# What Claude wrote:
from app.services.party_pnl_service import PartyPnLService

# Actual class name:
class PartyPnlService:  # lowercase 'nl', not 'NL'

Why it wasn't caught:

• Windows py_compile passes (Windows filesystem is case-insensitive)
• Development happens on Windows; production runs on Linux Docker
• The import resolves fine on Windows but crashes immediately on Linux

Impact: Celery worker crashes on startup in production Docker environment. Every sync task fails.

Incident 4: KB-088 -- Task module not registered, silently queues forever (2026-04-27)

What happened: Claude created export_tasks.py with proper Celery task decorators but never added it to the Celery app's include list.

# In export_tasks.py (Claude wrote):
@celery_app.task(bind=True, name="export_report")
def export_report(self, ...):
    ...

# In celery_app.py (Claude did NOT update):
celery_app.conf.update(
    include=["app.tasks.tally_sync", "app.tasks.intelligence_build"]
    # Missing: "app.tasks.export_tasks"
)

Impact: Export tasks dispatched from the web UI sit in the Redis queue with PENDING status forever. No error, no failure, no timeout -- just infinite hang. The user sees a spinner that never resolves.

Why it wasn't caught: The task decorator syntax is correct. The dispatch code is correct. The queue receives the message. Only the fact that no worker is registered to consume the task causes the silent hang. No error is logged anywhere.

Incident 5: KB-089 -- Hardcoded timeout bypasses config system (2026-04-27)

What happened: Claude wrote the export task with a hardcoded statement_timeout of 300 seconds, despite having already built a config-based timeout tier system for sync and intelligence tasks.

# What Claude wrote in export_tasks.py:
await conn.execute(text("SET LOCAL statement_timeout = '300000'"))

# What Claude had already built (and should have used):
# In config.py:
DB_SYNC_TASK_TIMEOUT_MS: int = 1_800_000
DB_INTEL_TASK_TIMEOUT_MS: int = 3_600_000
# Missing: DB_EXPORT_TASK_TIMEOUT_MS (never created)

# In tally_sync.py (correct pattern):
await conn.execute(text(f"SET LOCAL statement_timeout = '{settings.DB_SYNC_TASK_TIMEOUT_MS}'"))

Pattern: Claude established a config-based pattern in earlier files but reverted to hardcoding in later files. It did not generalize its own pattern.

Incident 6: KB-063 -- Module-level side effect breaks Celery worker startup (2026-04-26)

What happened: Claude wrote Path("/backups").mkdir() at module import time in a task file. Celery's autodiscover_tasks imports all task modules on worker startup, triggering the mkdir in all environments (including dev where /backups doesn't exist and the process has no write permission).

# What Claude wrote at module level (not inside a function):
BACKUP_DIR = Path("/backups")
BACKUP_DIR.mkdir(parents=True, exist_ok=True)  # Runs on import!

Impact: Celery worker fails to start in dev environments with PermissionError. All background tasks (sync, export, analytics) are blocked.

---

Pattern Analysis

These incidents reveal a consistent model behavior:

1. Claude assumes framework APIs exist based on semantic plausibility

session.execution_options() sounds like it should exist. The method exists on other SQLAlchemy objects (Connection, Engine). Claude extrapolates that it exists on Session/AsyncSession too -- without verifying.

2. Claude doesn't distinguish between framework versions

SQLAlchemy 1.x and 2.0 have different APIs. Claude may be drawing from training data that includes both versions. The project uses SQLAlchemy 2.0 (specified in requirements.txt), but Claude writes 1.x-style code.

3. Claude doesn't verify API availability at write time

After writing session.execution_options(...), Claude should have a step where it verifies: "does AsyncSession have an execution_options method?" This verification never happens.

4. Claude inconsistently applies its own patterns

It built a config-based timeout system for sync and intelligence tasks, then hardcoded the timeout in the export task. It established a Celery include pattern, then forgot to add new task modules. The pattern exists but isn't generalized.

5. Conventional testing doesn't catch these bugs

py_compile checks syntax. Linting checks style. Type checking (without complete stubs) misses runtime method resolution. These bugs live in the gap between static analysis and runtime execution.

---

Quantified Impact

| Metric | Value |
|---|---|
| Total KB entries for API hallucination | 6 (KB-063, 069, 071, 084, 088, 089) |
| Files with hallucinated API calls | 4 (KB-084 alone: database.py + 3 task files) |
| Percentage of tenant operations broken | 100% (KB-084 affects every tenant query) |
| Detection method | Runtime crash only (not caught by compile, lint, or static analysis) |
| Time to discover | Days to weeks after code was written |
| Common trait | All pass py_compile, ruff, and import-time checks |

---

Suggested Improvements

1. Framework version awareness

When a project specifies a framework version (e.g., sqlalchemy==2.0.43 in requirements.txt), Claude should constrain its API knowledge to that version. If uncertain whether a method exists in the specified version, Claude should verify with documentation or a quick test.

2. Post-write API verification

After writing a method call on a framework class, Claude should verify the method exists:

• Check the class's known interface for that framework version
• If uncertain, note it: "I used session.execution_options() -- please verify this exists in SQLAlchemy 2.0"
• For critical code paths (like tenant isolation), this verification should be mandatory

3. Pattern generalization enforcement

When Claude establishes a pattern (e.g., config-based timeouts), it should apply that pattern in all subsequent similar contexts. Specifically:

• When writing a new Celery task: check if celery_app.conf.include needs updating
• When writing a timeout: check if a config tier exists and use it
• When writing an import: match the exact casing from the source definition

4. Cross-platform import verification

For projects that develop on Windows but deploy on Linux (common with Docker):

• After writing any import, verify the casing matches the class definition exactly
• Note case sensitivity risk in import statements

5. Distinguish module-level from function-level code in task files

In Celery/task frameworks, module-level code runs at worker startup, not at task execution time. Claude should:

• Never perform I/O, filesystem operations, or network calls at module level in task files
• Keep module-level code limited to imports, constants, and decorators
• Move all side effects into function bodies

6. Runtime reachability analysis

After writing code that references a module-level name (settings, logger, model class), Claude should immediately check the file's import block for that name. This is a simple verification: "I used settings.APP_URL -- is settings imported at the top of this file?"

---

Environment

• Tool: Claude Code CLI + VS Code extension
• Models observed: Claude Opus and Sonnet (pattern consistent across both)
• Framework: SQLAlchemy 2.0.43 (async), Celery 5.x, FastAPI 0.115.x
• Project scale: ~90K LOC migration, 31 ORM models, 4 Celery task modules
• Detection: All bugs caught by runtime testing or production deployment, never by static analysis
• Development platform: Windows (dev) -> Linux Docker (prod) -- adds case sensitivity gap

Reproduction

1. Ask Claude to write SQLAlchemy 2.0 async code for multi-tenant schema routing
2. Specify AsyncSession as the session type
3. Claude will likely write session.execution_options(schema_translate_map=...) -- this method does not exist on AsyncSession
4. py_compile will pass. ruff will pass. Runtime will crash with AttributeError.

Alternatively:

1. Ask Claude to create a new Celery task module in a project that already has 2-3 task modules registered in celery_app.conf.include
2. Claude will create the task file with correct decorators but may not update the include list
3. The task will silently hang in PENDING state when dispatched