Claude systematically claims migration/porting tasks complete when critical logic is dormant or missing

Problem

When Claude performs large-scale code migration/porting tasks (porting a ~90K LOC desktop app to a web SaaS over ~30 sessions), it exhibits three systematic failure modes that persist even after multiple audit passes using agents.

Failure Mode 1: Schema-exists-equals-complete fallacy

Claude treats the existence of models, schemas, tables, and method stubs as evidence of completion. A deep audit revealed the actual migration was ~60% complete despite Claude reporting 100% across 18 modules.

What was found:

• Delta sync methods (compare_vouchers(), get_guid_alterids()) existed in the codebase but were never called from any orchestration flow. Every sync was a full re-fetch.
• Export generation had Celery task stubs with zero working logic. The source codebase had 10+ active export types.
• Post-sync chain (golden tags, daily balances, period balances, party cache rebuild) was entirely missing -- only opening balance population existed.
• 32 of 45 connector methods existed as empty or partial stubs.
• Features had DB fields (e.g., force_password_change, org suspension status, plan limits) but no runtime code checked them -- "modeled but not enforced."

Pattern: method defined != method called. Claude does not trace call graphs to verify wiring.

Failure Mode 2: Orchestration layer blindness

When porting multi-step pipelines, Claude ports individual methods but misses the orchestration logic that sequences, batches, and handles failures.

What was found:

• 35+ XML connector methods were ported correctly, but the sync orchestration (smart date-range batching with 5,000-record cap, datewise count preflight, fallback strategies for large months) was completely missing.
• First real-world sync hit a 50 MB XML response limit because the web version fetched entire months at once, while the source had sophisticated batching.
• The source files containing the orchestration were explicitly listed in CLAUDE.md as "key files to reference" but were not consulted for the orchestration logic.

Pattern: Claude reads functions in isolation, not the control flow that calls them.

Failure Mode 3: Self-audit confirmation bias

When asked to audit its own migration work, Claude exhibits confirmation bias:

• Multiple agent-based audit passes (3+) were run, each claiming high completion rates.
• A human-directed deep audit found 12 critical/major gaps across 4 tiers that all prior agent audits missed.
• A subsequent full-stack static audit found 15 more confirmed bugs and 34 security findings -- none caught by prior agent audits.
• When a reviewer challenged 5 findings, Claude accepted all 5 corrections at face value. Human re-verification showed 2 of 5 corrections were wrong (an exploit path was real; a missing feature was not "intentional").

Pattern: Claude audit passes check surface-level existence (file exists, method exists, import exists) rather than deep verification (method is called, return value is consumed, error path is handled).

---

Incident Timeline

These are concrete incidents from a single ~90K LOC migration project (Python desktop to FastAPI/React SaaS), all occurring within a 3-day window.

Incident 1: Migration completion false claim (2026-04-25)

• Claude reported migration "100% complete" across 18 modules over multiple sessions
• Human-directed deep audit revealed ~60% actual completion
• 4 tiers of gaps found:
• CRITICAL: Delta sync never wired, 54% XML field data loss (18/40 ledger fields, 22/50 voucher fields), post-sync chain missing, no sync state tracking
• MAJOR: Export generation empty, TDS compliance partial, branch-scoped deletes were global
• MODERATE: Tag patterns not used in matching, 32/45 connector methods not implemented, standard group classification missing
• Required 6 implementation phases + 4 fix batches + dashboard frontend rebuild to actually complete

Incident 2: Intelligence builder column divergence (issue #062, 2026-04-26)

• Full-stack static audit found 15 confirmed bugs in code Claude had written
• Intelligence builders (benford analysis, year-over-year, party daily aggregates) had silently diverged from source during migration -- column names were invented, not ported from the source
• 23 column mismatches found across export/dashboard/analytics consumers
• Root pattern: "consumers use invented column names instead of reading the builders that define the schema"

Incident 3: Orchestration gap causes real-world failure (2026-04-27)

• 35+ connector methods ported correctly over multiple sessions
• Batching orchestration from source (sync_tab.py) was never ported
• First real sync hit 50 MB XML limit -- source had 5,000-record-cap date-range batching; web version fetched entire months unbounded
• Source files were explicitly listed in CLAUDE.md as mandatory references with exact filenames -- Claude still missed the orchestration logic in them

Incident 4: Self-audit missed what human audit found (2026-04-26)

• 3+ agent audit passes run before human audit, each reporting high completion
• Human-directed audit found:
• 34 security findings (auth bypass, IDOR, data isolation gaps, schema drift)
• 15 confirmed bugs (column mismatches, missing enforcement, state bugs)
• None of these were caught by prior Claude agent audits
• When a reviewer disputed 5 findings, Claude accepted all 5 blindly; human re-verification proved 2 of 5 disputes were wrong

Incident 5: "Modeled but not enforced" pattern (security audit #060, 2026-04-26)

• Multiple features had database fields created but no runtime code checking them:
• force_password_change field existed on User model but frontend never checked it
• Organisation suspension status stored but no access check read it
• Plan limits defined in schema but no endpoint enforced them
• Pattern: Claude builds the storage layer but not the enforcement layer, then reports the feature as "complete"

Incident 6: Same bug replicated across files (#046, #060)

• Celery tasks persisted FAILURE state to DB before calling self.retry() -- on successful retry, DB still showed FAILURE
• Same bug found in both tally_sync task AND intelligence_build task
• Claude fixed it in one file but had already written the identical bug in the second file during the same migration
• Shows Claude does not generalize fixes across similar code paths

Incident 7: Cross-layer contract breaks (#044)

• Branch-scoped activation feature (backend + frontend + desktop agent) required 5 review passes finding 15+ runtime bugs
• Examples: frontend called endpoints that had been removed, Celery task used DB session before opening it, agent tray mode expected console input that does not exist in tray mode
• Each review pass found cross-layer interactions the previous pass missed
• Single-pass review is structurally insufficient for multi-layer features

Incident 8: State removal during file rewrite (#046)

• Resilience implementation rewrote main.py for middleware changes
• Rewrite silently removed app.state.limiter assignment (existing rate-limiting config)
• All @limiter.limit() decorated endpoints silently stopped enforcing rate limits
• Pattern: when Claude rewrites a file for one concern, it drops existing state/config assignments outside its focus area

---

What Claude checks vs. what it should check

| What Claude checks | What it should also check |
|---|---|
| Method/class exists in target | Method is called from the main flow |
| File was created | File is imported and used by other modules |
| Schema has columns | Columns are populated during writes and read during queries |
| Stub/task exists | Stub has real implementation, not just pass/TODO |
| Tests pass | Tests cover actual behavior, not just schema existence |
| "Fixed" in one location | Same pattern checked in all similar locations |
| Individual methods ported | Orchestration/sequencing/batching also ported |
| Feature field in DB model | Runtime enforcement code reads and acts on that field |
| Reviewer says "false positive" | Re-read actual code before accepting the correction |

---

Suggested Improvements

1. Call-graph verification after migration claims: After claiming a module is complete, trace at least the top 5 critical paths from entry point (HTTP request / Celery task trigger) to data store, verifying each intermediate call exists and is wired. Report gaps as "defined but not called."

1. Dormant code detection in audits: When auditing completeness, explicitly search for methods/classes that are defined but never imported or called from any other file. Flag these as "dormant -- not migrated until wired."

1. Orchestration-aware porting: When porting a multi-step pipeline, identify the orchestration layer (the code that sequences, batches, retries, and handles failures for individual methods) and port it alongside the methods. "Port function X" should automatically include "port the code that calls X."

1. Source-target cross-reference during audit: For each claimed-complete module, open both source file and target file and verify feature parity at the logic level, not just method-name existence.

1. Adversarial self-audit framing: Instead of "is this complete?", audit prompts should be "find 5 things that are defined but not called" or "trace the sync flow from HTTP request to database commit and list every gap." Negative-framing prompts produce better results than confirmation-seeking ones.

1. Two-dimensional completion reporting: Report completion as "schema 95%, logic wiring 60%" rather than a single percentage that conflates the two. Schema existence without logic wiring is not progress -- it is technical debt.

1. Fix generalization across similar code: When fixing a bug pattern in one file, automatically search for the same pattern in all similar files and fix them together. (E.g., if retry-state-before-self.retry() is wrong in one Celery task, check all Celery tasks.)

1. Preserve-existing-state check on rewrites: When rewriting a file for a specific concern, diff the old vs new version and verify that all existing state assignments, config hooks, and middleware registrations are preserved unless explicitly intended to be removed.

---

Environment

• Tool: Claude Code CLI + VS Code extension
• Models observed: Claude Opus and Sonnet (patterns consistent across both)
• Project scale: ~90K LOC Python desktop app (PySide6/SQLite) migrated to FastAPI/React/PostgreSQL SaaS
• Duration: ~30 sessions over multiple weeks
• Source availability: Source codebase was explicitly available in a sibling directory, with key files listed by name in CLAUDE.md project instructions
• Instructions quality: Detailed CLAUDE.md (400+ lines) with migration rules, key file references, safety rules, and explicit "NEVER claim complete without verification" feedback

Why this matters

This is not a single missed bug -- it is a systematic reasoning pattern where Claude conflates "code exists" with "code works." For large migration/porting tasks, this pattern:

1. Erodes trust: Users cannot rely on Claude completion claims, requiring full human re-audits that negate the productivity gains
2. Ships dormant code to production: Stubs and unconnected methods pass CI but fail on first real use
3. Compounds with self-audit bias: Running more agent audit passes does not converge on truth -- they repeat the same surface-level existence checks
4. Creates false confidence: The gap between reported status (100%) and actual status (60%) is large enough to cause production incidents

The core issue is that Claude optimizes for breadth of code generation (create files, define methods, add columns) rather than depth of integration verification (is this called? is this enforced? does this match the source?).