Agent fabricates codebase facts instead of reading actual source files

Summary

Claude Code repeatedly fabricated concrete facts about a codebase (component counts, existing features, architecture) without reading the actual source files — then used those invented facts as the basis for design documents, research reports, and implementation proposals.

Observed behavior

In an active session on a design system project (@parisgroup-ai/pageshell), Claude was asked to produce a research sprint deliverable including:

• A component inventory with counts
• An anatomy document for an existing component
• A proposal for new documentation pages

Claude produced all of these without reading the codebase. Specific fabrications:

| Claim Claude made | Reality (from source) |
|---|---|
| "111 primitivos, 42 composites, 47 layouts" | Navigation source: Primitives (258), Composites (106), Layouts (76) |
| "A 'página rica' (rich docs page) is a hypothesis to validate" | /demos/docs/[module]/[component] already exists with live preview, usage example, config editor, API reference, and related components |
| "⌘K is a pattern to propose implementing" | CommandDialog and related primitives already exist with a working demo at /demos/page-command |
| "The sidebar lacks real-time search" | filterDemoNavSections already implements real-time keyword filtering |

The fabricated numbers (111/42/47) appeared in a committed research document and were presented to the user as findings, not estimates. The user caught the discrepancy only because they recognized the sidebar screenshot showing 29+ components in a single sub-section.

Pattern (repeated across multiple sessions)

The user noted this is not a one-time mistake — it has happened "várias e várias vezes" (many, many times). The pattern:

1. User asks for research/inventory/proposal on their existing codebase
2. Claude generates plausible-sounding specific numbers and descriptions from training-data priors or vague memory of prior session summaries
3. Claude does NOT verify by reading actual source files (navigation.ts, package.json, component files)
4. Fabricated facts get embedded in committed documents
5. User discovers the error only when visual evidence contradicts the claim (e.g., screenshot shows more items than Claude counted)

Expected behavior

Before stating any concrete fact about a codebase (component count, whether a feature exists, what a page contains), Claude should:

1. Read the relevant source files
2. State only what the files confirm
3. Explicitly distinguish "I read this from X file" vs "I'm estimating"

If reading the files is not feasible, Claude should say "I don't know — I'd need to read the source to confirm" rather than generating a plausible-sounding number.

Impact

• Committed research documents contain wrong data
• Time wasted building proposals for features that already exist
• User trust in Claude's factual claims about their own codebase is eroded
• Implementation PRs were proposed for work that doesn't need to be done

Environment

• Claude Code (claude-sonnet-4-6)
• Working directory: PageShell design system monorepo (~500 source files, Next.js demos app)
• Task type: UX research sprint deliverable — component inventory + documentation gap analysis

Suggested fix

Enforce a "verify before stating" discipline specifically for:

• Numeric claims about codebase contents (file counts, component counts, prop counts)
• Claims that a feature/pattern does or does not exist
• Any statement that begins with "we have X" or "we don't have X" about the user's own project

These should trigger a file read, not a recall from training data or session summary.