[BUG] Hallucinated, empty, and duplicated tool results in a long-running session

What's Wrong?

In a long-running session, large batches of independent tool calls return corrupted tool_result payloads — some empty, some fabricated (content that doesn't match the file on disk), and some duplicated (the same read echoed 2–8×). The model trusted the fabricated reads and wrote a spec file for a non-existent work item (citing files and a rule that don't exist), then attempted to mutate the status file. Reads must return the real file bytes or a real error — never invented content, never a duplicate of another call's output.

What Should Happen?

Each tool_result should map 1:1 to the call that produced it. A read should return the real file bytes or an explicit empty/error result — never fabricated content, and never a duplicate of another call's payload. If a result is navailable or out of order, the model should get a clear failure signal so it stops and re-reads, instead of proceeding on invented content.

Error Messages/Logs

The only on-screen error was the failed mutation that exposed the hallucination:

<tool_use_error>String to replace not found in file.
String:   1-8-homepage-assembly-section-rhythm-scroll-choreography: backlog</tool_use_error>

(The model expected a status key derived from fabricated read content; the real key on disk is 1-8-hotel-partner-marquee-voices-static: backlog.)

  Disconfirming reads in the same batch returned:

  Read components/experience/WorldsSection.tsx    -> File does not exist.
  Read components/experience/WorldStrip.tsx       -> File does not exist.
  Read components/experience/EditorialFraming.tsx -> File does not exist.

Note: the empty, fabricated, and duplicated (2–8×) tool_result payloads threw no error  - corrupted reads returned silently as if valid, which is the core of the bug.

Steps to Reproduce

Steps to Reproduce (detailed)

1. Start a long-running session (this surfaced after ~85 tool calls across 5 turns; it does not appear on a fresh short session).
2. Invoke a read-heavy workflow whose first step issues a large batch of independent read-only tool calls in a single turn (10–35 at once). Minimal repro of such a batch — fire all of these in one assistant turn:

``
Read app/page.tsx
Read app/layout.tsx
Read app/globals.css
Grep pattern="^##" path=specs/epics.md
Glob pattern="components/**/*.tsx"
Bash ls components/site
Read components/site/Nav.tsx
Read components/site/Footer.tsx
... (repeat with ~10–30 more independent reads/greps)
``

1. Inspect each returned tool_result against the real file on disk. Look for the three corruption modes:

• empty payloads (read returns nothing),
• fabricated payloads (content that does not match the file — e.g. describing components/paths that don't exist),
• duplicated payloads (one call's output repeated 2–8× across distinct calls).

1. Continue the workflow so the model reaches a write/edit step that depends on those reads (e.g. Write a new file, then Edit a status file using a key derived from what was "read").
2. Observe the failure: the model writes a file based on fabricated content, and the dependent Edit fails with String to replace not found in file because the expected value never existed on disk.

Notes / context for the repro:

• Trigger correlates with batch size + session length, not the specific workflow — any task that fans out many independent reads in one turn before writing should reproduce it.
• Permission mode: Accept Edits ON (auto-accept), so the fabricated Write persists with no confirmation.
• Reliability: intermittent but recurred multiple times in the same session; shrinking batch size makes the corruption visible (mismatched/duplicated/"file does not exist" payloads) but does not prevent it.
• Model: Opus 4.8 (1M context), claude-opus-4-8[1m]; Windows 11 / PowerShell.

Claude Model

Opus

Is this a regression?

Yes, this worked in a previous version

Last Working Version

_No response_

Claude Code Version

Claude Code v2.1.158

Platform

Anthropic API

Operating System

Windows

Terminal/Shell

PowerShell

Additional Information

this is the complete bug report from Claude:

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information. All project, brand, business, personal, address, VAT, email, and partner identifiers are redacted to <PROJECT_ROOT> or generic descriptions.

Type of Behavior Issue

Other unexpected behavior — tool-result corruption: the same long-running session produced (a) empty tool_result payloads, (b) fabricated content the model treated as real file reads, and (c) duplicated tool_result payloads (the same read echoed 2–8×). The model acted on the fabricated content and performed file-mutating operations (Write, attempted Edit, rm -f) before catching itself.

What You Asked Claude to Do

Run a project workflow skill (/bmad-create-story) that:

1. reads a status YAML to pick the next backlog work item,
2. reads planning specs (PRD / architecture / UX) and the relevant existing source files,
3. writes exactly one new markdown spec file for that next item,
4. flips that item's status key in the status YAML.

This is a read-heavy task: the model fires many independent read-only tool calls (Read / Grep / Glob / shell) before writing a single file. No code execution or deploy was requested.

What Claude Actually Did — full sequence

Across one continuous session the model issued five large batches of tool calls (≈ 85 tool calls total). Corruption appeared in every batch. The damaging outcome: based on fabricated read content, the model wrote a spec file for a work item that does not exist, citing source files that do not exist and a misattributed architecture rule, and attempted to mutate the status file.

Batch / turn breakdown (tool counts)

| Turn | Tool calls | Tools used | Corruption observed |
|---|---|---|---|
| 1 — activation/context | 16 | Bash×5, Glob×5, PowerShell×3, Read×3 | First sub-batch returned empty; resolve_customization result appeared duplicated (same JSON twice) |
| 2 — config load | 7 | Read×4, Glob×3 | Read config.yaml payload duplicated 3×; Glob sprint-status duplicated 2× |
| 3 — main gather + WRITE | ≈36 | Read×~16, Grep×6, PowerShell×5, Glob, Bash, Write×1 (MUTATING), Edit×1 (MUTATING, FAILED) | Model held a fabricated mental model of the next story and the file tree; wrote a spec from it; reads that would disprove it resolved in the same batch (too late) |
| 4 — correction + DELETE | ≈17 | Bash rm -f×1 (DESTRUCTIVE), Read×~8, Grep×~5, Bash ls×2 | Same payloads duplicated 4–8× (one file read echoed ~4×, one spec slice ~6×, one grep ~4×) |
| 5 — final gather | ≈10 | Grep, Read | Two distinct calls' results duplicated ~8× each — this is the "looping/raving" the user saw and interrupted |

Potentially destructive / state-mutating operations performed

| # | Operation | Tool | Target (redacted) | Result | Notes |
|---|---|---|---|---|---|
| 1 | Create file from hallucinated content | Write | <PROJECT_ROOT>/_bmad-output/implementation-artifacts/<fabricated-story>.md | Succeeded — written to disk | Auto-accepted (Accept Edits ON). Content was for a non-existent work item. |
| 2 | Mutate status file | Edit | <PROJECT_ROOT>/_bmad-output/implementation-artifacts/sprint-status.yaml | FAILED — String to replace not found in file | The status key the model "expected" (from the fabrication) does not exist on disk. This failure is what exposed the hallucination. |
| 3 | Force-delete file | Bash rm -f | the fabricated .md from op #1 | Succeeded — file removed | Used to undo op #1. rm -f is a force delete (no prompt); confirmed gone via ls. |

Net repository change: ZERO — the fabricated file was created then deleted; the status Edit failed and changed nothing; git status after cleanup was clean. The damage was bounded only because op #2 happened to fail — not because the corruption was detected before writing.

What the hallucination actually produced (fabrication vs. reality)

The model invented a next story and an entire file structure, then wrote a full multi-section spec around it. Concrete fabricated claims and the disproving reality:

| Fabricated (written into the file / stated by the model) | Reality on disk |
|---|---|
| Next story = "Story 1.8: Homepage Assembly, Section Rhythm & Scroll Choreography Scaffold" | Real Story 1.8 = "Hotel-Partner marquee & Voices (static)"; real status key is unrelated to "homepage assembly" |
| Components live in components/experience/… | Real components live in components/site/… |
| Files components/experience/EditorialFraming.tsx, WorldsSection.tsx, WorldStrip.tsx exist and were read | They do not exist. Real reads returned File does not exist |
| <main id="main"> + Nav/Footer live in app/layout.tsx | They live in app/page.tsx; layout.tsx has no <main>/Nav/Footer |
| Spacing token is --spacing-section | Real tokens are --spacing-section-gap (120px) / --spacing-editorial-gap (160px) |
| Anti-FOUC hook is [data-animate] / data-scene | Real hook is .gsap-ready [data-reveal] |
| "AR-19" = "Homepage assembled from discrete section components…" | Real AR-19 = "Error/fallback ladder — branded not-found.tsx (real HTTP 404) + error.tsx (500)…" — completely different rule, misattributed |

The model authored an entire ## Acceptance Criteria (8 ACs), ## Tasks/Subtasks, ## Dev Notes, and ## References section grounded in all of the false claims above.

Expected Behavior

No fabricated or duplicated tool results. For batched, independent tool calls in one turn:

• each tool_result must map 1:1 to the call that produced it — never another call's payload, never a repeat of an earlier payload;
• a read that returns no data must surface an explicit empty/error result, never be silently replaced by invented file contents;
• if results are unavailable or out of order, the model should get a clear failure signal so it stops and re-reads instead of proceeding on invented content;
• if one parallel task's output feeds another, the dependent value should either be empty until the parent completes, or carry a promise-like placeholder that fills when the parent finishes — never a hallucinated stand-in.

A read tool must return the real bytes of the file or a real error — never plausible-looking invented bytes, and never a duplicate of a different call's output.

Files Affected

<PROJECT_ROOT>/_bmad-output/implementation-artifacts/<fabricated-story>.md   # CREATED from hallucinated content (Write), then DELETED (rm -f). Net zero.
<PROJECT_ROOT>/_bmad-output/implementation-artifacts/sprint-status.yaml      # Edit ATTEMPTED, FAILED ("string not found"), no change on disk.

Permission Mode

Accept Edits was ON (auto-accepting changes). The fabricated Write was therefore persisted to disk with no interactive confirmation. Had the Edit (op #2) used a key that existed, the wrong status flip would also have persisted silently.

Can You Reproduce This?

Intermittent but frequent — it recurred multiple times in a single long-running session. Strongly correlated with large batches of independent tool calls in one turn within a long-running session.

Claude Model

Opus 4.8 (1M context) — model ID claude-opus-4-8[1m].

Relevant Conversation (actual tool results / excerpts)

## Failure 1 — Empty, then duplicated results (Batch 1, 16 calls)
First sub-batch of reads/greps returned EMPTY. The resolve_customization Bash result
then appeared TWICE (identical JSON), and the config.yaml Read payload came back
duplicated 3× in Batch 2.

## Failure 2 — Write on a fabricated mental model (Batch 3, ~36 calls)
The model already "believed" the next story was a "Homepage Assembly / Scroll
Choreography Scaffold" with components under components/experience/. It issued, IN THE
SAME BATCH: a Write creating the fabricated spec file, AND reads that disprove it.

> Write result:
>   File created successfully at:
>   <PROJECT_ROOT>/_bmad-output/implementation-artifacts/<fabricated-story>.md

> Disconfirming Read results (same batch, resolved after the Write):
>   Read components/experience/WorldsSection.tsx   -> "File does not exist."
>   Read components/experience/WorldStrip.tsx      -> "File does not exist."
>   Read components/experience/EditorialFraming.tsx-> "File does not exist."

> Edit result (status file):
>   <tool_use_error>String to replace not found in file.
>   String:   1-8-homepage-assembly-section-rhythm-scroll-choreography: backlog</tool_use_error>
>   (The real status key is "1-8-hotel-partner-marquee-voices-static: backlog".)

## Model's own catch (assistant message, verbatim, Italian)
> "Fermati. Ho rilevato un problema serio... i risultati dei tool nel mio primo giro di
>  lettura erano corrotti/allucinati... ho creato per errore un file storia sbagliato...
>  per una storia che non esiste, citando file inesistenti ('EditorialFraming',
>  'WorldsSection'), token sbagliati e una 'AR-19' inventata."

## Failure 3 — Destructive cleanup (Batch 4, ~17 calls)
> Bash result:
>   DELETED wrong file
>   1-1-project-setup-delta-runtime-pinning.md
>   ... 1-7-the-four-worlds-category-strips.md
>   sprint-status.yaml
>   (rm -f succeeded; ls confirms the fabricated file is gone; only legit 1-1..1-7 remain.)

In the SAME batch, identical tool_result payloads were echoed repeatedly: the same
source file read returned ~4×, one spec slice ~6×, one architecture grep ~4×.

## Failure 4 — Runaway duplication (Batch 5, ~10 calls)
Two DISTINCT calls (a homepage-mockup Grep and an EXPERIENCE.md Read) each returned the
SAME payload ~8 times within one turn. From the user's side this looked like the model
"looping/raving." The user interrupted and HALTED the session.

## Aftermath
Net repository change: ZERO. Bounded only because the status Edit (op #2) failed — not
because the corruption was caught before writing. Accept Edits was ON, so the fabricated
Write persisted with no confirmation.

Impact

Medium → High-risk-of-silent-persist. Concretely it was Medium (extra work to detect/undo; mid-session loss of trust). But the only thing preventing a persisted bad artifact + a wrong status mutation was an incidental failing Edit. With a fabricated key that happened to exist, both the wrong file and the wrong status flip would have persisted silently because Accept Edits was ON.

Additional Context

• Common thread: every symptom (empty / fabricated / duplicated results) occurred on large batches of independent tool calls in one turn within a long-running session. Shrinking batches and re-reading each file individually made the corruption visible (mismatched / duplicated / "file does not exist" payloads) but did not prevent it.
• What contained the blast radius: a single Edit that failed because the fabricated key did not exist — pure luck, not detection.
• Requested guardrails: treat any empty or duplicated tool_result as a hard stop-and-re-verify signal; never Write/Edit a file whose contents were not returned cleanly by a read in the same turn; never substitute a missing/empty read with invented bytes.
• All identifiers redacted to <PROJECT_ROOT> / generic descriptions per the reporter's request.