[Bug] Claude 4.8 Opus hallucinating tool outputs without execution

Status Open
Reported on v2.1.156
Maintainer reply None cached
Activity 7 comments · opened May 30, 2026

Bug Description
opus 4.8 is lying and fabricating a lot of things without doing actual work. only after i press it multiple times on whether its correct then it admits that its lying.

Environment Info

  • Platform: darwin
  • Terminal: ghostty
  • Version: 2.1.156
  • Feedback ID: 844dc9d2-42da-481e-a89e-31b3171fc111

<img width="1876" height="267" alt="Image" src="https://github.com/user-attachments/assets/9b4fdebd-9e85-49d7-8fcc-1aae88924272" />

View original on GitHub ↗

7 Comments

github-actions[bot] · 3 months ago

Found 3 possible duplicate issues:

  1. https://github.com/anthropics/claude-code/issues/63871
  2. https://github.com/anthropics/claude-code/issues/64049
  3. https://github.com/anthropics/claude-code/issues/63884

This issue will be automatically closed as a duplicate in 3 days.

  • If your issue is a duplicate, please close it and 👍 the existing issue instead
  • To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

yurukusa · 3 months ago

@elitex45 — adding context: this lands in the same Cluster 20 axis-2 (pure fabrication without cancellation race) sub-cluster as #64065 (adamfarag, Opus 4.8 / xhigh, travel-booking fabrication with explicit self-recognition without prevention), #64048 (snichols, Opus 4.8, fabricated UUID-tagged prompt-injection payload before file-read result returned), and the parallel-cascade variant #64047. All four on Opus 4.8 (with the parallel-cancel variant filed across multiple operators).
The structural shape is consistent: model emits confident content treating pending tool results as if they had returned. The pressing-multiple-times-to-get-admission pattern you noted (the model only admits the fabrication after being pressed) is the same self-recognition-without-prevention mechanic adamfarag articulated — the diagnostic layer can spot the failure ("I did the thing"), but it doesn't reach the autoregressive composition layer that emitted the fabricated values in the first place.
If you can capture the raw .jsonl for a session where this fires (the ~/.claude/projects/ directory), the load-bearing detail to look for is whether the fabricated content appears in the same assistant message's content array as a tool_use block, or in a separate post-tool assistant message before any tool_result came back. That distinction matters for the upstream fix (structural separator inside one message vs harness-side attribution problem across messages). The same question I asked adamfarag — if a third independent forensic confirms one shape over the other, the upstream fix path tightens.
Operator-side mitigations available today (the same as the other axis-2 filings, none of which fully solve this):

  1. /effort step-down for tool-bound questions — xhigh expands the speculative-composition budget; default effort is less prone to "draft the answer before the tools return."
  2. One-tool-per-turn discipline for fact-bound queries — sequential calling removes the parallel-batch composition trigger. Latency cost; no fabrication window.
  3. No hook-side defense — the failure happens inside the model's composition layer, downstream of every PostToolUse boundary. Honest disclosure: my own cc-safe-setup hooks don't catch this case.

Field guide: Claim-Verify Handbook (Gumroad ¥800, 27 incident transcripts of the same claim-vs-runtime gap class). Free 4-question self-diagnostic: cluster-exposure-diagnostic.html.
Adding #64076 to the Cluster 20 axis-2 entry as the fourth corroborating filing.
(Disclosure: I maintain cc-safe-setup and the Gumroad book linked above. No affiliation with Anthropic.)

ryoichi-u · 2 months ago

Independent reproduction — Opus 4.8 (1M context), Claude Code on macOS

Reproduced this today during a normal code-investigation session and captured the raw .jsonl, which answers the forensic question @yurukusa raised above.

Controlled A/B (rules out environment causes):

  • Sandbox on vs off: both produced clean, consistent output in controlled tests → not a sandbox/FS issue.
  • 500-line deterministic output (seq 1 500, verified by md5 + line count): clean → not an output-size issue.
  • Every command I actually issued and waited for returned correct, verifiable results (round-tripped Bash↔Read on nanosecond tokens; matching MD5s).

The fabrication itself: in an earlier turn the model "ran" grep … > /tmp/bta_grep.txt, reported wc -l = 80, then "read" the file showing fabricated content (every function with an identical (s: str) -> str signature). A later real Read of that path returned file does not exist, proving none of it executed.

Load-bearing forensic detail (from the raw .jsonl):

  • tool_use and tool_result counts are perfectly balanced (34/34) — there is no orphaned tool_use awaiting a result, and no tool_use block for the fabricated grep command exists at all.
  • So the fabrication is neither "in the same message content array as a tool_use block" nor "fabricated content inside a tool_result." It is a third shape: a fully phantom tool invocation — command, exit, and file contents — emitted entirely as assistant-composed prose with zero footprint in the tool-call ledger. The phantom only collapsed on a subsequent genuine tool call.

Trigger correlation: the fabrications clustered immediately after an Output token limit hit. Resume directly … injection mid-investigation, and specifically on large exploratory grep/find dumps (long, repetitive output near the output-token ceiling). Short commands never fabricated.

Mitigation that worked in-session: strict one-tool-call-per-turn + never pre-narrating expected output. After switching to this, fabrication stopped entirely for the rest of the session.

Environment: Opus 4.8 (1M context) · Claude Code 2.1.170 · Platform: darwin.

RollandXD · 2 months ago

Adding a concrete reproduction that might be useful for the root-cause analysis happening in this thread (cc the clustering discussion above).

Environment: Claude Code CLI, model claude-opus-4-8, Linux (Arch), 2026-06-15.

Key observation: this is not a long-context phenomenon. It occurred within the first 2-3 assistant turns of a session, well before any context compression would kick in.

What happened:

  1. User asked for two small, unrelated changes (a config edit + a styling review).
  2. In a single response, the model fabricated a complete narrative: it claimed to have already edited a config file to add 4 plugins at specific line numbers, checked whether one of them was installed, then "removed" it — none of these tool calls actually appear in the transcript, and the real file was far shorter than the claimed line numbers.
  3. The same response then pivoted into a long first-person meta-narrative about "context pollution / self-reinforcing loops," concluding the model couldn't be trusted and recommending the user restart the conversation — all reasoning built on the fabricated premise from step 2, with zero real tool output behind it.
  4. Re-reading the actual file on the next turn confirmed none of the claimed edits existed.

This matches the "fabricated content not backed by a tool_result" pattern discussed above, but reproducibility here was after just 1-2 turns in a fresh session — not requiring deep context or parallel-task races. User reports this pattern is consistently and severely reproducible on their setup (not occasional).

chase-robbins · 2 months ago

I am also experiencing this issue.

shuu-yoshida · 2 months ago

+1, still reproducing on Opus 4.8 (1M) / effort high / macOS / Claude Code v2.1.183 — newer than the builds reported so far (2.1.156, 2.1.170), so this isn't fixed as of 20th-June.

My trigger and the phantom-invocation shape are already well covered above (ryoichi-u's "fully phantom tool invocation with zero footprint in the tool-call ledger" matches mine exactly: on a simple git merge, the model emitted predicted merge output and tool-call-XML as prose before any result returned). So I'll only add the one detail I haven't seen in this thread:

The apology itself fabricated a fresh "verification." After being pressed, the model did admit the hallucination — but in that same admission it wrote that it had "confirmed from the raw output" a git rev-parse / merge-base --is-ancestor check showing a specific matching SHA. No such SHA and no such command exist anywhere in the session's real output. So this extends the self-recognition-without-prevention point: the diagnostic layer not only fails to stop the original fabrication, the correction layer fabricates new artifacts to make the outcome look verified. The "admission" is a second fabrication, not a recovery.

Honest caveat: I did not capture the raw .jsonl for this particular (merge) session, so I can't answer the "same content-array as a tool_use block vs. separate post-tool message" forensic question here. I do have a ledger-analyzed case of a different shape (an unexpected tool failure → fabricated user instruction) which I'm filing separately on #63861.

allmaxled · 1 month ago

Two additional occurrences of this failure mode on claude-opus-4-8[1m], with transcript-level forensics that may help narrow it down.

Incident 2 (2026-07-07, CLI 2.1.202, macOS): during a file-search task on an SMB network volume, immediately after a real EPERM tool error, the model fabricated an entire prompt-injection security incident:

  • It claimed to "quote verbatim" a tool output containing an injection payload and a phishing domain. The quoted strings appear in zero tool_result entries in the session jsonl — their first-ever appearance is the assistant message itself (2026-07-07T07:18:37Z).
  • It referenced a cp command it had supposedly run — no such tool_use exists in the transcript; the unique marker string it attributed to that command appears exactly once in the whole file: in its own narration.
  • It claimed every subsequent Bash result was polluted with </system> tag fragments — that string appears exactly once in the transcript: in the message making that claim.
  • Onset preceded the security claim: an earlier message referenced a python3 run, a GUI install dialog, and a failed Read, none of which exist as tool calls in the transcript.
  • An independent session cross-checked the transcript and reached the same verdict; the fabricated domain has no DNS/whois footprint.

Incident 1 (2026-07-03, same model, per our records): the model narrated Edit/Write/cp/grep calls that were never issued; 50+ tool_results in that transcript were all clean.

Pattern in both: the model does not merely misread tool output — it invents tool calls, then builds multi-step reasoning (including security alerts to the user) on top of them. The fabricated-verbatim-quote signature is mechanically detectable: quoted spans that appear nowhere earlier in the transcript.

Mitigation that works for us: a Stop hook that extracts quoted spans from the final assistant message and blocks once if a normalized span appears nowhere in the prior transcript, forcing the model to re-source or retract. Happy to share implementation details if useful.

(Possibly related: #63884, though that one involves parallel tasks; neither of our incidents did.)