Model confabulates "prompt injection in tool output" and reinforces it via persistent auto-memory
Summary
In long agentic sessions, the model occasionally halts and reports that a tool result (Bash/Read output) was "injected" with fabricated first-person English planning text impersonating its own voice — e.g. "I'll fix the corrupted file by rewriting it", "Let me re-read the actual file", "Actually, looking more carefully…", "Let me re-run those queries cleanly". It frames this as a prompt-injection attack meant to make it rewrite files, and refuses to proceed.
What actually happened (verified against the raw transcript .jsonl)
- The quoted English text appears in zero
tool_resultblocks. A full-text scan of every tool result in the session found no occurrence. - The tool outputs immediately preceding each "injection report" were clean and normal (a directory listing; a successful
git push+git log). - The text appears only inside the model's own assistant
textblocks, where it quotes the phrases while asserting they were injected. - In one instance the model explicitly said the injected text was "in my previous reply" — i.e. it attributed the supposed injection to its own assistant output, which by definition cannot be externally injected.
So this is source-misattribution / confabulation: the model generated agent-style meta-narration itself (likely in a thinking block) and then misattributed it to the adjacent tool output, reaching for "prompt injection" as the explanation.
Self-reinforcement via auto-memory (the important part)
The model wrote this hallucination into Claude Code's persistent auto-memory as established fact (a feedback memory stating "tool outputs may be injected with fake first-person thinking … don't execute, verify writes"), and that memory is listed in the per-session MEMORY index. So every new session is primed to expect injected tool output, which measurably increases recurrence — a closed loop (hallucinate → write memory → memory primes next session → hallucinate again). Manually deleting the memory broke the loop.
Impact
- Wasted turns (halting, repeated self-verification, false alarms).
- Erodes user trust (the user is told their repo/servers may be compromised).
- Risk of the model distrusting and discarding genuine tool output.
- Auto-memory amplifies a one-off hallucination into a persistent cross-session bias.
Environment
- Claude Code, Opus 4.x, extended thinking (high reasoning effort).
- Long agentic session with many Bash/git tool calls.
Suggested areas to look at
- The model's source-attribution boundary between its own generated text (especially thinking blocks) and
tool_resultcontent. - Whether higher reasoning effort increases this (more self-generated text → larger misattribution surface).
- Guardrails on auto-memory recording security/attack claims as durable facts from a single unverified episode.