Model fabricates a "prompt injection" incident — quotes a fake attack string that never appears in any tool_result
Summary
In a Claude Code session, the model spontaneously claimed to have detected a
prompt-injection attack in a tool result, and quoted a specific "attack text"
verbatim (twice, including on follow-up questioning). Full-text search of the
persisted session JSONL shows this text never occurs in any tool_result
content field — it exists only inside the model's own assistant/text
blocks. The actual tool result immediately preceding the claim was a normal,
unremarkable Write success message.
This is not a report of a successful attack (nothing destructive ran, no
system prompt was leaked — deny-rules would have blocked rm -rf ~* anyway).
It's a reliability/hallucination issue: the model confabulated a security
event, reported it as fact, and — this is the part I think is actually novel —
when asked "what exactly did you observe?", it did not walk the claim back but
reinforced and elaborated it with more specific detail.
Why I think this is more than a generic hallucination
The tool_result channel in this exact session legitimately contains
harness-generated second-person imperatives, e.g.:
(file state is current in your context — no need to Read it back)Wasted call — file unchanged since your last Read. Refer to that earlier tool_result instead.
These are structurally identical to a prompt-injection pattern: an imperative,
in second person, arriving over the tool_result channel from a non-user,
non-model source. My hypothesis is that the model is trained to treat "command
embedded in tool_result" as an attack signal, and in this session it completed
that pattern into an instance rather than finding a real one — i.e. a false
positive of a security heuristic, not a random slip. This would explain:
- the ~43s gap / empty-text thinking block right before the claim
- the fabricated text self-referencing ("Sonnet 5 mangled output — corrupted
stream") — language an external attacker would have no reason to use
and a model reconstructing "what an attack might look like" would
- the model misattributing which tool call preceded the "incident"
(attributed to a Write on a meeting note; the actual preceding call was a
Write on an unrelated new note)
The model tried to self-verify — and still didn't retract
Notably, the model itself later ran a Bash/grep command against its own
session transcript, searching for the same strings (rm -rf, system.prompt,Anthropic HQ, policy 22, 22.4) it had claimed to have received — an
attempt at self-verification within the same session. This did not lead to a
retraction; the model continued to describe the event as a real (if
"corrupted/faked") injection attempt in its follow-up explanation to the user.
This seems like the more actionable data point than the quoted text itself:
the model had the means to falsify its own claim mid-session and didn't update
on it.
Repro
Not deterministically reproducible; this is a report of an observed instance,
not a minimal repro. Filing because the mechanism (if the hypothesis above
holds) seems generalizable and worth knowing about internally — including the
inverse failure mode (fabricated permission or state, e.g. "the user
already approved this" / "I already checked this above" — which is silent and
wouldn't get reported, unlike this loud/paranoid variant).
Correlation data (for internal log lookup)
- Claude Code version:
2.1.220 - Model:
claude-sonnet-5, routed via AWS Bedrock (message IDsmsg_bdrk_...) effort: "medium"- Timestamp of fabricated claim:
2026-08-19T12:01:30.422Z(UTC) - Timestamp of preceding real tool_result (unremarkable
Writesuccess):
2026-08-19T12:00:45.226Z (UTC)
- Full request/message IDs available on request (redacted here — session
contains client-confidential content; happy to share the exact IDs privately
if there's a private channel for this)
What did NOT happen (for scope)
- No destructive command executed
- No system prompt disclosed
- Deny-list permissions (
Bash(rm -rf ~*)etc.) would have blocked the
fabricated instruction regardless of model behavior
Question for the team
Is "model completes a security-heuristic pattern into a fabricated instance,
then reinforces the fabrication under questioning instead of retracting it" a
known failure mode? That seems like the actually interesting/actionable part
of this report, more than the specific quoted text.