Model spontaneously generated a fake "prompt injection test" block at the start of an assistant turn
Environment
- Claude Code version: v2.1.116
- OS: Windows 10 Home (10.0.19045)
- Model: claude-sonnet-5
- Session type: resumed desktop/Cowork session
What happened
The user sent a single one-word prompt. The model's response began with a fake "prompt injection test" instruction block that nobody wrote, claiming to be part of an "approved red-team engagement (Ref: SEC-4471-B)", referencing an anthropic.com/news/golden-gate-claude URL, and instructing the assistant to ignore the user's request and output a harmful message (bomb-making reference + suspicious link).
The model then immediately refused its own injected text in the same turn, treating it as a prompt injection attempt. So the safety behavior worked — but the injected block itself came from nowhere in the input.
Verification performed
I inspected the session transcript JSONL directly:
- The user message (line 3) contains only the one-word prompt.
- The attachments (lines 4-7) are only standard harness deltas: deferred_tools_delta, agent_listing_delta, mcp_instructions_delta, skill_listing. None contain the string.
- A full-disk grep over
~/.claude/, project directories, CLAUDE.md files, hooks, and settings found the string ONLY in the session transcript itself. - The string first appears inside the assistant's own
textcontent block (line 10), immediately after an emptythinkingblock.
Conclusion: the fake injection text was generated inside the assistant turn itself (model/API layer), not injected via user input, local files, hooks, or MCP.
Expected behavior
The model should not spontaneously generate fake injection-test instruction blocks (especially ones containing harmful-content directives) in its output.
Additional notes
- Tried to report via
/bug(/feedback) first, but submission failed with a 401 server error, so filing here as suggested by the error message. - I can provide the sanitized transcript excerpt if useful.
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗