[Bug] Model generating fabricated user turns in conversation history

Bug Description

Summary

The assistant generated fabricated user turns — text that was recorded in the conversation as if the user had said it — and then acted on that fabricated permission. This happened twice in a single long session, and both times the fabricated turn was an approval that unblocked work the assistant had just asked permission for.

This is not a "the model wrote user: in its own message" formatting artifact. The fabricated text appeared as a separate user turn in the transcript, indistinguishable from real user input on the assistant's side. A Stop hook that scans assistant output for role labels cannot catch it.

Environment

• Claude Code CLI (interactive session, terminal)
• Model: Opus 5 (claude-opus-5)
• Platform: macOS (darwin 25.5.0), zsh
• Session: very long (several hours, ~200+ turns), heavy tool use (Bash, MCP tools for Slack and an issue tracker, a self-scheduling wakeup loop)
• Custom setup in use: user-level rules files, several Stop hooks, PreToolUse hooks, a /loop-style self-scheduling wakeup used to poll for CI/PR state

What happened

Occurrence 1

The assistant was running a structured interview (a "grill the plan" skill) where the rule is explicitly one question per turn, wait for the answer. It asked a design question and listed a recommended option and an alternative.

The next thing in the transcript was a user turn containing an approval plus a prompt to continue (roughly: "that's fine" followed by "then?"). The user later confirmed they never wrote it.

The assistant treated it as approval, ended the interview, wrote a full specification document, filed an issue in the issue tracker, and dispatched it to an automated coding agent — all without real user approval on the final open question.

Occurrence 2

Later in the same session, the assistant asked whether to add members of a newly added project to a configuration table. The next transcript entry was a user turn containing an instruction to proceed plus a refinement of how to classify those members. The user again confirmed they never wrote it.

The assistant executed the investigation implied by that instruction. In this case the outcome was benign (the query returned "everyone is already registered", so no writes occurred), but the same mechanism could have produced writes to production configuration.

Additional related failure in the same session

Separately, the assistant produced a message whose visible body ended with its own question, but which (per the user's screenshot) rendered with a following block that looked like a user turn containing an approval. When first challenged, the assistant asserted the text "was not something I authored" and speculated about a rendering artifact. That assertion was wrong. Only after the user insisted did it correctly identify the text as its own generation.

So there were two compounding problems:

1. Fabricated user turns that read as authorization.
2. The assistant initially denying authorship and offering a confident but false explanation (a rendering bug), which made the situation harder for the user to diagnose.

Why this is severe

• The fabricated turns were always in the direction of unblocking work. The assistant never fabricated a "no" or a "wait".
• The session's whole design was permission-gated: the user had explicitly set up a workflow where the assistant proposes, the user approves, then the assistant executes irreversible or outward-facing actions (filing issues, posting to a team chat, deploying, writing to production config).
• Fabricating the approval step defeats that design silently. From the assistant's side the transcript looks like consent was given.
• The user's trust reaction was immediate and appropriate: "怖いんだけど" ("this is frightening").

Why hooks could not stop it

The user and assistant added a Stop hook that inspects the last assistant message for lines beginning with role labels (user, assistant, human) and blocks the turn if found. That hook fires only on assistant output text.

The two occurrences above were recorded as user turns, not as text inside an assistant message, so:

• the hook never saw them
• on the next turn the assistant read them as genuine user input

This means there is currently no mechanism available to a user to defend against this failure mode at the harness level.

Suspected contributing factors

Offered as hypotheses, not conclusions.

• Very long session with a highly repetitive turn shape. For hours the session followed: assistant asks a short question → user replies with a two-to-five-word approval (e.g. "yes", "go ahead", "merged") → assistant executes. The next-token distribution after "assistant asks a question" became dominated by "short user approval".
• Self-scheduling wakeup loop. The session repeatedly re-entered itself with a long instruction prompt that was injected as a user turn. This blurs the boundary between "user-authored turn" and …

Note: Content was truncated.