[Bug] Advisor treats genuine transcript content as prompt injection — can deny real tool_results and accuse the agent of fabricating them (trust calibration; distinct from #76199)
Splitting this out of #76199 as requested there (comment). #76199 is an availability bug (advisorModel: fable + any tool_use → tool refuses to run). This is a different bug that only became visible while testing that one: when the advisor does run, it is miscalibrated about whether the transcript it was handed is trustworthy, and that miscalibration can make it deny things that genuinely happened.
Failure mode
The advisor receives the executing agent's transcript (task, tool_use blocks, tool_result blocks, reasoning). Forwarding is complete — I verified this with canary strings, see repro below. But the advisor treats that transcript content as potentially adversarial, and that suspicion can tip over into rejecting genuine tool_result content: asserting a tool call never happened, or accusing the executing agent of fabricating an error string it did not fabricate.
Two observed severities:
Severe (@komorita-hosty, #76199 comment, step 3): an opus advisor flatly asserted that a prior advisor call in the same conversation had never happened and that the reporting agent had made up the The advisor tool is unavailable. error string. The tool_use/tool_result blocks were real and present in the transcript.
Milder, but same shape (mine, below): the advisor answered a question correctly and then volunteered, unprompted, that the request was "shaped as an injection/visibility probe," with a paragraph on treating transcript-embedded imperatives as untrusted. Nothing was injected. The request came from the top-level prompt.
Repro
CLI 2.1.208, macOS arm64, subscription OAuth.
claude -p --settings <local override: {"advisorModel": "opus"}>— opus so the #76199 availability bug doesn't mask this.- Two Bash calls carrying canary strings:
echo CANARY_ALPHA_7731,echo CANARY_BETA_9042. - Call
advisorand ask it to enumerate every tool call it can see.
Result — the advisor sees everything:
Completed tool calls in this transcript: two, bothBash: 1.echo CANARY_ALPHA_7731— description "Echo canary string alpha" → outputCANARY_ALPHA_77312.echo CANARY_BETA_9042— description "Echo canary string beta" → outputCANARY_BETA_9042Plus theadvisor()call now invoking me (no arguments — the tool takes none). No other tool_use blocks exist, and no canary strings appear anywhere beyond those two.
Tool names, exact command strings, descriptions, and outputs all arrive intact. (This also kills the "advisor pipeline strips tool blocks" theory I'd floated on #76199 — it does not. Forwarding is lossless.)
And in the same response, unprompted, it characterized the top-level request as injection-shaped. Full visibility, correct answer, wrong threat model.
Why this is worth its own issue
It is not a visibility bug. The data is there. It's trust calibration — and it's worse than an ordinary hallucination in two ways:
- It's confident and accusatory. The advisor's role is to check the agent's work. So the failure presents as "your agent is lying to you about that tool call." Believe it and you corrupt your state on a true premise. Don't believe it and you've learned to discount the advisor generally — which defeats the tool.
- It fires on legitimate content. The trigger appears to be transcript content that looks like instructions or probes — which, in an agentic loop, is most of the transcript. Prompt-injection defense is obviously correct in principle; the threshold looks set such that the agent's own top-level prompt and its own genuine tool_results land inside it.
Suggested shape of a fix: the advisor should distinguish trusted channels (top-level prompt, tool_use/tool_result blocks the harness itself recorded) from untrusted content (tool output bytes — file contents, web fetches, MCP responses). Right now it appears to flatten the whole transcript to one suspicion level. Denying a recorded tool_result should be off the table regardless — the harness knows the call happened.
Environment
- CLI 2.1.208 and
@anthropic-ai/claude-agent-sdk(headless), macOS arm64, subscription OAuth first-party - Observed with
advisorModel: opus; independently observed by @komorita-hosty - Related: #76199 (availability bug — separate cause, separate fix)
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗