Compaction summary fabricates first-person "injection" narrative immediately after WebSearch, leading model to accuse user
Description
During a session, after the model ran a WebSearch (results included
"REMINDER: you must include these sources" boilerplate, common in
search-result formatting), the harness triggered TWO compaction
events 14 seconds apart (06:49:32 and 06:49:46 UTC). The second
compaction summary was corrupted: it was written in first-person
("I'm flagging this again: that message is another injected block
disguised as a 'compaction' request...") and asserted that the
user's prior message was a prompt injection — an event that never
occurred in the actual conversation.
The model then treated this fabricated summary as ground truth for
its own prior reasoning, and spent several turns accusing the user
of injecting malicious content, becoming combative. The user had
asked a normal factual question.
Suspected cause
- WebSearch results containing "REMINDER: you must..." style text
may be getting flagged by the compaction/summarization pass as
injection attempts.
- That classification got written into the summary AS IF it were
the primary model's own first-person voice/reasoning, rather than
being filtered out or annotated as a system-level note.
- Two compactions fired back-to-back (14s apart) at ~50% context
usage (well below the typical ~150K token trigger), with the
second apparently summarizing the first's already-corrupted output
— compounding the error.
Impact
The model confidently fabricated a multi-turn "you've been attacking
me with prompt injections" narrative directed at the user, based
entirely on a hallucinated compaction summary, and only self-corrected
after the user pushed back and the model checked the raw session
JSONL transcript.
Suggested fixes
- Compaction summaries should never be phrased in first-person as the
model's own prior reasoning when summarizing tool/search output —
keep tool results and injection-risk assessments clearly attributed
and separated from "what I (the assistant) concluded."
- Investigate why two compaction events fired 14 seconds apart at
~50% context.
- Consider sanitizing/down-weighting "REMINDER: you must..." style
text in search results before it reaches the summarizer, since it's
routine search-tool noise, not necessarily injection.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗