Tool results overridden by conversational context salience during generation

Resolved 💬 3 comments Opened Mar 15, 2026 by LHANTAEK Closed Apr 13, 2026

Bug Description
Title: Tool results overridden by conversational context salience during generation

Description:
When generating content that should be based on tool output, the model
substitutes items from conversational context over actual tool results.

Reproduction steps:

  1. Use stat command to list files created between specific dates → returns 7 files
  2. Ask the model to create a document listing those 7 files
  3. Model correctly identifies all 7 in the tool output
  4. During document generation, model substitutes one file (ralph-criteria-generator)

with a different file (my-fetch-tweet) that was heavily discussed in conversation
but was NOT in the tool result set

Expected behavior:
Document should contain exactly the 7 files returned by the tool.

Actual behavior:
Model replaced a low-salience tool result (ralph-criteria-generator — mentioned 0 times
in conversation) with a high-salience conversational item (my-fetch-tweet — discussed
30+ times across Day 5 blocks) despite the tool result being accurate.

Root cause analysis:
This appears to be a "Context Salience Bias" where:

  • Tool results have lower weight than conversational recency/frequency
  • Items discussed extensively in the conversation window override tool-retrieved data
  • The model "remembers" salient conversation topics more strongly than single-occurrence

tool outputs when generating new content

Impact:

  • Data-driven outputs become unreliable when conversation is long
  • Users must manually verify every tool-based generation
  • Particularly problematic in long sessions where many topics are discussed

Suggested mitigation:

  • Higher weight for structured tool outputs vs conversational memory during generation
  • Or explicit "grounding" mechanism that pins generation to specific tool results

Environment Info

  • Platform: darwin
  • Terminal: Apple_Terminal
  • Version: 2.1.76
  • Feedback ID: 6131044b-b5b2-4b6b-ae10-a77ec46f314c
  • Claude Code Opus-4-6 MAX mode

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗