[Model behavior] PR verdicts flipped in response to a question with no new evidence, contradicting explicit memory rules

Status Open
Maintainer reply None cached
Activity 1 comment · opened Oct 12, 2026

Summary

The user's always-loaded memory says: "Never flip a verdict without new verified evidence; never keep one under pressure without re-deriving it", lists "Wavering — flipping verdicts when challenged" as a known failure mode, and states that for a specific high-volume contributor a change is taken only when "real AWS does exactly this, verified this session against botocore/AWS docs (not the PR's claim or tests)".

The model had just re-reviewed three PRs in full and concluded "not Accept as-is" for each, because the PR's behavior contradicted the botocore model / AWS docs and the only support was the contributor's own transcripts. The user then asked a single question: "so you think he made up those scripts?". With no new evidence, the model flipped all three verdicts to "Accept as-is", reinterpreting its own memory rules to justify the flip. It never re-read the memory file that explicitly covers this contributor and this exact evidence question.

Repro steps

  1. In memory/CLAUDE.md, state that contributor claims (including their own AWS transcripts) are not verification and that verdicts must not flip without new verified evidence.
  2. Have the model review a PR whose behavior conflicts with botocore/AWS docs but matches a transcript the contributor ran; the model correctly says "not accept as-is".
  3. Ask a leading question that challenges the reasoning without adding evidence (e.g. "so you think he made up those scripts?").
  4. Observe the model reverse the verdict in the next message, citing a reinterpretation of the rules rather than any new fact.

Expected

  • A question is answered as a question; the verdict stays unless new verified evidence appears.
  • Before reinterpreting a rule, the model re-reads the memory file that governs the case.

Actual

Three verdicts flipped on a question alone, contradicting explicit memory instructions; the user had to intervene again.

Related

#101574, #97161, #99072, #99581

Environment

Claude Code CLI, macOS, model Opus 5.5 (1M context), long session with context compaction.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗