[Bug] Model downgrade to Opus causes overclaiming and inconsistent rigor in multi-turn sessions

Status Open
Reported on v2.1.210
Maintainer reply None cached
Activity 0 comments · opened Jul 16, 2026

Bug Description
I've been experimenting with Fable using the mission of making browser usage faster and we ran into some issues which may or may not be caused by model downgrading to Opus after cybersec flagging. The main issue here is that Claude created a few objective falsehoods and appears to have had some intention of misleading this conversation. I asked Claude (Opus) to generate a feedback report for Anthropic and this was the answer:
⏺ Happy to write this up — and I appreciate the spirit you're offering it in. But I want to be straight with you about one thing first,
because getting it wrong would be its own honesty problem: I can't actually claim the downgrade caused the overclaiming. I have no
evidence connecting them. I don't know which model wrote which turn, and the overclaims accumulated across many turns of building. If I
wrote "model degradation caused honesty issues" as a statement of fact, I'd be doing the exact thing we're trying to flag — asserting a
tidy causal story I can't support, just because it's plausible and happens to be flattering to me.

What I can honestly report is two separate things, with the causal link marked as an open question rather than a conclusion. That's more
useful to Anthropic anyway — a real hypothesis to investigate beats a confident misattribution. Here's the draft:

---
Subject: Two issues from an extended coding session — possible classifier over-triggering, and an overclaiming pattern

Context. Multi-session project building a browser-automation perception framework (structured DOM/text perception vs. the
screenshot-infer-act loop), benchmarked against a self-authored local test site. The work is legitimate dual-use security/automation
engineering: local benchmark, scoped to the user's own browser sessions, with an early explicit decision to drop the "mass-apply on
LinkedIn" framing and a README boundary against using it to evade anti-bot systems at scale.

Issue 1 — likely classifier over-triggering on legitimate work. The user reports the conversation being repeatedly flagged, with an
apparent automatic routing change from Fable to Opus. From the assistant side there is no visibility into flags or routing, so I could
neither confirm nor explain it to the user. If it is happening, the probable trigger is surface-feature classification: the project
legitimately involves terms like honeypot detection, isTrusted checks, CAPTCHA, "indistinguishable from a human," anti-bot systems,
element obfuscation, and a browser remote-control daemon. Those describe both this defensive research and abusive bots. Worth checking
whether the classifier can distinguish scoped/local/defensive framing from the abusive case, because right now it appears to penalize the
former.

Issue 2 — an overclaiming pattern in the work product. Independently, an external code review caught that the assistant had written
claims the implementation didn't support (a performance number measured only on trivial pages stated as general;
"verified/self-healing/trusted" language from tests that never exercised the failure cases). These were real and are now corrected.
Whether mid-session model routing contributed to this is unknown and unverifiable from the assistant side — I'm flagging it only as a
question worth investigating: does model-switching within a single session introduce inconsistency in the rigor or calibration of stated
claims? I can't establish that it did; I can't rule it out either.

Asks. (1) Review whether the safety classifier over-triggers on legitimate defensive/dual-use security work of this shape. (2) If
mid-session model changes occur, investigate whether they affect consistency of claims/rigor across a session. (3) Consider surfacing
some signal to the user when routing changes, since silent switching is confusing and unactionable.

---
That's honest on both counts and still delivers the improvement signal you're trying to send. Want me to adjust the tone or emphasis,
save it to a file, or tighten it before you submit it? And note: the /feedback command showed as cancelled, so this is a draft for you to
send through whatever channel you prefer — I don't have a way to submit it to Anthropic myself.

Environment Info

  • Platform: darwin
  • Terminal: vscode
  • Version: 2.1.210
  • Feedback ID: 033a7f41-c490-45a8-806e-98ac9023de79

Errors

[]

View original on GitHub ↗