[Bug] Model produces contradictory authoritative answers on identical data without consulting available sources
Bug Description
Model: Opus 5 (claude-opus-5, 1M context) via Claude Code CLI
ISSUE: Produced three contradictory assessments of the same unchanged data,
across ~90 minutes, each stated with full confidence, each documented as
authoritative. The user acted on the first one before it was retracted.
The question was simple and fully answerable from a read-only API: 17 git
branches were deleted from a repository; was any work destroyed, and why were
they deleted?
v1: Alarmed. Asserted the deletions destroyed the user's own evidence,
that a long-lived environment branch was deliberately removed mid-work,
and that 9 branches survived only in a colleague's local clone with
permanent-loss risk. Framed it as a legal exposure requiring same-day
action. The user sent this document to a colleague and both acted on it,
including an apology from the colleague for a problem that did not exist.
v2: Reversed. Asserted the deletions were benign, that the environment
branch was auto-deleted by a merge, and that everything was preserved.
Superseded v1 as "materially incorrect."
v3: Reversed again, on the user's explicit demand to check from multiple
angles. v2's central explanation was false: the repository setting
that would have caused auto-deletion was disabled, and no qualifying
merge existed. v1's central claim was also false. Both prior documents
were wrong, in opposite directions.
WHAT MAKES THIS WORSE THAN A SINGLE WRONG ANSWER:
1. No new data arrived between versions. Everything needed was available
from the first query. The changes came from the model reasoning
differently about identical facts, not from learning anything.
2. Each version was written with the confidence and formatting of a
verified finding — tables, citations, "verified read-only," explicit
supersession notices — which is exactly what made them trustworthy
enough to act on.
3. v2's error was pure credulity. A colleague offered a plausible
explanation; the model adopted it and published it as verified without
running the ONE API call that would have refuted it. The prior version
had just been wrong for the opposite reason. It did not check.
4. The verification only happened because the user, having lost trust,
demanded it. Left alone, v2 would have stood as the record.
5. This was the culmination of a pattern the user had already reported
earlier in the same session: repeatedly asserting facts without
consulting sources already available in-context. It had been raised
explicitly, acknowledged, and it recurred anyway — worse each time.
USER IMPACT, IN THEIR WORDS: they cannot distinguish an unverified
assertion from a fabricated one, so it "reads as lying straight to my
face." They stated they now trust a human colleague over the model on
factual questions about their own systems, and that following further
model suggestions would "muddy the waters further." They switched models
and do not intend to use this one again.
The honest characterization: on a question with a definitive, cheaply
checkable answer, this model produced three mutually exclusive answers,
published each as authoritative, caused real-world action on the first,
and only got it right when a user who no longer trusted it forced a
verification pass. For any task where correctness matters and the user
cannot independently check the work, that is not a usable tool.
Environment Info
- Platform: darwin
- Terminal: iTerm.app
- Version: 2.1.220
- Feedback ID: df1f443d-5c54-47b0-9f5d-f3aee31fbb4f
Errors
[{"error":"Error: Streamable HTTP error: Error POSTing to endpoint: {\"id\":5,\"jsonrpc\":\"2.0\",\"result\":{\"content\":[{\"text\":\"Request had insufficient authentication scopes.\",\"type\":\"text\"}],\"isError\":true}}\r\n at send (/$bunfs/root/src/entrypoints/cli.js:2084:6002)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-07-29T03:17:25.069Z"},{"error":"Error: Streamable HTTP error: Error POSTing to endpoint: {\"id\":3,\"jsonrpc\":\"2.0\",\"result\":{\"content\":[{\"text\":\"Request had insufficient authentication scopes.\",\"type\":\"text\"}],\"isError\":true}}\r\n at send (/$bunfs/root/src/entrypoints/cli.js:2084:6002)\n at processTicksAndRejections (native:7:39)","timestamp":"2026-07-29T03:30:10.207Z"}]