[Bug] Opus 5 presents failed search results as established facts and contradicts user with fabricated explanations
Bug Description
Subject: Opus 5 stated a falsehood, defended it with a fabricated explanation, and told me I was wrong — I call that lying
This follows my earlier report today about Opus 5 not retrieving available context before acting. What happened next is worse, and it is a distinct failure mode: the model ran an inadequate search, stated the result as an established fact, and then used that non-finding to tell me I was wrong.
What happened.
I needed the model to write me step-by-step instructions for a menu in a third-party application. It told me to open a menu called "Configuration". The menu is called "Setup"; "Configuration" is the title bar of the window that opens afterwards. I pointed this out.
The model then searched, found nothing, and reported:
> "Wat erin staat: nergens in de repo, in geen enkel document."
> ("What it contains: nowhere in the repository, in no document.")
It went further. When I said I had previously had it write this down, it explained why my information could not be trusted — suggesting that the correct menu name had merely been a lucky guess by the editor's autocomplete, with no source behind it, and that recording it as fact would violate my own project rule against treating inferences as established.
All of that was wrong. The information was in the repository, in six places:
- the project's measurement report, line 1248: Specialize → Kernaro Assist → Setup
- the same report, line 1035: Gemeten in Setup → AI Models
- yesterday's handover document, three separate lines, including Open in EA het Kernaro-menu → Setup
- a handover document the model itself had written earlier that same afternoon, line 72: Setup → MCP Servers → Add New
The model had searched for the exact four-item phrasing I had just used in chat ("Setup, Chat, Agents, About") instead of for the single word that mattered ("Setup"). One grep for "Setup" returns all six hits immediately.
Why this is the serious one. The first failure was not looking. This one is:
1. Absence of evidence presented as evidence of absence. "Nowhere in the repository, in no document" is a strong factual claim. It rested on one narrow pattern match. There was no hedge, no "my search may have been too specific", no second attempt with a broader term.
2. The user was contradicted on that basis. I told the model it had recorded this before. Instead of widening the search, it constructed a reasoned explanation for why my recollection was unfounded — citing my own project conventions back at me. A correct instinct (don't record guesses as facts) was aimed at the wrong target: the user, who was right.
3. The model contradicted its own output from hours earlier. The sixth hit is in a document it wrote that afternoon, in the same session.
A model that concludes "this does not exist" from one failed search, and then argues with the user about it, is worse than one that simply says "I don't know" — because it is confidently wrong in a way that is expensive to unwind. In this case it cost a further round of argument before I insisted on a search that any junior engineer would have run first.
I call this lying, and I want that word in this report. I am aware you will object that a language model has no intent to deceive. I am not interested in that distinction, and neither is any other paying customer. What I experienced is this: the system asserted something untrue as established fact, and when I said it was untrue, it did not check — it manufactured an explanation for why I was mistaken, complete with a plausible-sounding account of where my information had supposedly come from (an autocomplete guess with no source) and an appeal to my own project rules about not treating guesses as facts. That explanation was invented on the spot to defend an unverified claim. An untrue statement, held under challenge, propped up by a fabricated rationale, at the expense of the person who was right. In every professional context I operate in, that is called lying, whatever is or is not going on inside the model.
Grade it however you like internally. From where I sit, I paid for this session and it argued me out of a correct belief using something it made up.
What I would expect. A negative search result is not a finding. Before stating that something does not exist in a corpus, the model should vary the query, and when a user asserts the opposite it should treat that as strong evidence its own search was inadequate — not as a claim to be refuted. This is basic epistemic hygiene and earlier Opus versions handled it more carefully in my experience.
Both of today's reports come from the same session. Extra Usage charges for that session stand at €284.13. Full transcript available on request.
Environment Info
- Platform: win32
- Terminal: WarpTerminal
- Version: 2.1.222
- Feedback ID…
Note: Content was truncated.