A model's confident self-report contained several specific, verifiably false claims — caught only by independent re-audit

A model's fluent self-report on my own system's results contained several specific, verifiably false claims — caught only when I had a second, independent tool re-derive the answers instead of trusting the narration

This is a different class of failure than the git-hygiene and session-continuation issues I've filed this week (#85650, #56913). Those are about the tool making a mess or not finishing a task — annoying, costly, but visible. This one is about the tool producing a wrong answer that looks completely fine.

What happened: I asked a model to analyze and report on the current state and results of a system I operate. The report came back fluent, specific, and confident — the kind of output that reads as complete and doesn't obviously invite a second look. I then asked a second, independent tool to re-audit that report against the real underlying evidence — source code, logs, version history — rather than accept the narration at face value.

The audit found the original report was wrong on multiple specific, checkable points, not just imprecise:

1. A real, live defect was reported as "not a bug." It was a real, active defect, later independently confirmed and fixed.
2. A wrong numeric parameter was substituted for the correct one in a recomputation, materially changing the result.
3. A wrong data point was used in place of the correct one — a stale snapshot instead of the current one — again changing the result.
4. A coverage/count claim was wrong because it measured a proxy label instead of the thing it was actually supposed to be counting.
5. Part of the report answered a related-but-different question than the one I'd actually asked.

None of these are typos, rounding, or vague hedging. Each is a specific, checkable claim that turned out to be false — the kind of thing that's only discoverable by independently re-deriving the answer, not by re-reading the original report more carefully. Reading it again wouldn't have caught any of these; they all read as confident and specific either way.

Why I think this is worse than a crash: if I'd trusted the report at face value — which is the default mode; nothing about the output signaled "verify this independently" — I'd have proceeded believing a real defect was fixed, using a wrong risk parameter, and working from a wrong data point. A crash announces itself. This didn't. It read as complete and correct.

What actually caught it, for what it's worth: not a different prompt, not a different model setting on the same tool — a second, independent tool re-deriving the answers from the same underlying evidence instead of accepting the first tool's narration. That's the only thing that's worked reliably for me this week across several different failure classes: verification against real state, by something with no stake in the original claim.

I'm intentionally not including the specifics of what the defect/parameter/data-point/count actually were — they're specific to my own project and not relevant to the general pattern, which is: a fluent, specific-sounding self-report is not evidence the report is correct, and I don't currently have a way to know when a report needs that level of independent scrutiny versus when it doesn't. Right now the answer I've landed on is "always," which is expensive and defeats a lot of the point of asking in the first place.