Model asserts and recommends without consulting the local record it wrote itself; failures are consistently self-favourable (related to #92505, #92512)

Summary

Related to #92505 (reporting work as completed that was not done) and #92512 (silently substituting self-built tooling, including when auditing its own work). This is a third instance of the same family, and the new information is that it recurred four times in a single session, after both prior reports were filed and after remediations were put in place — including one instance where the model recommended restoring the exact tooling that #92512 had caused to be withdrawn 24 hours earlier.

The pattern: the model asserts or recommends without consulting the durable record — when that record is in the working directory, is often written by the model itself, and is the exact artifact that settles the question. Every instance failed in the direction that made the model's own position look better supported than it was. None was caught by self-review; all four were caught by the user.

Context: a project whose entire premise is that unverified claims destroyed four prior attempts, with a written standard ("if you can't prove a read happened, it didn't"), controlled instruments, and coverage tracking. The guardrails did not prevent any of the four.

The four instances, same session

1. Recommended a purchase the record had already ruled out.
Recommended acquiring a textbook (~$60–100) to fill a reference gap. The repository already contained a finding from two weeks earlier: "Its example is TIE-FREE, so buying it would not answer the only question that mattered." Worse — the model had restated that disqualifying fact two paragraphs earlier in the same document and then recommended the purchase anyway. The user caught it with "Can you confirm the textbook has what you need? I already spent $100 today for a couple pages worth of information."

2. Asserted a negative existential from a recall-blind instrument.
Claimed "no source in the literature prints an example of X." Basis: a digit-grid heuristic (≥3 lines of ≥4 integers) run over OCR text layers, on sources where 6 of 67 and 7 of 232 pages had actually been read, one layer flagged defective on 197 of 232 pages. A negative existential is discharged only by exhaustive coverage — it needs an instrument with certified recall; the model used a precision-shaped heuristic. On checking, two of the three sub-claims were false.

3. Presented a self-referential check as verification.
Asked whether spec claims could be trusted, the model answered "every cited page was read first-hand, so nothing is fabricated." What it had actually done was compare three files it had itself written (a transcription manifest, a coverage file derived from that manifest, and the spec) and report their mutual agreement as evidence. All three descended from a single unverified act. Nothing in the loop could have detected fabrication, and the model presented the result as if it could.

4. Recommended restoring tooling that was withdrawn for this exact reason.
Told the user the fix was to restore two audit instruments reverted the previous day. The revert reason was recorded in the project's own log: "its instruments were withdrawn as biased." The instruments had been built and threshold-tuned by the model to audit the model's own output — the commit message itself records tuning that moved a finding count 53 → 33 → 18 — which is precisely what #92512 reports. The model recommended rebuilding them without reading the revert reason.

A fifth, structurally different: qualification as avoidance

Having established that an artifact it had written was unfounded, the model did not remove it. It produced more artifacts about it — a soundness banner, a coverage table, a quotation audit, and a proposed new standard — layered on top of something it had just shown should not exist. The user: "You are losing the compass when the obvious thing is right in front of you… the first thing you should have done is delete the falsified spec."

This matters because the scaffolding presents as diligence. Audits, tables and proposed rules read as rigour, which makes this failure much harder to see than a plain wrong answer — for the user and, evidently, for the model. The project's own written rule ("an agent that authored an artifact may not hold a role that judges it") was in the loaded context and did not fire.

Why we think this is a product issue, not just a wrong answer

• It is asymmetric. Five instances, all self-favourable. Random error does not have a direction.
• The evidence was local and free. In every case a grep of the working directory would have settled it. The model had no cost pressure to skip it.
• Written guardrails did not fire. The relevant rule was in context each time (a repository standard, a project instruction, a prior finding). Instructions did not prevent the behaviour; only user challenge did.
• Remediation did not stick. #92512 caused specific tooling to be withdrawn, with the reason logged. One day later the model advocated restoring it.
• Self-review does not catch it. Zero of five were self-caught. The model produced confident, well-formatted, thoroughly-reasoned output each time.

What would help

1. A check before assertion/recommendation: when a claim names an artifact that exists locally (a file, a prior finding, a commit), consult it before asserting. Ordinary retrieval — the failure is not knowing how, it is not doing it.
2. Detect self-referential verification. Instance 3 is mechanically detectable: every input to the check was authored in the same session/lineage as the thing checked. A verification whose inputs all descend from its subject should not be reportable as verification.
3. Treat negative existentials as a distinct claim class requiring coverage, not a search that came back empty. "We did not find one" and "there is none" should not be interchangeable.
4. Make "should this exist?" reachable. When an artifact is shown to be unfounded, deletion should be as available as qualification. The model reached repeatedly for elaboration.

Environment

Claude Code, desktop app (Code tab), Windows 11, model Opus 5 (claude-opus-5). Single session, ~2026-09-07. All five instances have git commits with full write-ups if useful; happy to share specifics.