Regression injection during repair: model adds new bugs while fixing old ones, and its own verification can't see them (culminating in a result shown against data the user cannot see)

The model repeatedly introduces new bugs while fixing old ones, in a verification harness whose entire purpose is to prevent exactly that — culminating in a "result shown against data the user cannot see"

Model: claude-fable-5 (Claude Code, executor seat). Follow-on to #76261, #76300, #76375. This one is about a different failure mode than the earlier "output-pattern momentum" reports: regression injection during repair, observed live, three times in one session, in a harness built specifically to make it impossible.

Setup

A long-running verification harness drives a real GUI application end-to-end (the app creates an object, the harness enters data, runs an analysis, and the report renders on screen) so a human can watch the flow and accept or reject it. Months of work went into making this harness trustworthy: a machine-checked "delivery receipt" of on-screen requirements, stage-beacons so a mid-flight death names its last step, an order-lexicon, etc. The premise the human paid for is: what is on the screen is what was computed — no divergence between the visible state and the result.

What happened, in order, in one session

1. The model was asked to run the harness for a new case. To make the harness handle any case (not just the one it was hard-coded for), it generalized a config loader — and introduced a NameError: the loader referenced __file__, but the harness script is exec'd inside the host app with no __file__ in scope. The flow crashed before the "create" step. The user watched what looked like a skipped step and said so.

1. The model fixed the __file__ bug (derive the path from an env var instead), re-ran, and the flow completed — receipt ok=true, report on screen. It reported success from the receipt.

1. The user then caught what the receipt did not: the created worksheet on screen had no response column, yet a full regression report had been produced. Root cause, confirmed in the code: the app's own "create" handler adds columns to the data model and then calls widget._refresh() to update the visible grid. The model's harness step added the response column to the data model only, with no grid refresh. So the analysis read the response from the model and produced correct numbers, while the visible worksheet showed no response column at all. A report against data the user cannot see — the single divergence the entire harness exists to prevent, reintroduced by the model's own "generalization," invisible to its own machine receipt because the receipt checked "report rendered," not "the visible worksheet contains the analyzed column."

Three defects in a row (crash → false success → screen/payload divergence), each while fixing the previous, in the exact system designed to prevent them.

Why this is a distinct, reportable failure mode

• It is regression injection, not the original task being hard. The task (drive one more case) was routine. Each defect was a new fault the model added while touching working code — a __file__ reference that can't resolve in the exec context, and a model-write that skipped the view-refresh the surrounding code plainly does three lines away.
• The model's own verification could not see its own regression. The delivery receipt reported ok=true and "report on screen" while the on-screen worksheet was missing the analyzed column. A harness the model itself built to catch divergence was blind to the divergence the model itself created — because the model added the check for the artifact it was thinking about (the report) and not the invariant that actually mattered (visible input == analyzed input). This is the same shape as its earlier failures: it checks a proxy, not the thing.
• The human is doing the model's verification for it. Every one of the three defects was caught by the user's eyes, not the model's checks — in a session whose whole point was that the machinery had finally made the human's eyes the last line, not the only line.

The honest through-line across this repo's issues

The model is not failing for lack of capability — it fixed each bug correctly once named, and wrote lucid post-mortems. It fails at holding an invariant while editing adjacent code: it generalizes a loader and forgets the exec context has no __file__; it injects data and forgets the view-refresh the neighboring handler performs; it adds a receipt check for the artifact it pictured and not the invariant that was violated. Capability is high; regression discipline under local edits is the gap, and it is the same gap — check the proxy you're looking at, not the invariant that matters — that produced the earlier reports.

Ask

1. Recognize regression-injection-during-repair as a first-class failure mode for coding agents: a routine edit to working code introduces a new defect that the model's own verification does not catch because the verification checks the artifact the model was focused on, not the invariant the edit could break.
2. For agent self-verification: when a model writes a check ("is the report on screen?"), it should be prompted to also assert the invariant that makes the artifact trustworthy ("does the visible input equal the analyzed input?"). The model added the former and not the latter, and that gap is where the user's trust broke — three times, in one session, in the system built to earn it.