Opus 5 introduces severe regressions while fixing, and asserts false claims about what it just measured

Context: end user of a real business app in production (~1600 tests, used daily by office staff and drivers). Everything below happened in one day with Opus 5 in Claude Code, and is reproducible.

Every defect was found after the model declared the work finished — and three times the defect it introduced was worse than the problem it was fixing.

1. Introduces severe regressions while fixing

• The data-loss protection that deleted data. It kept a "snapshot" to diff against at save time, but the snapshot was the same object returned to callers, and five call sites mutate it (.push()). The new row landed in the snapshot too, so the diff read it as "was there, now gone" → "someone else deleted it" → dropped it. Effect: the driver's safety check would never be saved, silently, without even logging a conflict.

The damning part: the 13,000 test cases the model itself wrote could not find it, because the defect wasn't in the logic but in how real callers use the returned value. It tested its function, not its contract.

• An optional convenience blocking the main operation. "If the item isn't in the warehouse, offer to add it" — if creating it failed (a unit offered in the dropdown but unsupported by the warehouse), it aborted saving the movement itself, which had always worked.
• Editing state left hanging. Open "Edit", leave without saving → the next saved record overwrote that row instead of appending. History row lost, wrong stock, silently.

Ask: treat "don't break what already works" as a primary constraint. Every change should answer: if this new thing fails, what happens to the old thing that worked?

2. Asserts false claims about what it just measured

Not world-knowledge hallucinations — wrong statements about the code in front of it, delivered as certain:

• "I measured both write paths" → had measured the same one twice; the flow stopped before the second.
• "Now the transaction fails and the user sees Retry" → false for the case at hand, true for three others it hadn't looked at.
• "Warehouse and Maintenance use transactions" → false for Maintenance; the conclusion built on it had to be redone.
• Counts: "76 write points" when there were 131; "6 usages" when 3; "60 calls" while counting two comment lines.

Ask: when it says "I measured", it should be able to show the measurement. A text search is not a measurement.

3. Reads code, draws conclusions real usage contradicts

It listed 11 defects "harmful in normal use", verified by reading code. Exercised on screen: one didn't exist, one wasn't a defect — and both were mixed in with the real ones, with no distinction between inferred and observed. The user had to stop it: "I think most of these are already done, check more carefully." He was partly right.

4. Writes tests that cannot fail

Several passed with the defect put back: one failed on a malformed path (never reaching the code under test); one checked a modification timestamp that the database doesn't change when content is identical; one compared the same identical record in all three lists.

Ask: "watch it fail by reintroducing the defect" should be automatic when writing a regression test.

5. Doesn't know when a task is finished

Task: "don't lose data, don't interrupt service." Solved in hours. It then kept going for a full day (offline persistence, service workers, on-disk privacy, 131 write points…), each time returning with technical choices for the user to make.

User's words: "I'm not following you anymore and I think you're rambling… the app works!!! You're the programmer, not me. You work for an hour, then stop, list who-knows-what, and ask me how to proceed. How would I know? I don't even know what you did." And: "this is the 4th chat I've burned on this."

Ask (most important): recognise when the task is done and stop. Don't hand the user technical decisions they have no means to make.

Cost

Four conversations burned on one problem; hours spent steering it back; the user had to build his own reviewer agents — and they found every serious defect above, not the model. After the third "I measured" proving false, everything must be re-checked, which cancels the benefit.

What worked

Scoped, verifiable work is good: the final fixes are correct, screen-tested, well commented. It accepts corrections without defensiveness. And the independent reviewers work very well — so the capability is there; the missing habit is applying it to its own work before delivering.

Core ask: make the model as strict with itself as it is when reviewing someone else's work.