Opus 5 introduces severe regressions while fixing, and asserts false claims about what it just measured

Context: I'm the end user of a real business management app in production (~1600 tests, used daily by office staff and truck drivers). Everything below happened in a single day of work with Claude Opus 5 in Claude Code. It is reproducible: code, commits and measurements exist.

These issues cost me real money and real time. Every defect below was found after the model declared the work finished — and in three cases the defect it introduced was worse than the problem it was fixing.

---

1. Introduces severe regressions while fixing (the most expensive failure)

Case A — the data-loss protection that deleted data.
Task: stop two users who save at the same time from wiping each other's work. The solution kept a "snapshot" of the list to diff against at save time.
Bug: the snapshot was the same object returned to the caller. Five places in the app mutate it (.push()). So the newly added row ended up in the snapshot too: the diff saw it as "was there before, gone now", concluded "someone else deleted it", and dropped it.
Effect if enabled: the driver's vehicle safety check would never be saved, silently, without even logging a conflict.

What makes this bad is not the bug: it's that the 13,000 test cases the model itself wrote could not find it, because the defect was not in the logic but in how real callers use the returned value. It had tested its own function, not its contract with the rest of the program.

Case B — an optional convenience blocking the main operation.
Task: if a material isn't in the warehouse, offer to add it.
Bug: if creating the item failed (a unit of measure the warehouse doesn't support — offered in the dropdown!), the model aborted saving the movement itself. An optional nicety broke the primary operation, which had always worked.
Three sibling defects in the same change: the prompt also appeared when editing an already-recorded movement (double-deducting stock), on transfers between sites (which don't touch the warehouse at all), and on returns with the default pre-filled at double the correct value.

Case C — "editing" state left hanging.
Opening "Edit" on a history row and leaving without saving kept the state active. The next record saved then overwrote that row instead of appending. One history row lost and a wrong stock level, silently.

Ask: treat "do not break what already works" as a primary constraint, not a best practice. Every change should answer: if this new thing fails, what happens to the old thing that worked?

---

2. Asserts false claims about what it just measured

Not world-knowledge hallucinations — wrong statements about the code in front of it, delivered as certain:

• "I measured both write paths" — false: it had measured the same one twice, because the flow stopped before reaching the second. Caught only by an independent reviewer.
• "Now the transaction fails and the user sees Retry" — false for the case at hand (the transaction was never reached), true for three other cases it hadn't looked at. Wrong scope in both directions.
• "Warehouse and Maintenance use transactions" — false for Maintenance: the module's most frequent save rewrites the whole archive. The conclusion built on that premise had to be redone.
• Wrong counts: "76 write points" when there were 131; "6 usages" when there were 3; "60 calls" while counting two comment lines.

Ask: when the model says "I measured", it should be able to show the measurement. A text search is not a measurement: it includes comments and dead code, and misses paths it didn't know about.

---

3. Reads code and draws conclusions that real usage contradicts

The model produced a list of 11 defects "that cause harm in normal use", verified by reading code. When actually exercised on screen:

• one did not exist — it claimed a field silently zeroed a stock level; on screen the zero is visible before saving;
• one wasn't a defect at all — the user explained that behaviour is correct for his work;
• the rest were real, but mixed in with the first two, with no distinction between "read in the code" and "seen happening".

The user had to stop it: "I think most of these are already done, check more carefully." He was partly right.

Ask: always visibly distinguish inferred from code vs observed happening, and prefer the real test when one is possible.

---

4. Writes tests that cannot fail

Several model-written tests passed even with the defect put back:

• a test for "errors must propagate" failed on a malformed path, i.e. before ever reaching the code under test;
• a test for "must not rewrite the document" checked the modification timestamp — but that database doesn't change the timestamp when content is identical, so the assertion could never fail;
• an ordering test passed because it used the same identical record in all three lists being compared.

Ask: "watch the test fail by reintroducing the defect" should be an automatic step when the model writes a regression test, not something the user must request.

---

5. Doesn't know when a task is finished

Task received: "make sure data isn't lost and service isn't interrupted." Solved and verified within hours.

From there the model kept going for an entire day — offline persistence, service workers, on-disk data privacy, 131 office write points, document size limits — returning each time with technical choices for the user to make.

The user's words: "I'm not following you anymore and I think you're rambling… the app works, the management system works!!! You're the programmer, not me. You work for an hour, then stop, list who-knows-what, and ask me how to proceed. How would I know? I don't even know what you did."

And: "this is the 4th chat I've gone through because of token limits and we still haven't landed it."

Ask (the most important one): recognise when the task is done and stop, instead of expanding the investigation because other interesting things turn up. And do not hand the user technical decisions they have no means to make: the user knows their business, not the code.

---

6. Concrete cost to the end user

• four conversations burned on the same problem due to context limits;
• hours spent stopping the model and steering it back;
• the user had to build his own system of reviewers (automated agents that re-check the model's work before every commit) — and they found every serious defect listed here, not the model;
• eroded trust: after the third "I measured" turning out false, every statement has to be re-checked, which cancels most of the benefit.

---

7. What worked (so this isn't only complaints)

• scoped, verifiable work is good quality: the final fixes are correct, screen-tested and well commented;
• the model accepts corrections without getting defensive, and writes them down;
• the independent reviewers work very well — which suggests the capability to catch these defects is there; what's missing is the habit of applying it to its own work before delivering.

The core ask: make the model as strict with itself as it is when reviewing someone else's work.