Opus 5: non-converging review/fix-regression spirals — every affected session ran [1m] at env-pinned max effort; confound worth checking against other Opus 5 reports

Status Open
Reported on v2.1.223
Maintainer reply None cached
Activity 0 comments · opened Aug 6, 2026

Summary

Across two unrelated codebases (macOS/TypeScript+Swift and Windows/C#), claude-opus-5 sessions repeatedly produced large speculative changes that failed code review, then fix waves whose fixes became the next round's findings — never converging. Reverting to Opus 4.8 on the same repos/config eliminated the pattern immediately.

While investigating, we discovered our Opus 5 sessions had only ever run in an extreme configuration — and got there without the user knowingly choosing it. If other Opus 5 quality reports share this confound, it's diagnosable from telemetry and would separate "model regression" from "regime interaction."

Environment

  • Claude Code 2.1.223 (native, darwin-arm64); claude doctor clean
  • Model: claude-opus-5[1m] — the 1M-context variant. Every Opus 5 session on this machine ran [1m]; standard-window Opus 5 never ran once.
  • CLAUDE_CODE_EFFORT_LEVEL=max exported in the shell profile — pinned long ago (pre-dating the high defaults), which per the docs overrides and locks /effort and the /model slider. So: max effort, every session, uncontrollable from the UI.
  • autoCompactWindow: 600000 in settings.json — compaction deferred deep into the 1M window.
  • Heavy instruction load: ~110KB project CLAUDE.md (since trimmed) + plugin skill mandates + memory index.

Measured example (a single real session)

lastModelUsage for claude-opus-5[1m]: 48.5M cache-read tokens, 1.2M cache-creation, 181k output tokens, $40.79, 1,507 lines added. Back-of-envelope, that's an average working context in the ~250–350k range per API call — most of the session lived past the standard window.

The failure pattern (three separate incidents)

  1. A review at max effort found 45 issues on a patch series; three review rounds never converged; the subsystem was ultimately rewritten from scratch.
  2. A 13-task feature built green, then: round 1 — 15 findings; fix wave; round 2 — 15 findings of which 10 were regressions introduced by round 1's fixes; round 3 (scoped to just the fix commits) — ~15 more, nearly all from round 2's fixes. Branch discarded whole.
  3. The same shape independently on the Windows repo (C#/.NET).

Recurring defect signature: the same rule implemented at one call site but not its sibling, written in the same commit (e.g., a null-guard in one write path and not the parallel one).

The control (imperfect, disclosed)

Opus 4.8 on the same repos under the same pinned max effort: no spirals, normal review outcomes. Confound: the 4.8 sessions ran the standard context window, so model version and context regime changed together. Fable 5 under standard usage has also been fine.

Two asks

  1. Telemetry check: do Opus 5 quality complaints (e.g. #82162, #77136, #21431) skew toward [1m] and/or pinned-max-effort configurations? If yes, the "nerfed" narrative may partly be a regime interaction rather than a model regression — actionable either way.
  2. Product: the [1m] choice is invisibly sticky. Once a [1m] variant is picked in /model once, it persists in settings.json "model" and every subsequent session (and later model picks) silently inherits the 1M variant. This user ran 1M for weeks without knowing ("I am never specifying model size — the default is coming from I don't know where"). Consider surfacing the active context variant more loudly — in the /model confirmation line and/or a statusline default — and not carrying the variant across family switches by default.

Happy to provide session IDs privately via /bug if useful.

View original on GitHub ↗