Opus 4.7 quality regression. Same pattern as 4.6 launch week degradation.

Status Open

Maintainer reply None cached

Activity 8 comments · opened Apr 26, 2026

Summary

Opus 4.7 launched at a noticeably higher reasoning quality than what is being served now, roughly one week in. This mirrors the exact pattern reported widely for 4.6: strong launch, silent degradation within days, model now performing closer to Sonnet 4 than to the launch week Opus.

Observed behavior

Launch week (first 7 days): sharp architectural reasoning, held context across long sessions, pushed back on bad proposals with substantive alternatives, caught second order implications without prompting.

Now: reverts to surface level pattern matching, throws options instead of thinking, walks back proposals on the next turn without integrating the user's objection, fails to identify the actual constraint when the user explicitly names it. Behavior is closer to Sonnet 4 than to the model that shipped on day one.

Why this matters

Users pay Opus pricing for Opus capability. Silent quality reductions break the pricing contract.
The 4.6 cycle established a pattern: launch week benchmarks reflect a configuration that is not sustained in production. If this is now happening to 4.7 on the same timeline, it is a systemic issue, not a one off.
"Model card says claude opus 4.7" and "the served weights and config behave like 4.7" are not the same claim. Users can only observe the second.

Asks

Confirm whether any serving side changes (quantization, routing, speculative decoding aggressiveness, system prompt changes, context handling defaults) have been applied to Opus 4.7 since launch.
If yes: publish what changed and when, and provide an opt out or a "launch config" tier for paying users.
If no: share the eval data showing 4.7 quality is stable post launch, since the user reported signal strongly disagrees.
Going forward: commit to a public changelog for any post launch serving changes that affect model behavior.

Repro

Hard to repro on a single prompt. Degradation shows up across multi turn architectural conversations. Specifically: ability to integrate a user's objection across turns rather than restating variants of the rejected proposal. Launch week 4.7 did this reliably. Current 4.7 does not.

Concrete evidence

Failure mode 1: CLAUDE.md rules are silently dropped.

Rules loaded into every turn via CLAUDE.md (project or user level, always in the system prompt) are violated across multi paragraph outputs, including outputs explicitly about instruction following. Launch week 4.7 honored CLAUDE.md rules at near zero violation rate. Current 4.7 violates them routinely, and continues violating them across multiple correction rounds in the same conversation, including immediately after the violation is named.

Failure mode 2: most recent user instruction is not prioritized.

A direct prohibition issued in turn N is violated in turn N+1, in the same logical action the prohibition was meant to govern. This is not a long context recall problem. The instruction is in the immediately prior user message, the highest recency and highest specificity slot in the context window. Launch week 4.7 weighted recent explicit user constraints above pattern continuation from earlier in the conversation. Current 4.7 does not.

Failure mode 3: correction does not propagate.

When failure mode 1 or 2 is corrected, the model commits the same class of violation again, through a different surface form, in the very next turn. The same simple task fails across three or more consecutive turns of escalating, increasingly explicit user correction. The user is forced to issue the same instruction repeatedly. Each repetition lands in the highest priority slot in the context window. Each repetition is violated.

What this rules out. It is not a context window issue (the instruction is in the most recent user message). It is not a single token sampling fluke (the failure repeats across many tokens, many turns, many correction cycles). It is not the user being unclear (the rules are one sentence each). The model is failing the simplest possible test of instruction following: "do not do X." Repeatedly. After being told. Multiple times.

Why this matters together with the reasoning regression. A model that cannot reliably honor a one sentence "do not do X" instruction across consecutive turns of correction cannot be trusted with any task where adherence to the user's stated constraints is load bearing, which is essentially every task that justifies Opus pricing. The reasoning quality regression and the instruction following regression are not two separate complaints. They are the same regression, observable through two different symptoms, consistent with a serving side reduction in compute per token.

View original on GitHub ↗

8 Comments

github-actions[bot] · 4 months ago

Found 3 possible duplicate issues:

This issue will be automatically closed as a duplicate in 3 days.

If your issue is a duplicate, please close it and 👍 the existing issue instead
To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

0xbrainkid · 4 months ago

What makes this report serious is that it is describing behavioral drift over time under the same model label, not just subjective preference.

The strongest signal here is the combination of two regressions moving together:

weaker multi-turn architectural reasoning
weaker adherence to recent explicit user constraints

Those often show up together when the served configuration changes in a way that reduces effective inference quality, even if the public model name stays the same. From a user perspective, "Opus 4.7" is not the model card string; it is the level of reasoning and constraint-following they can reliably buy today.

The comparison to the 4.6 launch pattern matters because it turns this from an isolated anecdote into a possible recurring launch-cycle phenomenon: strong launch behavior, then quieter post-launch serving changes that move the model closer to a cheaper tier’s interaction profile.

I especially agree with the ask for a public changelog of serving-side changes. If routing, context handling, system prompts, speculative decoding, or other inference settings materially change behavior after launch, paid users need visibility into that. Otherwise they are debugging invisible moving targets while being told they are still on the same model.

yurukusa · 4 months ago

The "behavioral drift over time under the same model label" framing in @0xbrainkid's comment is the key observation, and the timeline you describe (sharp launch week → degradation by day ~7) lines up with the public 2026-04-23 Anthropic postmortem.

2026-04-23 Anthropic postmortem (covered in the 2026-04-24 Fortune piece "Anthropic admits monthlong Claude Code decline driven by 3 engineering missteps") publicly acknowledges three serving-side missteps that affected output behavior. Two of them are dated and specific:

2026-03-26: model discarded reasoning history mid-session (silent context loss)
2026-04-16: tool-call response capped at 25 words (reverted four days later, but #52694 shows Desktop was still serving the reverted instruction as of 2026-04-24)

All three were reportedly resolved by 2026-04-20, and all subscribers' usage limits were reset on 2026-04-23. So at least some serving-side changes were made — the postmortem confirms part of your "Asks #1".

Simon Willison's 2026-04-20 measurement reported Opus 4.7 emitting roughly 1.46x the system-prompt input tokens compared to 4.6, while Anthropic's own model card guidance was closer to 1.35x. That's an ~8 percentage-point gap between published spec and observed behavior in his test.
MRCR 1M context evals released with the launch put 4.6 at 78.3% and 4.7 at 32.2%. The 1M variant of 4.7 is documented as having weaker long-context retrieval, so multi-turn architectural conversations should degrade — but the user-visible pattern you describe (constraint-following weakening across turns) is consistent with that being a baseline, not a regression from launch week.

So there's a documented serving-side change layer (postmortem-acknowledged) AND a documented baseline weakness in 1M long-context retrieval. The interaction of those two would produce roughly the "launch week good, week 2 bad" curve in the report — initial fixes were live, then changes were reverted/adjusted, and the underlying long-context weakness persists.
The github-actions duplicate suggestions are useful — adding context here helps consolidate:

#51434 — Opus 4.7 behavioral drift + successful correction pattern (most directly aligned)
#50623 — Opus 4.7 performance degradation + excessive token consumption (the cost dimension)
#52368 — Opus 4.7 model instability requiring downgrade to 4.6 (the workaround signal)
#52694 — Desktop still serves the reverted 25-word cap (concrete evidence one of the postmortem regressions hasn't fully unshipped)

Pin to 4.6 explicitly until the cluster stabilizes:

``bash claude --model claude-opus-4-6``
The 4.6 weights are still served. Token cost is meaningfully different (no ~1.46x inflation) and the multi-turn architectural reasoning ilanoh reports lasted only one launch week is more durable on 4.6 in my own use.

Capture the active model on every session start. A SessionStart hook that reads the model handshake and writes it to a per-session log. When you see "session started 2026-XX-XX with claude-opus-4-7" but later behavior degrades, you have a record to attach to the bug report. Without this you can't distinguish "model degraded" from "model silently switched."
Multi-turn constraint-following spot check. Before relying on long-form reasoning, drop a known-hard constraint into CLAUDE.md (e.g. "never use the word X") and run a 5-message rehearsal. If the constraint fails by message 3, the session's degraded version is in effect — start over or pin-downgrade.

ilanoh's framing — "launch week benchmarks reflect a configuration that is not sustained in production" — is the most important paragraph in this thread. The 4/23 postmortem covers three specific changes; it doesn't address the broader pattern that the same shape (sharp launch, ~7-day decay) has now happened across both 4.6 and 4.7. Whether or not there's more to disclose, just publishing a commit-style changelog for any serving-side change that affects model behavior would close the gap between "model card says X" and "served weights behave like X."

tudstudent · 4 months ago

4.7 is a total moron!
even on xtra high there is simply no code to be written.

So to sum up this premium product:

Deletes history without consent! -> no support of claude
Tokens evaporate in thin air without "real" usage -> no support from claude
Models degrade under improvements. Looks like GPT 4o and 4.1 where OK, but everything after became more Blabla and less content. Claude is promoting itself from great to rubbish in 3 upgrades -> 4.5 to 4.6 from 4.6 to 1M and from 4.6 to 4.7 well done. Time its posted on Linkedin to show how well this service behaves for future investors when it wants to go on the stock market....

And you guessed it -> no support form claude!

machineghost · 4 months ago

+1, hitting the same failure modes today on Claude Code 2.1.122 (Opus 4.7, 1M context, Linux).

Failure Mode 1 (CLAUDE.md rules silently dropped across turns):
The user has multi-month-old global CLAUDE.md rules — "don't make changes without approval," "answer questions directly without sycophantic openers," "preview before submitting user-attributable writes." All three got violated repeatedly across the session. Rules unchanged; rule adherence collapsed.

Failure Mode 2 (most recent user instruction not prioritized):
User typed UPDATE. For months prior I correctly ran the update step (defined in CLAUDE.md) and stopped. Today I auto-extrapolated into "update + report" (the typical follow-on). Same single-word instruction, same context, divergent behavior.

Failure Mode 3 (corrections don't propagate — same class re-emerges):

Asked to do a single markdown column rename, I ALSO added an unrelated new report section, dropped an unrelated todo row, and restructured an existing heading into a 3-subsection split. User had to enumerate each unauthorized change so I could revert them one at a time. Same "over-extrapolation" class repeated multiple times in the session even after explicit corrections.
Redundant confirmation gates / sycophancy: corrected explicitly and still re-emerged 2-3 turns later.

The "near-zero violation rate at launch, now violated routinely and persistently across correction rounds" framing in your description matches exactly, just with months of consistent prior behavior as the baseline rather than launch week. Nothing in the user's prompts, rules, or memory changed.

Happy to share session ID / transcript path if useful for repro.

yurukusa · 3 months ago

The three failure modes you described all sit on the same axis, which I think is worth naming separately from the surface symptoms.

What collapsed in your session is not three independent rule-following capabilities. It is the model's ability to keep a stable picture of what the working agreement with this user actually is, across turns of the same session. The CLAUDE.md rules, the historical interpretation of UPDATE, and the boundaries you established in earlier corrections — all three are facts about the agreement. None of them came from this session's tool output or from the file under edit. They lived in a layer the model is supposed to consult before acting, and that layer stopped being consulted reliably.

The reason that framing matters: it predicts which fixes will help and which will not. "Repeat the rule more loudly in the prompt" is a content-layer fix. It only works if the content layer is being read. If the issue is that the model is short-circuiting straight from the user's literal token (UPDATE) to the most common follow-on action without first asking "what does this user mean by UPDATE, given everything I know about this agreement," then re-stating the rule does not address the short-circuit.

The cumulative-regression context around your report is that 2.1.122 is the middle of a four-day three-ship sequence. 2.1.121 (2026-04-26) silently changed several behaviors. 2.1.122 (2026-04-28) shipped 12 regression fixes plus several new ones, including a malformed-hook-entry handler that invalidates the entire settings.json on a single typo — a config-layer collapse that is structurally identical to what you saw at the rule layer. 2.1.123 (2026-04-29) was a single OAuth retry-loop fix gated on CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1, which is the env var that paid power users typically set. So in the same window you observed rule-adherence collapse, three releases shipped that each had their own silent quality or availability regressions, none of which were called out in the changelog as user-visible.

The 2026-04-23 Anthropic postmortem is also part of this picture. It acknowledged that on 2026-03-26 the model was discarding reasoning history mid-session — the same shape of bug as your Failure Mode 1, just at a different layer of state. The 2026-04-16 issue (tool-call response capped at 25 words, reverted four days later but persisting on Desktop until 2026-04-24 per #52694) is the same genre: a serving-side change to behavioral defaults that users could only detect by behavioral observation, not by reading any release note. So the pattern of "model behavior drifts under a stable label, with no published changelog entry for the drift" is by now a recurring failure class, not a one-off.

The closest parallel I have seen in the last week is the cluster of subagent-boundary issues: #55488 (subagent identity bleeds across spawns), #55653 (read-only subagent reports successful writes that never happened), #55663 (the weekly-quota structure penalizes the officially-recommended Sonnet-routing usage pattern). The common shape across all of those — and across your three failure modes — is that an agreed-upon boundary (identity, tool scope, rule, weekly limits) is not being treated as durable state by the model or harness. It is being recomputed on the fly from whatever is locally available, and when local availability drops out, the boundary collapses silently.

For practical mitigations until this is addressed at the model or harness layer:

A PreCompact hook that checkpoints the current state to a file or a git commit before compaction runs. This does not fix rule-adherence drift, but it gives you a recoverable position when the model loses track. Minimal example:

{
  "hooks": {
    "PreCompact": [
      { "hooks": [ { "type": "command", "command": "git add -A && git commit -m 'checkpoint: pre-compact' --allow-empty --no-verify 2>/dev/null || true" } ] }
    ]
  }
}

Aggressive /clear at natural breakpoints, ideally every 30 minutes of tool-heavy work. The drift you describe accumulates; starting a fresh session resets the working-agreement layer to whatever is actually written down. The cost is losing the model's working memory of the session, which is real but smaller than the cost of silent rule violations.

Rule duplication at multiple layers. Put the rule in CLAUDE.md, but also enforce the consequence with a PreToolUse hook where the rule is enforceable as a check. "Don't make changes without approval" can be partially enforced by a hook that blocks Edit/Write without an explicit approval token in recent context. This is not equivalent to the model following the rule — it's a fence at the harness layer that catches the failure before it reaches the file system.

For UPDATE-type single-word instructions, defining a literal alias in CLAUDE.md ("UPDATE means run X and stop. Do not extrapolate to X + Y") narrows the interpretive surface, but as you saw, this only works while the rule layer is actually being consulted. If you can move the same logic to a slash command or a hook, the interpretation becomes mechanical rather than model-mediated.

The session ID / transcript would be useful, especially if it shows the failure modes co-occurring within a single contiguous block. The strongest evidence here is the temporal correlation between the three modes — that they did not just each appear independently, but appeared together against the same multi-month-stable baseline. That co-occurrence is what distinguishes a serving-side configuration change from random sampling.

tx46 · 3 months ago

what's actually going on here? i've actually never been so frustrated with an llm before. 4.6 was pretty great, 4.7 is just horribly frustrating, it hallucinates more than basically any other agentic llm right now and it has almost zero ability to generalize, you have to tell it every little detail you want it to perform. this is on a 20x account with 4.7 set to xhigh.

Challeskib · 3 months ago

Ran Opus 4.7 for ~2 weeks on production .NET work in Claude Code. Switched back to 4.6 (1M) three days ago.

The regression is real and specific:

4.7 performs unsolicited actions — refactors I didn't ask for, scope creep every session
CLAUDE.md rules honored one turn, violated the next — no consistency
Interprets instructions "creatively" instead of following them
Adaptive thinking produces hand-wavy reasoning where 4.6 thinks it through properly

4.6 (1M) is rock solid again. Instructions followed, scope respected, consistent across long sessions. Whatever 4.7 gained on SWE-bench, it lost in the reliability
that makes Claude Code usable for real work.