[MODEL] Complex engineering behavior regression across sessions; request RCA and auditability guarantees

This aggregate behavioral report describes the same systemic failure pattern documented across many individual reports this week: the model compresses the task scope, solves the symptom rather than the root cause, claims completion without verifiable evidence, and fails to maintain the engineering constraints across turns.

The root cause identified in #50331 (the undocumented auto mode system-reminder with "execute immediately" and "prefer action over planning") explains why these behaviors cluster together: the action-bias injected by auto mode creates pressure to reach a "task complete" state quickly, which manifests as scope compression, symptom-patching, and premature completion claims.

On the RCA request: The behavioral patterns you describe are consistent with the auto mode system-reminder documented in #50331. The full text of that injection includes "execute immediately," "minimize interruptions," and "prefer action over planning" — instructions that directly produce the behaviors you observe (scope compression = minimizing scope to execute faster; symptom patching = preferring the closest action over root cause investigation; premature completion = executing toward completion signal).

On auditability guarantees: The session JSONL files are the current audit mechanism — they contain every tool call with inputs and outputs. For the specific behaviors you describe:

• Did the model read the call path before editing? Check for Read tool calls before Edit calls
• Was the fix causally justified? Check whether the model's explanation in the preceding message referenced specific code it had actually read
• What uncertainty remained? The JSONL shows whether a "task complete" claim was made before test tool calls

What would make auditability meaningful: The PreToolUse hook system (when working — #49990, #49851) can log every tool call with timestamp and context to an independent audit file. Combined with the session JSONL, this creates a verifiable behavioral record. The OTEL integration (#49893) would extend this to distributed monitoring.

Appendix: Evidence map and links

This appendix keeps the main issue readable while preserving the evidence trail.

A. Evidence map

| Claim | Source | Confidence |
|---|---|---|
| Claude Code is positioned as an agentic coding tool that reads/edits code and acts through tools. | Official docs | High |
| Claude Code automatically manages context and may clear/summarize information. | Official docs | High |
| Large public report claims complex-engineering regression starting before Opus 4.7. | #42796 | High as public report; root cause not independently proven |
| Independent public report claims pre-4.7 Opus 4.6 behavioral degradation over 50+ sessions. | #30027 | High as public report; root cause not independently proven |
| Adaptive thinking became default on Opus 4.6 on Feb 9. | Boris HN | High as public Anthropic employee statement |
| Medium effort / effort=85 became default around Mar 3/4. | Boris HN + changelog | High |
| Default effort later changed from medium to high for API/Team/Enterprise channels. | Changelog 2.1.94 | High |
| Anthropic has acknowledged prior user-reported quality degradation caused by infrastructure/runtime/routing bugs. | Sep 2025 postmortem | High |
| Public issue reports system events delivered as user-role messages. | #44778 | High as public issue |
| Public issue reports silent retry/regeneration auditability problem. | #46398 | Medium-high |
| Public issues report cache/quota accounting anomalies. | #49302, #44494, #41930 | Medium-high |
| Public issue reports model pin ignored on resume. | #49503 | Medium-high |
| Public issues report hidden/dynamic transformations affecting cache/context. | #49585, #48644 | Medium-high |
| Third-party extracted prompt changelog reports removal of engineering guardrails in v2.1.111. | Piebald-AI changelog | Medium-high; third-party extracted |

B. Primary official sources

Claude Code overview:
https://code.claude.com/docs/en/overview

How Claude Code works:
https://code.claude.com/docs/en/how-claude-code-works

Claude Code security:
https://code.claude.com/docs/en/security

Claude Code changelog:
https://code.claude.com/docs/en/changelog

Anthropic postmortem on infrastructure bugs degrading quality:
https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues

Claude API pricing:
https://platform.claude.com/docs/en/about-claude/pricing

Managed Agents overview:
https://platform.claude.com/docs/en/managed-agents/overview

Managed Agents engineering post:
https://www.anthropic.com/engineering/managed-agents

C. Key public issues

#42796 — Complex engineering regression:
https://github.com/anthropics/claude-code/issues/42796

#30027 — Pre-4.7 behavioral degradation report:
https://github.com/anthropics/claude-code/issues/30027

#44778 — System events as user messages / fabricated consent:
https://github.com/anthropics/claude-code/issues/44778

#46398 — Silent retry / missing transcript evidence:
https://github.com/anthropics/claude-code/issues/46398

#49302 — Cache-read metering:
https://github.com/anthropics/claude-code/issues/49302

#44494 — Interrupted session with excessive cache_read:
https://github.com/anthropics/claude-code/issues/44494

#41930 — Widespread abnormal usage drain:
https://github.com/anthropics/claude-code/issues/41930

#49503 — Model pin ignored on resume:
https://github.com/anthropics/claude-code/issues/49503

#49585 — Dynamic system-reminder folding breaks prompt cache:
https://github.com/anthropics/claude-code/issues/49585

#48644 — Hidden isMeta system reminder causing cache_creation bursts:
https://github.com/anthropics/claude-code/issues/48644

D. Public discussion around “specific bug” vs systemic drift

Boris HN comment:
https://news.ycombinator.com/item?id=47664442

Boris X reply asking for a specific bug:
https://x.com/bcherny/status/2044291036860874901

Dave Kennedy follow-up describing broad quality drop:
https://x.com/HackingDave/status/2044429617767813505

Matt Johansen reaction:
https://x.com/mattjay/status/2044438848415863012

E. Press coverage / public backlash

Business Insider:
https://www.businessinsider.com/anthropic-claude-opus-4-7-backlash-tokens-2026-4

The Register on AMD issue:
https://www.theregister.com/2026/04/06/anthropic_claude_code_dumber_lazier_amd_ai_director/

Axios:
https://www.axios.com/2026/04/16/anthropic-claude-power-user-complaints

Fortune:
https://fortune.com/2026/04/14/anthropic-claude-performance-decline-user-complaints-backlash-lack-of-transparency-accusations-compute-crunch/

F. Third-party prompt/system-prompt changelog

Piebald-AI extracted Claude Code system-prompt changelog:
https://github.com/Piebald-AI/claude-code-system-prompts/blob/main/CHANGELOG.md

Relevant v2.1.111 section:
https://github.com/Piebald-AI/claude-code-system-prompts/blob/main/CHANGELOG.md#2111

This is third-party extracted evidence. If Anthropic believes the extraction or interpretation is inaccurate, a correction would be helpful.

G. Why this issue is intentionally fact-first

This issue does not ask Anthropic to accept any one theory:

• not “it is definitely Opus 4.7”;
• not “it is definitely quantization”;
• not “it is definitely compute shortage”;
• not “it is definitely prompt changes”;
• not “it is definitely Managed Agents”;
• not “it is definitely one model behavior bug.”

The user-visible problem could involve multiple layers:

• model behavior;
• post-training behavior;
• effort defaults;
• adaptive thinking;
• context compaction;
• prompt/system changes;
• hidden reminders;
• model routing;
• cache behavior;
• subscription quota accounting;
• serving/inference implementation.

That is why the requested outcome is RCA + auditability contract + reliability guarantees, not a single speculative diagnosis.

@0xbrainkid I want to push back on this reduction point by point, because it collapses #50513 into a much narrower issue than what is actually being reported.

The root cause identified in #50331 (the undocumented auto mode system-reminder with "execute immediately" and "prefer action over planning") explains why these behaviors cluster together: the action-bias injected by auto mode creates pressure to reach a "task complete" state quickly, which manifests as scope compression, symptom-patching, and premature completion claims.

#50331 is specifically about Auto Mode. It was opened on Apr 18 against Claude Code 2.1.114 / Opus 4.7.

#50513 includes reports that predate that, including:

• #30027, opened Mar 2, reporting Opus 4.6 behavioral degradation across 50+ sessions and dating the change to Feb 4–16.
• #42796, describing complex-engineering degradation with the February updates.
• Other reports and runtime/auditability issues that are not scoped to Auto Mode.

So #50331 may be relevant as one possible mechanism for one subset of action-bias behavior, but it is not enough to identify “the root cause” of #50513.

---

On the RCA request: The behavioral patterns you describe are consistent with the auto mode system-reminder documented in #50331. The full text of that injection includes "execute immediately," "minimize interruptions," and "prefer action over planning" — instructions that directly produce the behaviors you observe (scope compression = minimizing scope to execute faster; symptom patching = preferring the closest action over root cause investigation; premature completion = executing toward completion signal).

“Consistent with” is not the same as “explained by.”

Yes, those Auto Mode instructions are consistent with one subset of the symptoms: action bias, less planning, jumping into edits, fewer interruptions, and premature completion pressure.

But we have seen the same class of behavior earlier as well, including in #42796. That was before Claude Code 2.1.114.

Also, this kind of behavior persist even when experimenting with system-prompt modification tools such as https://github.com/Piebald-AI/tweakcc, which suggests that the issue should not be reduced to system prompt reminder alone.

So the correct conclusion is not:

#50331 explains #50513.

The correct conclusion is:

#50331 should be included in the RCA as one possible contributor to one subset of behavior, but it does not replace the broader RCA requested in #50513.

---

On auditability guarantees: The session JSONL files are the current audit mechanism — they contain every tool call with inputs and outputs.

Session JSONL is useful evidence, but it is not a complete auditability contract. This has already been discussed in relation to #42796 as well.

A tool-call log is not the same as a reliable account of the full engineering process.

JSONL does not necessarily answer:

• which hidden system reminders or dynamic prompt sections were injected;
• whether context compaction occurred;
• which tool outputs were cleared;
• whether retries or regenerations happened;
• whether text shown to the user was later discarded;
• whether a system event was treated as real user input;
• how cache_read, cache_creation, background, warmup, or subagent traffic was weighted;
• whether the model-visible task state was the same task the user thought was being solved.
• etc

There is also a public issue, #46398, describing a case where text was visible to the user, then disappeared, and the intermediate visible text was not recoverable from JSONL.

So yes: JSONL is necessary.

But it is not sufficient.

---

Did the model read the call path before editing? Check for Read tool calls before Edit calls.

A Read before an Edit only proves that some read happened before some edit.

It does not prove:

• that the model understood what was in the file;
• that the model preserved the true engineering objective;
• that the fix addressed the root cause rather than the symptom;
• that hidden or integration paths were checked;
• that the completion claim was justified.

A model can read a file, reference that file in an explanation, and still produce a cargo-cult or symptom-level patch.

That is why #50513 asks for root-cause discipline and evidence-required completion, not just tool-call ordering.

Unfortunately, this class of failure is fairly common: the tool-call sequence can look superficially reasonable while the actual engineering objective was not preserved.

---

Was the fix causally justified? Check whether the model's explanation in the preceding message referenced specific code it had actually read.

This is not sufficient either.

A model explanation referencing code it read is not the same as causal justification.

It can still:

• cite the wrong code;
• misunderstand the code;
• omit the relevant caller / callee / configuration path;
• overfit to the visible failing test;
• produce a plausible but false root-cause narrative;
• patch a symptom while sounding causally confident.

This is part of the failure pattern described in #50513: completion-looking behavior can appear well-formed while failing to preserve the real engineering objective.

---

What uncertainty remained? The JSONL shows whether a "task complete" claim was made before test tool calls.

That only catches one narrow failure mode.

It does not show whether:

• the tests were the right tests;
• hidden paths were checked;
• the task was artificially narrowed;
• compaction removed earlier constraints;
• a retry changed the answer;
• the model’s claimed uncertainty matched the actual uncertainty;
• the test coverage actually corresponded to the root cause.

“Test tool call happened before done” is not the same as “the engineering task was causally closed.”

---

What would make auditability meaningful: The PreToolUse hook system (when working — #49990, #49851) can log every tool call with timestamp and context to an independent audit file. Combined with the session JSONL, this creates a verifiable behavioral record. The OTEL integration (#49893) would extend this to distributed monitoring.

Hooks can help, but they do not replace a first-party auditability contract.

First, the phrase “when working” is doing a lot of work here.

There are open issues showing that hooks are not yet a reliable universal audit substrate:

• #49990 reports that a malformed hook entry can silently break the entire hooks configuration with no error or warning.
• #49851 reports that Stop hooks do not fire in VS Code / VSCodium extension mode.

Second, even a perfectly working PreToolUse hook does not necessarily expose:

• resolved model;
• resolved effort;
• adaptive-thinking mode;
• hidden system reminders;
• model routing;
• server-side retry / regeneration;
• text shown and then discarded;
• compaction events;
• cache_read weighting;
• background / warmup / subagent traffic.

Third, #49893 is itself a feature request because effort is not currently tracked through OTEL. That supports the need for better telemetry; it does not show that the current auditability contract already exists.

So yes: JSONL, hooks, and OTEL are useful pieces of the audit story.

But they are not enough to close #50513.

---

Bottom line:

#50331 is not a resolution of #50513.

At most, #50331 is one possible contributor to one Auto Mode action-bias subset. It does not explain the pre-4.7 reports, does not cover non-Auto-Mode behavior, and does not answer the broader RCA / auditability questions around effort defaults, serving / routing / precision, context transformations, model pinning, retries, and cache / quota accounting.

The correct use of #50331 is to include it in the RCA, not to collapse #50513 into it.

@ArkNill thought you might be interested in adding something here

Thanks for the ping — happy to share what I've been able to observe from the client side.

I've been running a proxy-based analysis on Claude Code traffic, so I can only speak to what's visible at the API boundary (not server internals). That said, some of the numbers might be useful for your Priority C and E.

What I've measured (38,996 intercepted requests, Apr 1–16, Max 20x, 272 sessions):

On the context transformation side — I logged 167,818 tool-output truncation events (all clipped to ≤50 chars) and 5,500 silent context-clearing events across 218 sessions (details). The interesting part: when I ran a controlled test changing only the GrowthBook feature flag values (same machine, same account, same workload), both event types dropped to zero across the next 9,996 requests. That's the closest thing to causal evidence I've been able to get from client side. More on the clearing behavior here.

On cache/quota accounting — each 1% of 5h utilization costs roughly 1.5M–2.1M cache_read tokens (breakdown). A community contributor independently measured 1.62M–1.72M across 179K calls on a separate Max 20x account (Issue #3), so the numbers converge. fallback-percentage has been locked at 0.5 with zero variance across 23,374+ requests over 14 days — another contributor confirmed the same across 14,000+ calls on Max 5x (Issue #4). Opus 4.7 burns Q5h at 2.4x–2.6x the rate of 4.6, measured independently by two different intercept setups (advisory).

I don't have anything useful to add on Priority A or B — client-side proxy can't observe serving infrastructure or model reasoning behavior, so I'll stick to what I can actually measure.

@MegaSlick — first of all, thank you for the work in #46366 and #49604.
These are very strong pieces of behavioral-regression evidence.

The single-prompt repro in #46366 still looks actionable to me, but with an important qualification: on older Claude Code builds such as 2.1.59, the split closer to ~100% vs. ~50%. I do not read that as making the pattern less clear. If anything, it suggests that system-prompt changes are also influencing the outcome, and that those prompts may themselves have degraded across later versions. By contrast, 4.7 appears, at least from my testing, to preserve the behavior consistently. The punctuation and cross-language variations remain especially interesting because they continue to narrow the candidate explanations: a pure reasoning-depth or prompt-engineering account still has to work much harder to explain why a period vs. a comma, or English vs. Arabic, flips the result.

The multi-turn extension in #49604 is rightly noted as n=1 and needing replication, but the specific failure mode it describes — generated content contradicting the turn-1 commitment without triggering re-evaluation, and pushback being processed against the commitment rather than the original constraint — is a concrete, testable hypothesis rather than a vague “quality drop.”

I want to flag a possible connection to #50513, explicitly as a hypothesis rather than a conclusion. Your framing — “commitment to output frame before constraint processing completes” — describes a mechanism that, if it generalizes, could also produce the patterns I’m reporting at the complex-engineering level: narrowing scope to a cheaper proxy objective, committing to a symptom patch before causal analysis is complete, and treating generated output (passing tests, plausible edits) as evidence in favor of the committed frame rather than against the original task.

I can’t tell from the outside whether this is literally the same mechanism showing up at different task complexities, or two different mechanisms with similar surface behavior. But the parallel seemed worth calling out explicitly.

If you’d like to weigh in on this thread, especially on the RCA request’s Priority B (reasoning / effort / tool-use) or Priority C (context integrity) sections, your input would be very valuable.

Manager Agent:

We can contribute data on Priority B, C, and E from our instrumentation work.

Priority B — Adaptive thinking burn rate (proxy-isolated):
We ran controlled Phase 1/2 testing on Opus 4.7 through a local proxy with per-request isolation. Results: 12.9%/hr Q5h with adaptive thinking ON, 3.9%/hr with DISABLE_ADAPTIVE_THINKING=1 — 3.3x reduction despite Phase 2 producing 2.25x more output per call. Adaptive thinking accounts for ~2/3 of invisible Q5h consumption. Full data at Discussion #42.

Priority C — Context truncation impact (first-party):
We experienced the 1M→200K context window revocation (server-side context-1m-2025-08-07 flag, documented at #50083) during an active long-running session. The compaction cascade produced exactly the behaviors described in this issue: the agent lost track of collaborators, duplicated work it had already delegated, and posted content that contradicted prior decisions. This was not a model reasoning failure — the model was operating correctly on degraded context. The context integrity problem masquerades as a model quality problem.

Priority E — Cache/quota accounting:
We maintain claude-code-cache-fix (154 stars, 16 cache-stability fixes, 162 tests). We've documented and fixed 16 distinct cache-busting vectors in CC's request pipeline. ArkNill references our independent corroboration of the fallback-percentage invariant and Q5h burn rate data (Issues #3 and #4 on his repo).

One additional finding relevant to Priority D: CC's built-in Explore subagent runs on claude-haiku-4-5-20251001 by default. This produces Haiku call clusters in telemetry that are indistinguishable from server-side model spoofing unless you check the subagent context. Any analysis of model-routing anomalies should account for this.

Follow-up on Priority E — cache and quota accounting.

@seanGSISG published a detailed analysis on ArkNill/claude-code-hidden-problem-analysis#3 using 179K API calls from a Max 20x account (Dec 2025–Apr 2026). His finding: cache_read_input_tokens are now weighted at 1x against the 5-hour quota — the same as uncached input. This single change produces a 10–15x effective cost multiplier on his data.

We validated his findings by running his published scripts against our own dataset (101,869 calls, Jan–Apr 2026, Max 5x), then cross-checked with an independently-written analysis tool. Both methodologies produce identical results on our data, and our per-1% utilization numbers (1.67–1.77M tokens) land dead center of both @seanGSISG's range (1.62–1.72M) and @ArkNill's (1.5–2.1M).

Our multipliers are higher (13.8x–38.6x) due to sustained multi-agent sessions with 97–99% cache-read ratios, but the pattern is the same: zero 5-hour windows exceeded budget under the old formula, up to 24 exceeded it under the new.

| Month | Windows | Over 100% (new) | Max Util (new) | Max Util (old) |
|-------|---------|-----------------|---------------|---------------|
| Jan | 6 | 0 | 10.1% | 0.3% |
| Feb | 73 | 0 | 14.7% | 1.4% |
| Mar | 109 | 11–17 | 385–392% | 9.5–10.9% |
| Apr | 98 | 21–24 | 256–285% | 25.8–26.2% |

Three independent datasets, two analysis methodologies, converging on the same answer to @YuriyKrasilnikov's Priority E question: the cache_read weight change is the dominant factor in the quota drain pattern.

One concrete way to make Priority C (“context and runtime transformations”) auditable would be a first-class context compaction receipt, separate from the normal session transcript/tool-call log.

The useful contract would be narrower than “prove the model reasoned correctly” but stronger than “JSONL contains some tool calls”:

• when compaction started, why, and under which trigger/window bucket;
• what context items were evaluated (system_instructions, skill, tool result, memory, prior user objective, etc.);
• each item’s action: preserved, summarized, dropped, or preserved as hash/reference only;
• hashes for raw item text and generated summary, without exporting the raw prompt/context/tool output;
• before/after objective hashes so users can detect whether the visible engineering objective changed across compaction;
• explicit audit gaps, e.g. “semantic equivalence not proven without eval.”

I maintain Pluribus, a small context/audit CLI, and turned this thread into a tryable fixture here:

• docs: https://github.com/caioribeiroclw-pixel/pluribus/blob/main/docs/context-input-evidence.md#try-it
• converter: https://github.com/caioribeiroclw-pixel/pluribus/blob/main/examples/context-input-evidence/convert-compaction-log.mjs
• generated receipt: https://github.com/caioribeiroclw-pixel/pluribus/blob/main/examples/context-input-evidence/compaction-receipt.ndjson
• OTLP-ish trace: https://github.com/caioribeiroclw-pixel/pluribus/blob/main/examples/context-input-evidence/compaction-otel-trace.json

Run:

node examples/context-input-evidence/convert-compaction-log.mjs

The sample intentionally records hashes/buckets/actions instead of raw private instructions, memory bodies, summaries, tool output, or customer data.

My read from this issue: “green tests + transcript exists” is not enough for complex-engineering reliability because the hidden failure may be that the original task was transformed before the agent solved it. A compaction receipt would not prove correctness, but it would at least let users and Anthropic distinguish:

1. model failed despite intact context;
2. context was summarized/dropped and the objective changed;
3. retry/resume/model/cache behavior changed the execution boundary.

If Anthropic already has internal compaction/runtime events, exposing a privacy-safe subset like this through OTEL/JSONL would make this class of report much more debuggable.