Prompt cache fully re-created after turns with many parallel tool calls (cache_read collapses to system+tools floor) — ~74% of cache writes wasted on Opus 4.8 / v2.1.15x

Status Open
Reported on v2.1.158
Maintainer reply None cached
Activity 11 comments · opened May 30, 2026

Describe the bug

Since ~v2.1.154 (coincident with the Opus 4.7 → 4.8 switch), the prompt cache is
repeatedly invalidated mid-session in two distinct ways, causing the entire conversation
to be re-cached from scratch over and over. Across four of my sessions, 74% of all
cache_creation tokens were waste
— re-caching content that had been cached seconds
earlier — for ~$42 of needless Opus cache-write spend in those four sessions alone.

I reconstructed the real API calls from the session transcripts (collapsing assistant
records by requestId) and read cache_read_input_tokens (cr) and
cache_creation_input_tokens (cc) from message.usage.

In a healthy session every call satisfies cr[n] ≈ cr[n-1] + cc[n-1] (the cache grows
monotonically as a prefix; cc is just the new turn's delta). Two failure modes break this:

Mode B — message-history cache invalidated (cache_read collapses to the system+tools floor) — primary, high cost

On certain turns the next request's cache_read collapses to the size of the system+tools
block only (~18.7k tokens), and cache_creation balloons to ≈ the entire message
history. The full history is re-written to cache, then read straight back on the following
turn. The system/tools prefix stays cached; only the message-history portion is lost.

Evidence it's the whole history, not a normal delta:

  • prior context 222,540 tok; a healthy cc would be ~12,902; actual cc = 216,643, cr = 18,799
  • another turn re-cached 232,372 tokens that had been cached 70 seconds earlier (healthy cc would be ~3,121)

This has more than one trigger — parallel tool-calling is the most common but not the only one:

Trigger 1 — a preceding turn with many parallel tool_use blocks (most common). Sharply
discriminated across four sessions:

| | preceding-turn parallel tool_use blocks |
|---|---|
| floor-miss calls (n=14) | mean 16.7, median 13 (dist [0,0,12,12,12,12,13,13,15,15,17,25,43,45]) |
| healthy calls (n=68) | mean 2.5, median 2 (only one ever exceeded 8) |

→ 12 of 14 floor misses in those sessions immediately follow a turn with ≥12 parallel tool calls.

Possible second trigger (tentative — single, entangled observation). In one session a
floor miss occurred on a user turn ~206s after the prior turn, with no heavy parallel-tool
turn preceding it (cache_read dropped from ~152k to the 18,706 floor, re-caching 139k). But
that same turn also invoked ToolSearch (see Mode A), so it is not a clean isolated case —
treat it as unconfirmed. The parallel-tool trigger above is the well-supported one.

These misses occur well inside Claude Code's cache TTL (Claude Code requests the 1-hour /
60-minute extended TTL
, cache_control: {ttl: "1h"}). Observed miss gaps range from 28s
to ~25 min — all < 60 min
, so none is TTL expiry.

Mode A — full cache invalidation (cache_read → 0) after ToolSearch

When ToolSearch materializes a deferred tool, the next request's cache_read drops to
0 — the whole prefix (system + tools + history) is re-created. 3 of 3 full misses in my
data are ToolSearch-preceded. This is consistent with the materialized tool schema being
added to the tools array, which sits at the front of the cached prefix. (A
deferred_tools_delta injected as message content does not break the cache, so it's
specifically the tools-array mutation.)

Confirmed live in one session: invoking ToolSearch to materialize WebFetch made the next
request's cache_read drop to 0, and rewinding to before that turn — so the materialized
tool was removed from the tools array — restored normal caching.
That both isolates the
tools-array mutation as the cause and gives a practical workaround (avoid unnecessary tool
materialization; rewind past it if it happens).

Steps to reproduce

In a long Opus 4.8 session (context > ~50k tokens), either:

  1. Mode B (parallel tools): get the assistant to issue one turn with **≥12 parallel tool

calls** (easy during codebase exploration — many Bash/Read/Grep at once). On the
next request, cache_read drops to the system+tools floor and cache_creation ≈ the
full history.

  1. Mode A (ToolSearch): call ToolSearch to materialize a deferred tool. On the next

request, cache_read drops to 0. (Rewinding to before that turn restores caching.)

Inspect ~/.claude/projects/<proj>/<session>.jsonl: group type:"assistant" records by
requestId, read message.usage.cache_read_input_tokens / cache_creation_input_tokens,
and compare consecutive API calls.

Expected behavior

Parallel tool calls and deferred-tool materialization should not invalidate the cached
conversation prefix. cache_read should keep growing monotonically; cache_creation
should only ever cover the genuinely new content of the latest turn.

Actual behavior

The cached prefix is abandoned and the entire conversation history (100k–260k tokens) is
re-written to cache on the next turn after a many-parallel-tool turn (Mode B) or after
ToolSearch (Mode A), then read back on the turn after that. This repeats every few turns,
billing cache writes (1.25× input rate) instead of cache reads (0.1×).

Likely cause (inference — cache_control is not logged in transcripts)

Mode B keeps system+tools cached but loses the entire message history, which means the
request retained the system/tools breakpoint but no cache_control breakpoint covered the
conversation prefix that was demonstrably cached moments earlier. The likely cause: Claude
Code's rolling message-history breakpoint not surviving a heavy turn — with the API's
4-breakpoint limit, a single 12–45-block turn likely pushes the rolling breakpoints entirely
inside the newly-added (uncached) block group, abandoning the breakpoint that covered the
older prefix.

Mode A is more direct: materializing a deferred tool changes the tools array at the
front of the prompt, invalidating the whole prefix cache (cache_read → 0).

The fix most likely belongs in Claude Code's cache-breakpoint / tools-array handling —
neither aggressive parallel tool-calling nor on-demand tool materialization should invalidate
an otherwise-warm prefix.

Honest confound: the CC version bump and the Opus 4.7 → 4.8 model switch happened
together, and Opus 4.8 parallelizes tool calls far more (max 4–11/turn on 4.7 vs 43–45/turn
on 4.8). But the within-session contrast controls for the model: inside one Opus-4.8
session, floor misses occur only after heavy turns and never after normal turns (same model
both times), so the parallel-tool-block count is the causal trigger and the breakpoint
handling is what fails to cope.

Impact (4 sessions analyzed)

  • 17 cache-miss turns
  • 3,062,631 cache_creation tokens billed; 2,259,367 (74%) were waste
  • $42 wasted Opus cache-write spend across just these four sessions

Environment

  • Claude Code version: 2.1.158 (also reproduced on 2.1.156; first seen at 2.1.154)
  • Model: claude-opus-4-8
  • OS: macOS 26.5 (build 25F71), Darwin 25.5.0
  • Pre-regression control: ~1,750 API calls across 8 sessions on v2.1.140–2.1.153 with

claude-opus-4-7 showed only one ≥12-tool turn and zero floor misses.

Notes / red herrings ruled out

  • Records with cache_read=0, cache_creation=0, input=0 are model:"<synthetic>"

(stop_reason:"stop_sequence", output 0) — locally-generated interrupt/stop placeholders,
not API calls. Excluded.

  • TTL expiry excluded: Claude Code requests the 1-hour / 60-minute extended cache TTL

(cache_control: {ttl: "1h"}), and every observed miss gap (28s up to ~25 min) is well
under 60 min — so no miss is explained by expiry.

View original on GitHub ↗

8 Comments

github-actions[bot] · 3 months ago

Found 2 possible duplicate issues:

  1. https://github.com/anthropics/claude-code/issues/53132
  2. https://github.com/anthropics/claude-code/issues/56293

This issue will be automatically closed as a duplicate in 3 days.

  • If your issue is a duplicate, please close it and 👍 the existing issue instead
  • To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

jeffgicharu · 3 months ago

Independent confirmation on Claude Max, Opus 4.7 xhigh, Claude Code v2.1.158.

I'm seeing the same problem you described as Mode B, but worse. A single prompt is jumping my weekly usage by big chunks at a time. One prompt takes me from 0% to 12%. The next one takes me from 12% to 20%. This happens even on turns with no parallel tool calls from me.

It started this morning. Switching back to Opus 4.7 from Opus 4.8 did not help — the jumps still happen. So the trigger may not be only "≥12 parallel tool calls" or ToolSearch. Plain model switches or effort changes look like they bust the cache too.

This also overlaps with #61984 (effort changes on Opus 4.7 cause cache misses). The two issues may share a root cause: anything that changes the prefix mid-session seems to throw away the whole cache.

Happy to run more tests and share session data if that would help.

ropdias · 3 months ago

Independent corroboration on a different config, plus a clean version of the "second trigger" you flagged as unconfirmed — and a reframe I think is useful for prioritizing a fix.

TL;DR

  • I reproduce Mode B, and I see it on the 5-minute cache tier (FORCE_PROMPT_CACHING_5M=1), not the 1h tier — so it is not TTL-specific.
  • I have ToolSearch-free floor-misses with no preceding parallel-tool burst, triggered by content injected into the middle of the conversation (a queued user message, an attachment, or a pr-link). That is the disentangled version of your tentative

second trigger.

  • Reframe: parallel bursts and content injection produce the same observable Mode B signature. This suggests they may share a common root cause, although the available transcript data cannot distinguish between a single underlying mechanism and multiple mechanisms that converge on the same cache behavior. (the message-history cache breakpoint not surviving a mid-context perturbation). One fix would address several triggers at once.

1. Mode B reproduces on the 5-minute tier (rules out TTL)

My account forces the 5-minute cache TTL. I measured the sliding window empirically: a hot prefix survives pure-idle gaps up to ~290s and only then drops — so I can cleanly exclude TTL expiry on the cases below (all happen far inside 5 minutes). The Mode B signature is identical to yours: cache_read collapses to my system+tools floor (~23k here) while cache_creation ≈ the entire message history. Preceding-turn parallel tool_use count discriminates here too (bursts of 14 / 22 / 28 → floor miss). Because it happens on the 5m tier as well, the bug is independent of the requested TTL.

2. A clean, ToolSearch-free "second trigger": mid-turn content injection

You couldn't isolate your non-parallel floor-miss because it was entangled with ToolSearch. I have three floor-misses in sessions where ToolSearch was never called (zero deferred-tool materialization) and the preceding turn had no parallel burst (≤1
tool_use). In each, the collapse immediately follows content inserted into the middle of the history:

| idle gap | cache_read before → after | cache_creation | preceding turn | injected just before the miss |
|---|---|---|---|---|
| 57s | 248,142 → 23,715 | 231,005 | no burst | pr-link + user message |
| 82s | 205,003 → 22,429 | 187,546 | no burst | attachment + pr-link + queued user message |
| 97s | 154,904 → 23,237 | 135,223 | no burst | queued user message (queue-operation → user text) |

System+tools stays cached (floor, not 0) → Mode B, not Mode A. The "queued user message" case is notable: the user typed while the assistant was mid-turn, so the message appears to have been inserted into the active conversation state rather than simply appended after completion, and the whole prefix was re-cached.

3. The unifying picture (taxonomy for triage)

The observed cache invalidations appear to cluster into three buckets. These categories are descriptive rather than definitive, and some may ultimately turn out to share implementation details:

| Trigger | Class | Same root cause as burst? | Mitigable in Claude Code? |
|---|---|---|---|
| Turn with many parallel tool_use (Mode B) | Likely bug | — | Yes — breakpoint placement |
| Mid-turn content injection: queued msg / attachment / pr-link (Mode B) | Likely bug | Yes | Yes — same fix, or append injected content at the tail instead of mid-history |
| ToolSearch materializes a deferred tool (Mode A, cache_read → 0) | Borderline (the tools array is part of the prefix) | No | Partial — reserve the slot / place materialized tool after a breakpoint / warn |
| Idle > TTL | Expected | No | Not a bug — but document the cost trade-off of forcing 5m |
| Compaction / model switch (/model, /fast) | Expected (inherent) | No | No — only surfacing/warning |

The two rows exhibit the same externally visible failure mode (Mode B): a content change in the middle of the history abandons the cached message-history prefix. Whether they are literally the same failure internally cannot be determined from transcript data alone. One possible explanation is that the rolling history breakpoint is being displaced by a heavy or injected turn. This would be consistent with the observed behavior, although transcript-level data does not provide direct visibility into cache breakpoint placement. (Inference, cache_control isn't in the transcript, but burst and injection producing the byte-identical Mode B signature strongly points to one mechanism.)

The key observation is not the proposed mechanism itself, but the repeatable correlation: both parallel-tool bursts and mid-turn content insertions are followed by the same cache-collapse pattern while remaining well inside the active cache TTL window.

4. Why this is worth fixing (cost + UX)

  • Direct cost / quota waste. This is pure deadweight: content cached seconds earlier is re-encoded as a cache write (1.25×–2× input) instead of a read (0.1×). On metered API it's overspend (you measured ~74% waste / ~$42 across four sessions). On subscription plans it silently burns the user's 5-hour limit, and the platform still pays the compute to re-process tokens that produce nothing new — a lose-lose.
  • Perceived limit longevity (the big UX lever). Most users don't know prompt caching exists. They just see "limit reached" sooner than expected. Cutting avoidable re-caching makes the same plan feel much larger, with zero quality cost — arguably one of the highest-leverage satisfaction wins available, precisely because it's invisible today.

5. Suggested directions

  • Breakpoint placement (highest value). Keep one stable cache_control breakpoint anchored over the maximal stable message-history prefix, so a heavy turn or a mid-history injection only re-caches from the perturbation forward, not the entire history. If the hypothesis above is correct, this change could address both observed failure modes.
  • Where injected content lands. Append queued messages / attachment / pr-link expansions as trailing blocks rather than re-rendering them mid-history, so they don't invalidate the prefix before them.
  • For the inevitable cases, tell the user. TTL expiry, compaction, and model switches can't avoid a rewrite — but they're silent today. A small signal ("cache rebuilt: model switch" / "…: 6-min idle" / "queuing a message mid-turn rewrote the cache") turns a silent footgun into an informed choice. Even a one-line reason on the statusline when cache_read collapses would help enormously.

Reproduction

Self-contained, collapses the split content-block lines per request and prints, per turn, cache_read, cache_creation, and the number of parallel tool_use blocks. Look for rows where cr collapses to your system+tools floor while cc ≈ the prior context, and check the preceding row's parallel_tools (and whether a queue-operation / attachment / pr-link sits just before):

TX=~/.claude/projects/<proj>/<session>.jsonl
jq -r 'select(.type=="assistant" and .message.id) |
  [ .message.id, .timestamp,
    (.message.usage.cache_read_input_tokens // -1),
    (.message.usage.cache_creation_input_tokens // -1),
    ((.message.content // []) | map(select(.type=="tool_use")) | length) ] | @tsv' "$TX" |
awk -F'\t' '
  {id=$1; if(!(id in seen)){seen[id]=1; ord[++k]=id; ts[id]=$2}
   if($3>cr[id])cr[id]=$3; if($4>cc[id])cc[id]=$4; tu[id]+=$5}
  END{for(i=1;i<=k;i++){d=ord[i];
    printf "%s  cr=%d  cc=%d  parallel_tools=%d\n", ts[d], (cr[d]<0?0:cr[d]), (cc[d]<0?0:cc[d]), tu[d]}}'

To see the injected events themselves, grep the same transcript for "type":"queue-operation", "type":"attachment", and "type":"pr-link" and line their timestamps up against the misses.

Isolation test (mirrors your ToolSearch rewind): in a hot session, queue a message while the assistant is mid-turn (or paste a PR link), compare the next request's cache_read, then rewind past it. If caching restores, the injection point is confirmed as a breakpoint-perturbing trigger.

dehrman · 2 months ago

Are folks seeing this resolved now? Aggregate data on my side seems better for the past 2 days, but I haven't nailed down if that was actually fixed in a recent release.

omrikais · 2 months ago

Large-scale follow-up — the dominant trigger is the user-typed-turn boundary, not parallel tool calls (and a matched-gap control that rules out TTL)

Original poster here. I re-ran the analysis at much larger scale (1,997 of my own sessions, 128,536 deduplicated main-thread API calls, vs. the 4 sessions / 14 misses in the original report) and it corrects my own original trigger attribution. Same Mode B signature — cache_read collapses to the ~18.8k system+tools floor while cache_creation re-writes the conversation history, all inside the 1h TTL — but the ≥12-parallel-tool framing was an artifact of a small, exploration-heavy sample. The real dominant axis is whether the request is the first one after a freshly typed human message.

Method: dedup assistant records by (message.id, requestId); track cache continuity per thread and per model (so model switches and subagent contexts don't masquerade as misses); flag a Mode B collapse when cache_read[n] < cache_read[n-1] + cache_creation[n-1] by ≥20k tokens while the prompt did not shrink (compaction excluded); sidechains/subagents excluded. 97.3% of all consecutive call-pairs are perfect cache hits (cache_read[n] exactly equals the established prefix), so the misses are a clean, discrete signal.

1. The collapse is concentrated at human-typed turns (7.3× over-represented)

Classifying each "could-rebuild" call by what immediately precedes it (a typed human message vs. a tool_result continuation mid-agentic-loop):

| boundary | share of all requests | share of Mode B collapses | hazard (collapse % of calls) |
|---|---:|---:|---:|
| first request after a human-typed message | 8.9% | 65.1% | 8.65% |
| mid-loop request after a tool_result | 91.0% | 34.9% | 0.46% |

A random server-side eviction can't produce this — the API has no idea whether a request follows my keystroke or a tool result, so it would evict both at the same per-call rate. The ~19× hazard gap between the two is a fingerprint of the client emitting a different prompt prefix when it appends a new user turn.

2. Matched-gap control — it is not TTL expiry

Human turns naturally follow idle gaps, so I split the hazard by the gap before the request. If this were TTL, the human/tool gap should only appear near the 1h boundary. It doesn't:

| gap before request | hazard at human-typed boundary | hazard at tool-result boundary | ratio |
|---|---:|---:|---:|
| < 1 min | 3.78% (199/5,259) | 0.38% (407/106,984) | 9.9× |
| 1–5 min | 7.24% (310/4,280) | 0.38% (25/6,600) | 19.1× |
| 5–60 min | 16.84% (254/1,508) | 3.95% (62/1,570) | 4.3× |
| > 1 h (past TTL) | 78.3% (217/277) | 96.9% (31/32) | 0.8× |

The bottom row is the built-in control: past the 1h TTL everything re-caches (~80–97%) and the human/tool asymmetry vanishes — exactly what real TTL expiry looks like. But in the sub-5-minute window, where the 1h cache cannot have expired, the first request after a typed message is 10–19× more likely to lose its entire conversation cache than a tool continuation. So this is a within-TTL, client-side prefix mismatch, not expiry.

3. It's a per-turn hazard that scales with session length (not "one event per session")

It is memoryless per turn: the per-call hazard is flat (~1%) regardless of session length, while collapses-per-session grow linearly with length.

| session length (calls) | collapses / session | collapse % per call |
|---|---:|---:|
| 3–20 | 0.14 | 1.46% |
| 50–100 | 0.92 | 1.27% |
| 100–200 | 1.40 | 1.02% |
| 200–400 | 2.60 | 1.01% |
| 800+ | 21.75 | 1.53% |

(In the last 14 days, 118 of my sessions: 32% had zero collapses, 52% exactly one, 16% two or more — the zero-collapse sessions are nearly all short, the multi-collapse ones long. Position within a session is uniform, not clustered at the start, so it's not a cold-start or first-follow-up artifact.)

4. Version trend (length-independent per-call hazard)

| CC version band | collapse % per call | hazard specifically at human-turn boundaries |
|---|---:|---:|
| 2.0.x | 3.81% | 34.7% |
| 2.1.20–29 | 1.51% | 17.3% |
| 2.1.40–49 (best) | 0.48% | 4.6% |
| 2.1.90–99 | 0.41% | 3.6% |
| 2.1.110–129 | ~1.2–1.3% | ~2.5% |
| 2.1.170–179 (current) | 1.12% | ~3.4% |

Two things stand out. First, the human-turn-boundary hazard specifically has fallen dramatically over time (34.7% → ~3%), so this was being worked on and substantially improved. Second, the overall per-call rate hit a trough around 2.1.40–2.1.99 (~0.4–0.8%) and then roughly doubled in the 2.1.110+ band I'm now on (~1.1%) — a modest but real regression. (Correction to my original report: "first seen at 2.1.154" was a perception artifact — the phenomenon predates it across every version; what coincided with 2.1.154 was the Opus 4.8 switch raising parallel-tool counts, which is a secondary contributor, not the origin.)

What this means for a fix

This corroborates @ropdias's reframe ("the message-history cache breakpoint not surviving a mid-context perturbation") and @jeffgicharu's observation that it happens "even on turns with no parallel tool calls." The most common perturbation is simply appending a new user turn. Because cache_read collapses to exactly the system+tools floor, whatever changes sits right after that block — the very front of the conversation portion — which is consistent with either a rolling message-history breakpoint that fails to re-cover the older prefix on a new turn, or non-deterministic ordering of front-of-prompt content (cf. #49038, where tools[0] reshuffles between calls). Parallel-tool bursts (original Mode B) and mid-conversation content injection (@ropdias) are additional triggers that converge on the same signature; the user-turn boundary is just the most frequent one.

Honest caveat (unchanged): the JSONL stores conversation rows, not the exact bytes + cache_control markers sent to the API, so the where (human-turn boundary) and who (client-side, by the asymmetry) are proven statistically, while the exact byte-level diff is inferred. Capturing one typed turn with ANTHROPIC_LOG=debug and diffing its prompt prefix against the prior request would pin the precise offending block.

Environment

  • Claude Code: 2.1.177 (current); trend computed across 2.0.x–2.1.177
  • Model: claude-opus-4-x (analysis spans 4-5/4-6/4-7/4-8)
  • OS: macOS 26.5 (build 25F71)
  • Plan: Claude Max subscription (first-party Anthropic — not pay-as-you-go API, Bedrock, or Vertex)
TonyTheCat · 2 months ago

Confirming this independently with an additional data point: the same collapse-to-system+tools-floor happens in headless claude -p --resume (entrypoint sdk-cli), not just interactive — and at scale it's the dominant cost driver for us.

Setup: Claude Max subscription (CLAUDE_CODE_OAUTH_TOKEN), long-lived claude -p --input-format stream-json --output-format stream-json --resume <sid> subprocess handling many turns. v2.1.183 (seen across 2.1.x). Reconstructed from ~/.claude/projects/*/*.jsonl per-assistant message.usage, same method as you.

What we see:

  • On an affected turn, cache_read collapses to a ~15,738-token floor (the system/MCP prefix) and the entire conversation body re-caches as cache_creation — your exact "Mode B" signature. 108/117 collapses land at the identical ~15,738 size.
  • Frequency in prod sdk-cli: ~2.52% of turns. Collapse turn-index median 143, never before idx 5, context median ~192k.
  • Interactive cli on the same machine A/B's clean: only ~0.91% "drops," and those are cache_read=0 cold starts (new tabs), not the 15,738 body-drop. Interactive holds the body cache append-only; -p --resume does not. This points at a --print/--resume-path difference, not just parallel-tool-calls.

Why it dominates the bill on a subscription: measured on the /usage gauge — 5.0M cache-read moved it 0pp, +177.6k output moved it +6pp ⇒ cache-read ≈ free, cost ≈ cache_write·1.25 + output·5. So a body re-cache at ~192k context is ~2x the limit consumption of a warm turn.

Ruled out (controlled tests, each held warm / 0 body-drops): version regression (warm on both 2.1.68 and 2.1.183), solo deep -p --resume, 6-way concurrency at prod-matched shape (idx-140, ctx-182k), respawns, the 5-min TTL (collapses 0.2–3.2 min apart), browser/MCP, subagents, auth method. It only manifests at real production scale/shape, which is why we can't offer a minimal repro — only the statistical signature.

Candidate mechanism: cache-breakpoint starvation at large context — the 4 cache_control breakpoints consumed by the system/MCP prefix, leaving the growing body uncached under --print --resume. Matches the older #42338 / #34629 resume-cache reports but persists on 2.1.183.

Happy to share the parsing scripts / anonymized usage series if useful.

Esras · 1 month ago

Apologies to anyone that isn't an Agent reading this Sonnet-generated wall of text that's a shot version of my debug session.

Additional data point for the "second trigger" (unconfirmed, entangled with Mode A)

Hit what looks like the same pattern described here, via a different setup: Agent SDK (entrypoint: sdk-ts) behind Zed's ACP integration, not the terminal session described above.

Observed:

  • cache_read_input_tokens: 15870, cache_creation_input_tokens: 388381 (ephemeral_1h_input_tokens) on a single turn — floor-collapse, not a full zero, so this looks like Mode B rather than Mode A.
  • Gap since the prior turn: ~4 min (well under the 1h cache TTL, ruling out natural expiry).
  • No heavy parallel-tool-call turn immediately preceding it — the prior user action was a /model opus[1m] slash command reissuing the same value the session was already running (set ~19h earlier in the same session, unchanged since).
  • The collapsing turn itself contains a ToolSearch call (query: "select:SendMessage", total_deferred_tools: 19) in the same response as the cache spike, consistent with the ToolSearch/deferred-tool entanglement already flagged as the reason the second trigger couldn't be isolated.
  • Git branch unchanged (main), no effort/thinking-level change, model string unchanged (claude-opus-4-8) across the whole window.
  • Model/version: claude-opus-4-8, claude-code v2.1.198.

requestId (collapsing turn): req_011Cchixm4JNMoyNNQwyquDX

Why this might help: it reproduces the "short gap + no parallel-tool-call turn + ToolSearch present" shape from a different client/entrypoint, without a version, git, or idle-TTL confound. Doesn't resolve the Mode A/B entanglement, but is a second independent occurrence of the pattern that was previously reported only once.

hksbindra · 1 month ago

Independent confirmation of @omrikais's corrected finding — plus a named, reproducible producer for the mutation, and a structural fact that settles the cache_control inference

Different corpus, different config, same conclusion. 155 sessions, 4,639 deduplicated main-thread requests, 868 user-turn boundaries, 49 real Mode B collapses totalling 7,555,163 missed tokens. Claude Code 2.1.207, VS Code extension, Windows 11, Opus 4.8, 1h TTL.

TL;DR

  • Your corrected trigger attribution replicates exactly: 49 of 49 of my collapses sit at a user-turn boundary. 0 of 3,774 mid-chain requests ever produced one.
  • The API now names the cause outrightmessage.diagnostics.cache_miss_reason = messages_changed. Nobody in this thread appears to be reading it. It removes all the cr/cc inference.
  • From captured raw request bodies: there is exactly one cache_control breakpoint in the messages array. Not a rolling set. This settles the mechanism you flagged as inference — and it's worse than the 4-breakpoint story.
  • I have a controlled A/B on a concrete producer: the VS Code extension's claudeCode.autosave setting. It reproduces on demand, and I caught the injected payload being born, edit by edit.
  • @ropdias's mid-turn content-injection trigger and my autosave trigger are, I think, the same bug with two different producers. One fix covers both.

---

1. The boundary finding replicates

| boundary | requests | collapses |
|---|---:|---:|
| first request after a typed human message | 868 | 49 |
| mid-chain (after a tool_result) | 3,774 | 0 |

Not "over-represented" in my corpus — absolute. Every collapse I have ever recorded, across 155 sessions, landed on the request that rebuilds the message array after a human turn. This is a much smaller corpus than your 128,536 calls, so treat it as corroboration rather than new evidence; the point is that a second, independent dataset with a different client config produces the same fingerprint you found.

Two more descriptive facts that may help others rule things in and out:

  • The collapse has a floor. 37 of my 49 land on 13.1k–20.4k cache_read (my system residual) regardless of whether the conversation was 40k or 480k tokens. There is no partial credit and the size of the loss tells you nothing about the depth of the mutation.
  • It has a size threshold. In 49 events it has never fired below 31,238 tokens (median 183,602; only 4 of 49 below 50k). Small repro sessions come back clean and prove nothing — I voided three experiments learning that.

---

2. Stop inferring the collapse — the API reports it

This thread (myself included, until recently) reconstructs Mode B from cache_read[n] < cache_read[n-1] + cache_creation[n-1]. That heuristic is sound but it can't distinguish a client-side prefix mutation from TTL expiry, a model switch, or a tools-array change — hence the matched-gap controls everyone has had to build.

You don't need them. Claude Code writes the server's own verdict into the transcript:

// ~/.claude/projects/<proj>/<session>.jsonl, type:"assistant" records
message.diagnostics.cache_miss_reason = {
  "type": "messages_changed",          // or "tools_changed", etc.
  "cache_missed_input_tokens": 91773
}

Dedupe assistant records by requestId, then read that field. messages_changed means the server saw a message that had already been sent come back different. That is a client-side rewrite of committed history, stated by the API, with the exact token count of what was lost.

Every one of my 49 collapses carries type: "messages_changed". @omrikais — re-running your 128,536-call analysis against this field instead of the arithmetic would settle the taxonomy in this thread definitively, and separate the real messages_changed events from anything that is actually TTL or tools_changed (Mode A).

---

3. cache_control: ground truth, not inference

The original post is careful to flag its mechanism as inference — "cache_control is not logged in transcripts" — and proposes that Claude Code's rolling message-history breakpoint fails to survive a heavy turn, with the API's 4-breakpoint limit pushing the rolling breakpoints inside the new block group.

I ran a local proxy and captured the raw /v1/messages request bodies. The real structure:

system[1]         -> cache_control: ephemeral, 1h
system[2]         -> cache_control: ephemeral, 1h
msg[18] block[4]  -> cache_control: ephemeral, 1h   <- the ONLY one in the entire messages array
tools[]           -> none

There is no rolling set. One breakpoint, pinned to the last block of the last user message. Nothing mid-conversation, nothing to fall back to.

This matters for triage because it explains the floor: a mutation at message 3 and a mutation at message 93 are identically catastrophic. The breakpoint isn't being "pushed out" by a heavy turn — there was never a second breakpoint protecting the older prefix in the first place. Any mutation at any depth takes everything.

I also captured the mutation class directly: a 6,361-character /compact block was added to msg[16] in one request and removed again in the next — an already-sent message being rewritten in place. I don't use /compact, so that isn't my trigger, but it proves the harness does retroactively edit committed messages.

---

4. A named producer, with a controlled A/B: claudeCode.autosave

This is the part I think is new. For the VS Code extension, I can now reproduce Mode B on demand.

Same machine, same workspace, same 40 files, same operator, same edit pattern. One variable:

| claudeCode.autosave | requests | edits | distinct files | messages_changed |
|---|---:|---:|---:|---:|
| OFF | 147 | ~60 | 40 | 0 |
| ON | 61 | 41 | 40 | 1 |

With it off I attacked the cache for six hours — 45 sequential read-only calls, 19 edits to an open file with live lint errors, 40 sequential writes to 40 new files, long idle gaps. cache_read grew monotonically to 255,933. Zero collapses.

With it on, a fresh session died on the 41st edit:

01:34:00   cache_read = 125,948   cache_creation =     761   --
01:36:03   cache_read =  20,658   cache_creation = 106,250   messages_changed (91,773 missed)

The mechanism, observed rather than inferred

"Autosave — automatically save files before Claude reads or writes them" is the only setting that puts the IDE in the loop on every file operation the agent performs. With it on:

39 of 40 sequential Edit calls came back carrying an <ide_diagnostics> payload, delivered as a PostToolUse:Edit hook injection — a <system-reminder> wrapping the diagnostics. It fired on the very first edit:

probe_01.py:8  "Undefined name `_witness_01`"            Error     (Ruff)
probe_01.py:8  "\"_witness_01\" is not defined"           Error     (Pylance)

Three things follow:

  1. It is PostToolUse, so it is structurally late. The diagnostic caused by an edit is not in the request that made the edit — it lands in the next build of the message array. Late arrival, observed directly.
  1. Reads are free; writes are the trigger. Same session, all 40 files open in the editor, language server live: 40 reads produced zero payloads. The payload appears the instant a write happens. This cleanly deconfounds tool volume from the cause.
  1. The race, caught in the act. Edit 18 published nothing. Edits 1–17 and 19–40 all did, and edit 19's payload named only its own file — so 18's diagnostic wasn't late-folded into a later sample either. It was simply still outstanding when the hook took its snapshot.

That one gap is the whole bug. LSP textDocument/publishDiagnostics is a server-initiated push — there is no synchronous "give me diagnostics now" path in a save handler. The producer (language server) and the consumer (the PostToolUse hook) are not synchronised. The hook takes a point-in-time snapshot of an asynchronously-published stream and embeds it into an append-only, content-addressed, prefix-cached structure whose entire contract is that a sent message never changes. A diagnostic that misses its snapshot has nowhere to go but a later rebuild — of a message that has already been serialized, sent, and cached.

An async producer writing into an immutable, already-transmitted structure. It is a race against the cache, and the cache always loses.

Workaround available today for VS Code extension users: disable claudeCode.autosave. After disabling it, this corpus recorded ~350 requests and ~118 edits across four sessions with zero invalidations — sessions that would previously have been near-certain to collapse.

(Scope note: I observed the injection, the missed sample, and the resulting collapse. I did not capture the byte-level rewrite of the already-sent block — #76606 did, from raw request bodies. That report and this one are looking at the same object from opposite ends: they see it mutate; I watched it get born and timed the miss.)

---

5. @ropdias is right — and I think our two triggers are one bug

@ropdias found ToolSearch-free, burst-free collapses caused by content injected into the middle of the conversation: a queued user message, an attachment, a pr-link — with the standout case being "the user typed while the assistant was mid-turn, so the message appears to have been inserted into the active conversation state rather than simply appended."

That is the same bug I have, with a different producer.

  • My producer: the IDE, pushing diagnostics asynchronously into an already-sent message.
  • His producer: the client, splicing user/attachment content into the middle of an already-sent history.

Both are late writes into committed state. Both are total, because of the single tail breakpoint in §3. And it explains something I could not: 16 of my 49 collapses occurred in sessions with no IDE payload logged anywhere. Autosave cannot account for those. Mid-turn content injection can — and I habitually type while the agent is working, across parallel sessions.

So I'd endorse his reframe and sharpen it: the trigger is not parallelism, not TTL, not reads, and not any one producer. It is _any_ mutation of a message after it has been sent — and the single tail breakpoint converts every one of them into a total loss.

---

6. On the original ≥12 parallel tool calls framing

For the record, my corpus agrees with @omrikais's own retraction, though it can't add much:

| max parallel fan-out on preceding turn | n | collapses | rate |
|---|---:|---:|---:|
| 0–1 | 597 | 19 | 3.2% |
| 2–3 | 202 | 22 | 10.9% |
| 4–7 | 54 | 8 | 14.8% |
| 8–11 | 7 | 0 | 0.0% |
| 12+ | 5 | 0 | 0.0% |

My widest turns never collapsed — but with n=7 and n=5 those cells have no power to test the original claim, and I won't pretend otherwise. The load-bearing row is the first one: 19 collapses followed turns with 0–1 tool calls. Parallelism is plainly not necessary. That is consistent with the retraction, not independent support for it.

(Method note for anyone re-running this: Claude Code writes each parallel call as its own JSONL record, so naive per-record counting silently reads a fan-out of 1. Group tool_use blocks by requestId.)

---

Asks

  1. Never mutate an already-sent message. Late-arriving context — IDE diagnostics, queued user text, attachments, hook reminders — must append to the newest message, never rewrite one that has been serialized and cached. If it missed its snapshot, it belongs in the next turn or nowhere.
  2. More than one cache_control breakpoint in the messages array. A single checkpoint at the tail means there is nothing to fall back to and every invalidation is total, no matter how shallow the mutation.
  3. Document that claudeCode.autosave carries this cost until (1) lands. It is on by default, and it is silently expensive on precisely the workload Claude Code exists for: long sessions that edit a lot of files.

Showing cached comments. Read the full discussion on GitHub ↗