Auto-compaction: two compounding cost bugs — stale precompute keeps ~200k tokens verbatim, and that prefix is repeatedly cache-created instead of cache-read
Summary
Two separate but compounding cost problems in the auto-compaction path. The first bloats the conversation; the second then pays to re-ingest that bloat repeatedly.
- Stale precompute → shallow compaction.
/compactconsumes a background precompute summary that was baked ~47 minutes earlier. Consuming it only summarizes up to the old anchor and preserves every message since (~200k tokens) verbatim, so compaction barely reduces context. There's no staleness/tail-size guard that would rebuild the precompute or fall back to a fresh summary. - **Cache-sharing miss → the bloated prefix is cache-created instead of cache-read. Because the main-thread request anchors its body
cache_controlbreakpoint on the last (volatile) message, the background precompute (and the following turn) cannot reuse the cached body — they re-create the full ~196k at the cache write rate ($10/MTok 1h) instead of the read** rate ($0.50/MTok), ~20× more, and it recurs on every precompute.
Environment
- Claude Code version: 2.1.186 (
claude-cli/2.1.186, native install) - Model:
claude-opus-4-8 - Platform: macOS (darwin arm64), Node v24.3.0
- Relevant settings (
settings.json→env, possibly influencing compaction trigger frequency):
"CLAUDE_CODE_DISABLE_1M_CONTEXT": "1",
"CLAUDE_AUTOCOMPACT_PCT_OVERRIDE": "80"
(The first caps the context window at 200k instead of 1M; the second sets the auto-compact threshold to 80% of the window — together they make auto-compaction/precompute fire earlier and more often.)
Issue 1 — Stale precompute kept ~200k tokens verbatim
Precompute lifecycle from the debug log:
20:13:24 precomputed compact: started (main, 211 msgs, ~147k tok, attempt 2, trigger api_response)
20:15:07 precomputed compact: ready (103s)
21:02:46 precomputed compact: consumed (main, ready) <- consumed 47 min after it was built
- The
/compactran at 21:02:45; the summary it consumed is timestamped 20:15:07 — a 47-minute-old summary. - ~250 messages (180 assistant / 70 user) were created in that 47-minute window. The consume path preserves all messages-since-anchor verbatim (
messagesToPreserve = [...precompute.messagesToPreserve, ...messagesSinceAnchor]), so none of them are summarized. - Result: the post-compact conversation stays huge — observed at the next precompute as 452 messages / ~238k tokens. A fresh summarization at 21:02 would have collapsed those ~250 messages into a short summary instead.
Why it's a bug: a precompute is consumed regardless of how stale it is or how large the verbatim tail has grown. When the tail is ~200k tokens, the "compaction" frees almost nothing.
Issue 2 — The bloated prefix is cache-created, not read
Two back-to-back requests, 9s apart, on that ~238k-token conversation (162 messages):
REQ1 — main-thread user turn (source=repl_main_thread):
"usage":{"input_tokens":1494,"cache_creation_input_tokens":196093,"cache_read_input_tokens":39766,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":196093},"output_tokens":312,"service_tier":"standard","inference_geo":"not_available"}
REQ2 — background precompute (source=compact, forked reactive-compact):
"usage":{"input_tokens":3737,"cache_creation_input_tokens":195784,"cache_read_input_tokens":39766,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":195784},"output_tokens":6549,"service_tier":"standard","inference_geo":"not_available"}
Debug log for REQ2:
21:16:55 precomputed compact: started (main, 452 msgs, ~237,765 tok, attempt 3, trigger api_response)
21:16:56 [API REQUEST] /v1/messages source=compact
21:18:11 reactive-compact finished: cacheRead=39766 cacheCreate=195784
21:18:11 precomputed compact: ready (main, 75333ms)
cache_read_input_tokens is pinned at 39,766 (system prompt + tool defs only) in both; the entire ~196k message body is cache-created both times. Structural diff of the two request bodies:
- Messages
[0..159]are content-identical. - They diverge only at
messages[160](the final user turn): the precompute appends aCRITICAL: Respond with TEXT ONLY...+<analysis>/<summary>instruction. cache_controlbody breakpoint: REQ1 onmessages[160](the volatile last message); REQ2 onmessages[159].- The main thread only cached up to
messages[160](which the precompute then modifies), so the precompute has no usable checkpoint atmessages[159]and re-creates from the system head.
Why it's a bug: the precompute is intended to cache-read the conversation (tengu_compact_cache_prefix / tengu_compact_cache_sharing). The breakpoint-on-the-volatile-message placement defeats that, so the share misses and the body is re-created. The tengu_compact_cache_sharing_fallback event is presumably firing; its reason would quantify fleet-wide frequency.
Impact
- Issue 1: every post-compaction turn carries ~200k unnecessary tokens (≈ +$0.10/turn in cache reads at $0.50/MTok, plus larger periodic re-creates), and the next precompute must summarize a needlessly large ~238k context.
- Issue 2: each background precompute re-creates the full body at write rates — ~196k × ($10 − $0.50)/MTok ≈ ~$1.9 of avoidable spend per precompute. This session ran the precompute 3 times (18:22, 20:13, 21:16) → ~$5+ avoidable, and it scales with conversation size (so the longest sessions pay most). The single pair above cost ≈ $4.16, ~$3.9 of it avoidable cache-creation.
Suggested fixes
- Issue 1 — add a staleness / tail-size guard. When the messages-since-precompute exceed a token threshold (or the precompute is older than N minutes / M messages), rebuild the precompute or fall back to a fresh live summarization, so
/compactactually reduces context instead of preserving ~200k verbatim. - Issue 2 — align cache breakpoints. Either have the main-loop request place a breakpoint at the last committed message boundary (before the in-flight turn) so forks/precomputes can read up to it, or have the precompute reuse the main thread's existing breakpoint and append its instruction after that boundary — so only the small instruction delta is cache-created.
Repro / detection
- Long Opus session with auto-compact enabled, until
precomputed compact: startedfires. - Issue 1: compare the consumed summary's timestamp to the
/compacttime, and check how many messages were preserved verbatim. - Issue 2: on the
source=compactrequest,cache_creation_input_tokens≈ full context whilecache_read_input_tokens≈ only the system prompt indicates the miss. Checktengu_compact_cache_sharing_fallbackfor frequency/reason.
Showing cached comments. Read the full discussion on GitHub ↗
6 Comments
Workaround + configuration switches (and version history)
While investigating, I mapped which settings/env vars actually affect this. Summarizing in case it's useful for triage and for others hitting the cost.
Clean mitigation for Issue 1: disable the precompute, keep threshold auto-compaction
This disables the background precompute (so no stale pre-summarized version is ever consumed) without disabling threshold-triggered auto-compaction — at the threshold it just runs a fresh live summarization instead.
The wiring (2.1.186 bundle): the precompute is armed only through one master gate (
I8n()in the deobfuscated bundle):The consume-vs-fresh branch falls back cleanly when no precompute exists:
And the threshold trigger (
WKp/XSo) does not referenceI8n()— it decides purely on token level +ZR()/Yz()/v4(). So turning the setting off changes the reactive path from consume → fresh; it does not stop auto-compaction.Note:
precomputeCompactionEnableddefaults totrue, so the precompute behavior is on unless explicitly disabled. There is also the internal statsig gatetengu_sepia_moth(defaultfalse) insideI8n(); statsig gates have no user-facing env/file override in this build, so the setting is the only user control.Env vars that do NOT do what their names suggest (checked, ruled out)
CLAUDE_CODE_COLD_COMPACT=1— only disables prompt-cache-prefix sharing on the manualNpt/malpath (let S = !cold && it("tengu_compact_cache_prefix", true)). It does not stop the reactive path from consuming a precompute. Wrong lever.DISABLE_AUTO_COMPACT=1— insideZR(); disables all auto-compaction (and transitively the precompute). Too broad.CLAUDE_CODE_DISABLE_PRECOMPACT_SKIP— despite the name, unrelated to summarization precompute; it controls whether large transcript JSONL files are read in full vs. only the post-boundary buffer. Ignore for this.CLAUDE_CODE_AUTO_COMPACT_WINDOW— pure threshold tuning (when compaction triggers); no effect on consume-vs-fresh.Version history of the setting
precomputeCompactionEnabled(the user-facing key) was introduced in 2.1.181 (defaulttrue). Absent in 2.1.145 / 2.1.154 / 2.1.177.precomputed compact:/reactive-compact/tengu_sepia_moth) predates the setting — already present in 2.1.145, and absent in 2.1.88 (where compaction was synchronous/live-only). So precompute landed somewhere in the 2.1.89–2.1.145 range, but the user-facing toggle only arrived at 2.1.181.Caveat on scope
precomputeCompactionEnabled: falseaddresses Issue 1 (stale precompute → ~200k verbatim) and removes the recurring background-precompute cache-creation. It does not by itself fix Issue 2 — the cache-breakpoint-on-the-volatile-last-message placement can still cause a fresh threshold summarization (which runs right after a main-thread turn) to re-create the body instead of reading it. The breakpoint-alignment fix is still worth making.Follow-up: the cache-sharing fix for Issue 2 already exists in the code, gated behind a default-off flag — and there's no user workaround
Tracing the breakpoint placer in 2.1.186, the alignment fix for Issue 2 is already implemented; it's just disabled by default.
The message cache-breakpoint placer
VImwrites one body breakpoint on the last (volatile) message by default, but also contains a stable-boundary pin keyed to aforkPointUuid(l4n(...)= last stable assistant turn) — which both the main turn and the compaction fork already pass:With
qmt()true, both the main turn and thesource=compactfork pin a breakpoint at the same stable boundary, so the ~196k body is cache-created once on the main turn and cache-read by compaction. This single flag would fix Issue 2 for both compaction paths (the background precompute build and the live threshold summarization both flow through the sameeIn → v1d → CI(skipCacheWrite:true) → VIm).Two points that make this worth prioritizing:
CLAUDE_CODE_AUTO_COMPACT_WINDOW),skipCacheWrite, TTL/scope betas, all compaction modes. Every full-compaction builder converges onskipCacheWrite:true+ an appended TEXT-ONLY turn + noforkPointUuidconsumed (becauseqmt()is off), so the body is always re-created.tengu_basalt_spuris statsig-only with no env/settings override. So enabling the flag is the only way to fix this for affected users.ZP()gates it on first-party auth, so even where the flag is on, non-first-party auth modes won't get the pin. If the intent is to fix this broadly, theZP()guard likely needs relaxing too.Suggested action: enable
tengu_basalt_spur(and theSsl()/tengu_basalt_scarpsub-case) for the relevant cohorts, and consider whether theZP()first-party guard should be dropped so the cache-read path applies across auth modes.Great decomposition into two separate bugs. They compound but they have different fix surfaces.
On bug #1 (stale precompute, 200k verbatim tokens): the core problem is that the consume path has no staleness guard — when the 47-minute-old summary is consumed, it only summarizes up to its original anchor, and everything since becomes verbatim preserve. The session ends up larger after compaction than it would have been without it.
cozempic addresses this specific footprint:
cozempic treat current -rx aggressive --executeprunes those 250 verbatim messages (the ones from the 47-minute gap) directly from the JSONL — stale tool-result records, repeated file-reads, accumulated output blobs. It doesn't know about precompute anchors; it just strips every category of bloat regardless of compaction history. On a 238k-token bloated session I'd expect 40–65% reduction from aggressive, which would get you under where a fresh precompute would have landed.On bug #2 (cache-create instead of cache-read): this one cozempic can't touch — it's about where CC places the
cache_controlbreakpoint in the outbound request, not about the JSONL content. The body's volatile tail keeps moving the anchor forward, preventing the background precompute from reusing the cache the main thread created. That's an architectural CC fix.If you want to verify the actual token count independently of what CC reports (useful for diagnosing whether the precompute anchor is where you expect it):
cozempic current --diagnosereads the JSONL and prints a category-by-category breakdown.I can reproduce a very similar cost pattern.
During a
/compact-related request, Claude Code created a large prompt cache segment that did not appear to be reused proportionally by later turns.Observed row from my usage trace:
| Field | Value |
| --- | --- |
| Model |
claude-opus-4-8|| Input |
2.0K|| Output |
6.5K|| Cache Read |
13.3K|| Cache Create |
182.2K|| Cost |
$1.3187|| TTFT |
4699ms|| Duration |
78556ms|| Timestamp |
2026-06-29 08:37:35|Subsequent requests only read much smaller cache ranges:
13.3K31.3K48.6K52.8K54.8K55.3K56.8KThis looks consistent with the issue described here: the compact/precompute path writes a large cache segment, but the following requests do not reuse that segment at the expected scale.
From the user perspective,
/compactis expected to reduce context pressure. In this trace, it instead became one of the most expensive requests in the session.Screenshot attached: the
182.2KCache Create row is highlighted.<img width="1882" height="534" alt="Image" src="https://github.com/user-attachments/assets/a95d1b15-1515-4039-a394-347b22b4b92f" />
@young1lin Your usage table is a clean trace of the same pattern. The 182.2K cache-create write after compact, followed by reads that top out at 56.8K, means the large precompute segment was written but the cache-read anchor for subsequent turns is set to a different breakpoint — the turns are reading from a point that predates the bulk of the precompute write. That incremental growth (13.3K → 31.3K → ... → 56.8K) is the session building fresh context from that anchor rather than reusing the 182K write.
@jens-f's workaround above () is the right call for Issue 1 — disables the stale-write path so compact runs live at the threshold instead. Your data helps confirm the bug isn't edge-case specific.
ngl that stale precompute kept ~200k tokens verbatim and then got cache created at the expensive write rate instead of cache read explains why ur costs spike. fyi wozcode cut my token spend ~50% with better caching, might help https://wozcode.com