Background/incremental compaction — summarize out-of-band and splice into live context without blocking or resuming
Problem
Today, context compaction (auto-compact and /compact) is inline and blocking: when context fills, the main-thread model stops and summarizes the whole history before the next turn can proceed. For long sessions on a large/expensive model this is a noticeable stall, and the summarization decode (generated serially) is the dominant wall-clock cost. There is no way to overlap it with ongoing work or run it on a cheaper model.
Observations about what already exists
- The auto-compact window is configurable (
/autocompact,--autocompact,autoCompactWindow,CLAUDE_CODE_AUTO_COMPACT_WINDOW), so the trigger point can already be moved earlier (e.g. ~80%). - A background summarization job on the small/fast (Haiku-class,
ANTHROPIC_DEFAULT_HAIKU_MODEL) model already pre-computes conversation summaries forclaude --resume. PreCompactfires before compaction but can only observe/block; its stdout is not injected as context and it cannot supply a summary.
So the pieces exist (early trigger + cheap background summarizer + writable transcript). What's missing is the seam to apply the result to the live session.
Proposed behavior
Eager, incremental, non-blocking compaction:
- At a configurable fill threshold (e.g. 80%), snapshot the stable prefix of the transcript.
- Summarize that prefix in the background, on a configurable (cheaper/faster) model, while the main thread keeps working — there's headroom because it fired early.
- When the summary returns, splice it in: replace the summarized prefix, keep the recent N turns verbatim on top. No resume, no full-turn stall.
Because compaction already keeps recent turns verbatim, prefix-only summarization avoids staleness from turns added during the background job.
Why it can't be built with current primitives
There is no agent/hook/SDK primitive to mutate/evict the live in-memory context; appending only grows it, and transcript edits on disk only take effect at --resume. This requires a harness-level capability.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗