[FEATURE] Background pre-computation of context compaction with differential top-up

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

Context compaction currently runs synchronously when the context window is nearly full, causing a noticeable pause that disrupts workflow. The entire conversation history must be summarized in one blocking operation. In long sessions, this pause can be quite significant.

Related: #26026 describes the same pain point ("stop-the-world" compaction) but proposes incremental compaction in general terms. This issue proposes a specific, complementary mechanism.

Proposed Solution

Background pre-computation with differential top-up:

1. Start summarizing early, in the background. When context usage crosses a threshold (e.g. 60-70%), begin generating a summary of the conversation so far as a background task — while the user continues working normally.

1. When compaction is actually needed, only the delta (messages added since the background summary started) needs to be processed. This small differential summary is merged with the pre-computed one.

1. Result: The blocking pause is reduced from "summarize the entire conversation" to "summarize the few recent messages and merge." The wall-clock stall drops dramatically.

This is analogous to how modern garbage collectors (G1, ZGC) do concurrent marking in the background and only pause briefly for a final reconciliation — the same insight noted in #26026, but here as a concrete implementation approach.

Key insight

Even if the background summary becomes somewhat stale (because the conversation extended or changed direction), a stale partial summary + small top-up is far faster than summarizing everything from scratch.

Alternative Solutions

• Incremental compaction during idle time (#26026) — complementary but different; that approach compacts older turns during gaps, whereas this pre-computes a full summary in the background.
• User-triggered /compact — helps with timing but doesn't reduce the pause duration itself.
• Larger context windows — delays the problem but doesn't eliminate it.

Priority

Medium - Would be very helpful

Feature Category

Performance and speed

Use Case Example

1. Developer is in a 2-hour session iterating on a feature
2. At ~70% context usage, background summarization begins (invisible to user)
3. User continues working — editing files, running tests, reviewing output
4. At ~95% context usage, compaction triggers
5. Instead of summarizing 2 hours of conversation from scratch (10-20s pause), the system merges the pre-computed summary with the last ~10 minutes of new messages (1-2s pause)
6. Developer barely notices and stays in flow

Additional Context

This approach has a nice property: the background summary can be progressively refined. If context usage grows slowly, the system could re-run the background summary periodically, keeping the delta small. The marginal cost of each background pass is just API compute during otherwise-idle time.