[FEATURE] Branch-level semantic eviction — collapse dead reasoning branches instead of whole-context compaction

Status Open
Maintainer reply None cached
Activity 1 comment · opened Jul 23, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

Auto-compact today is all-or-nothing. When the context window fills, the whole session gets summarized in a single pass — and useful reasoning gets flattened together with dead ends.

Microcompact and API context-editing improve on this, but they evict by the age or type of tool results, not by whether a line of reasoning actually went anywhere. The result:

  • A long exploratory detour that led nowhere stays in context at full weight until it simply ages out.
  • A critical decision from a few turns back can get compressed just because it's old.

In a real multi-step session (a large refactor, a data ingest), half the context often ends up spent on one branch that was ruled out early — crowding out the branch the final answer actually came from.

Proposed Solution

Make the reasoning branch — a sub-thread of exploration in the conversation — the unit of memory management, rather than the individual token or tool result.

When a branch reaches a dead end or is abandoned, collapse it to a one-line stub (explored X, ruled out because Y). This is a defocus, not a delete.
Keep branches that led to the current working state at full fidelity.
Make collapsed branches re-expandable — a cheap pointer, hydrated on demand if the model later needs to revisit that path.
Intuition: it's how a person closes a browser tab for a path that didn't work while keeping the ones that did — attention as active pruning, not passive decay by age.

Alternative Solutions

I've tried manually running /compact at natural breakpoints, but it summarizes everything — including the branch I want to keep sharp. I've also tried splitting work across separate sessions so a dead-end exploration doesn't pollute the main context, but that loses the shared state and forces me to re-explain the task.

Current workaround: I proactively /clear and re-paste only the relevant files once a session gets bloated — manual, lossy, and it throws away reasoning I might still need.

Related open requests reaching for the same primitive (deciding what to keep by relevance-to-outcome, reversibly): #6390 (context pruning as an alternative to compaction), #33026 (let Claude self-initiate compaction), #7708 (mark important context so it isn't compacted), #21132 (let Claude clear its own context).

Priority

High - Significant impact on productivity

Feature Category

Performance and speed

Use Case Example

Example scenario:

  1. I'm doing a large refactor across a React app and its Node.js backend in one session.
  2. Early on, I explore approach A (e.g. moving logic into a shared package). After ~15 tool calls it turns out to be a dead end — a circular dependency makes it unworkable.
  3. I switch to approach B, which works, and spend the rest of the session building it out.
  4. The context fills. Auto-compact fires and summarizes the WHOLE session — flattening approach B's live reasoning together with the abandoned approach A.
  5. With this feature: the moment I move on from approach A, its whole branch collapses to a stub ("tried shared-package extraction, ruled out — circular dep"). Approach B stays at full fidelity. Freed budget goes to the work that's actually alive — and if a later problem makes approach A relevant again, its stub can be expanded back on demand.

This would save time because the model stops paying full context cost for a path I already rejected, and I stop losing my good reasoning to a blind full-session summary.

Additional Context

Prior art — this principle is already validated in research, one level down the stack:

  • ArborKV (2026, https://arxiv.org/html/2605.22106) — tree-aware KV-cache eviction with reversible exploration and "lazy rehydration" (restore an evicted branch on backtrack). But it operates on the raw KV cache inside a single generation and requires an explicit tree-search engine (Tree-of-Thoughts / MCTS) to signal every branch and backtrack — it does not detect dead branches on its own.
  • "Not All Thoughts Need HBM" (https://arxiv.org/html/2605.09490) and "Beyond Compaction: Structured Context Eviction for Long-Horizon Agents" (https://arxiv.org/html/2606.11213) — semantics-aware eviction of reasoning rather than age-based.
  • Also related: H2O / Heavy-Hitter eviction, MemGPT-style hierarchical memory.

What's new here: ArborKV proves branch-level reversible eviction works, but requires an explicit search tree and operates on the KV cache. This proposal brings the same principle UP to linear agent sessions, with automatic dead-branch detection at the message/tool-result level — no external search tree required. The eviction signal is the outcome of a conversation branch, not token-level attention scores.

One important empirical caveat from that research: permanently deleting low-importance tokens is catastrophic for reasoning (accuracy can collapse when half the cache is dropped). That's exactly why the reversible "defocus, not delete" design matters — collapsed branches must stay re-expandable, not be discarded.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗