Enhancement: Hybrid Memory Architecture to Replace Full Conversation History in Context Window

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

Currently, Claude Code sends the entire conversation history with every prompt. This means context grows linearly with session length, leading to:

Rapidly increasing token usage and cost per query
Quality degradation as the context window fills and auto-compaction kicks in
Compaction summarizing summaries in long sessions, causing context drift
Practical limits on long-running, multi-hour coding sessions

Proposed Solution

Replace (or augment) the full conversation history feed with a hybrid memory architecture:

1. Rolling Structured Summary

Maintain a continuously updated, structured summary of the session containing:

Key decisions made (e.g., "chose PostgreSQL over SQLite in turn 3")
Architecture and design choices
Files created/modified and why
Open questions or unresolved issues

This summary gets updated each turn and always feeds into the context — keeping it deterministic and causally ordered.

1. RAG on File Contents / Code — Not Conversation

Use retrieval-augmented generation selectively for file contents and codebase, not the conversation history itself. Code files are large, discretely retrievable, and well-suited for similarity search.

1. Recency Window

Always include the last N turns verbatim (e.g., last 5–10 exchanges) to preserve immediate conversational coherence, while older history is covered by the rolling summary.
Why Not Pure RAG on Conversation History?

Retrieval errors are silent — if the retriever misses a critical earlier decision, Claude proceeds without it, potentially causing subtle bugs or inconsistencies
Code context is deeply interconnected — a decision in turn 3 may be critically relevant in turn 47; semantic similarity retrieval may not surface it reliably
Causality matters — later corrections must supersede earlier ones; similarity-based retrieval may surface both equally, causing confusion

Expected Benefits

Context window size stays roughly constant regardless of session length
Significant cost reduction for long sessions
More reliable, consistent responses throughout a session
Enables multi-hour / multi-day coding sessions without quality degradation

Alternative Solutions

_No response_

Priority

Medium - Would be very helpful

Feature Category

CLI commands and flags

Use Case Example

_No response_

Additional Context

_No response_