[FEATURE] Ephemeral File Context to Reduce Context Bloat by 30-40%

[FEATURE] Ephemeral File Context to Reduce Context Bloat by 30-40%

Problem

Claude Code sessions hit the 200k token limit rapidly, with file reads accounting for ~40% of context consumption (80k+ tokens). Files are read repeatedly throughout a session, creating massive duplication in the conversation history:

Turn 1: Read app.py (2,000 tokens) → injected into conversation
Turn 5: Read app.py again (2,000 tokens) → DUPLICATE
Turn 10: Read app.py again (2,000 tokens) → DUPLICATE
Turn 15: Read app.py again (2,000 tokens) → DUPLICATE

Total: 8,000 tokens for one file

Typical 200k session breakdown:

• 80k tokens: Repeated file reads (same files, multiple times) ← Target for optimization
• 40k tokens: Tool outputs (grep, bash, git)
• 30k tokens: System prompts
• 30k tokens: Assistant responses
• 20k tokens: User messages

This is the single biggest source of context bloat in coding sessions.

Relationship to Existing Context Management Features

Anthropic recently released two context management features for the API (September 2025):

1. Memory Tool (API Only)

• Purpose: Persistent knowledge storage across sessions
• How it works: External /memories directory outside context window
• Use case: Learning user preferences, project context over time
• Status: ✅ API only, ❌ Not available in Claude Code

2. Context Editing (API Only)

• Purpose: Automatically prune old tool results when approaching limits
• How it works: Removes stale tool results, keeps conversation flow
• Performance: 84% token reduction in 100-turn workflows
• Status: ✅ API only, ❌ Not available in Claude Code

What Claude Code Has Instead

Claude Code currently has:

• CLAUDE.md files: Static context loaded at session start (takes up context space)
• Auto-compact: Summarizes entire conversation at 95% capacity (loses detail)
• Session resume: Reload previous session state

Why This Proposal is Critical for Claude Code

The API has advanced context management, Claude Code doesn't.

| Feature | API | Claude Code |
|---------|-----|-------------|
| Memory Tool (persistent storage) | ✅ | ❌ |
| Context Editing (prune tool results) | ✅ | ❌ |
| Ephemeral File Context (deduplicate reads) | ❌ | ❌ ← This proposal |

Without this proposal:

Claude Code sessions:
→ Files read repeatedly (no deduplication)
→ Context fills with duplicates
→ Hits 95% → Auto-compact (loses detail)
→ Shorter sessions, frequent restarts

With this proposal:

Claude Code sessions:
→ Files cached (no duplication)
→ 30-40% context savings
→ 1.5-2x longer sessions before auto-compact
→ Better session continuity

How This Complements API Features

For API users, this proposal works alongside Memory Tool and Context Editing:

[System Prompt]
[Memory Tool - Persistent]      ← API feature (cross-session learning)
[Ephemeral File Context]        ← This proposal (deduplicate reads)
[Conversation Thread]           ← Context Editing (prune old results)

Combined benefits:

• Memory Tool: Learn preferences across sessions
• Context Editing: Remove stale tool results (84% savings)
• Ephemeral File Context: Deduplicate file reads (30-40% savings)
• Total impact: Much longer, more efficient agentic workflows

Key distinction: Memory Tool is for knowledge retention, Context Editing is for pruning stale content, this proposal is for preventing duplication upstream.

Proposed Solution

Implement Ephemeral File Context - a session-level cache that:

1. Stores file contents separately from conversation history
2. Updates files in-place rather than appending duplicates
3. Auto-detects external changes (git operations, formatters, etc.)
4. Leverages Anthropic prompt caching for 90% cost reduction

Architecture

Current flow:

[System Prompt]
[Conversation Thread]:
  - Tool Result: <2000 tokens of app.py>  ← Goes into main thread
  - Tool Result: <2000 tokens of app.py>  ← DUPLICATE
  - Tool Result: <2000 tokens of app.py>  ← DUPLICATE

Proposed flow:

[System Prompt]

[Ephemeral File Context - CACHED]:
  app.py: <2000 tokens, current state only>
  utils.py: <1500 tokens, current state only>

[Conversation Thread]:
  - Tool Reference: Read(app.py) → cached: true
  - Tool Reference: Read(app.py) → cached: true

When constructing API calls:

1. System prompt
2. Ephemeral file context with current file states only (marked for caching)
3. Conversation thread (tool calls without full results)

Core Implementation

File tool modifications:

// Read Tool
function handleRead(filePath: string) {
  const content = fs.readFileSync(filePath, 'utf-8');
  const hash = md5(content);

  // Update cache
  fileCache.set(filePath, { content, hash, lastRead: Date.now() });

  // Return reference only (don't inject content into conversation)
  return { ref: filePath, cached: true, hash };
}

// Write/Edit Tools
function handleWrite(filePath: string, newContent: string) {
  fs.writeFileSync(filePath, newContent);

  // Update cache IN-PLACE (not appended)
  fileCache.set(filePath, { content: newContent, hash: md5(newContent) });

  return { ref: filePath, updated: true };
}

Prompt construction:

function buildPrompt() {
  validateFileCache(); // Check for external changes

  return [
    systemPrompt,

    // Ephemeral file context (cacheable via Anthropic prompt caching)
    {
      type: "text",
      text: buildFileContext(fileCache),
      cache_control: { type: "ephemeral" }
    },

    // Conversation thread (no file contents)
    ...conversationHistory
  ];
}

Auto-update on external changes (check-on-use):

function validateFileCache() {
  for (const [path, cached] of fileCache) {
    if (!fs.existsSync(path)) {
      fileCache.delete(path); // File deleted
      continue;
    }

    const currentContent = fs.readFileSync(path, 'utf-8');
    const currentHash = md5(currentContent);

    if (currentHash !== cached.hash) {
      // File changed externally (git, formatter, IDE), refresh cache
      fileCache.set(path, { content: currentContent, hash: currentHash });
    }
  }
}

Benefits & ROI

Context Reduction

Current: 200k tokens per session
With ephemeral context: ~130k tokens per session

Result: 30-40% total context reduction

Session Length

• Current: Hit 200k limit after ~30-50 turns
• With ephemeral context: Hit 200k limit after ~50-80 turns
• Effective increase: 1.5-2x longer sessions

Cost Reduction

With Anthropic prompt caching:

• Cached input tokens: 10x cheaper than regular input tokens
• File context marked as cacheable
• After first API call, file reads are 90% cheaper

Estimated cost per call (assuming 40% of input is file content):

Current: 100k input × $3/1M = $0.30

With caching:
- 40k cached file tokens × $0.30/1M = $0.012
- 60k regular tokens × $3/1M = $0.18
Total: $0.192 (36% reduction)

Latency Reduction

Anthropic reports 85% latency reduction for cached content.
With 40% of context cached: ~30-40% latency improvement.

Implementation Effort

Phase 1: Core Implementation (1-2 weeks)

• File cache data structure: 1 day
• Modify Read/Write/Edit tools: 2 days
• Prompt construction refactor: 2 days
• Check-on-use validation: 1 day
• Prompt caching integration: 1 day
• Testing & edge cases: 2-3 days

Phase 2: Enhancements (Optional, 3-5 days)

• File watching for real-time feedback: 2 days
• Cache size limits and LRU eviction: 1 day
• Cache analytics/debugging: 1 day

Total: 2-3 weeks for full implementation

Use Cases

1. User Runs Formatter

- Claude edits app.py
- User runs: prettier --write app.py
- Claude references app.py again
- Cache auto-refreshes, sees formatted version ✓

2. Git Operations

- Claude reviews files on feature branch
- User runs: git checkout main
- Claude references files
- Cache auto-refreshes, sees main branch ✓

3. Hot Reload / Build Tools

- Claude edits component.tsx
- Next.js rebuild adds generated code
- Claude references component
- Cache auto-refreshes, sees generated additions ✓

Edge Cases Handled

• Large files: Max file size limit (e.g., 100KB), fall back to current behavior
• Binary files: Only cache text files
• Deleted files: Remove from cache during validation
• Cache eviction: LRU policy with configurable max (e.g., 50 files or 5MB total)

Configuration

interface EphemeralCacheConfig {
  enabled: boolean;              // Default: true
  maxFiles: number;              // Default: 50
  maxTotalSize: number;          // Default: 5MB
  maxFileSize: number;           // Default: 1MB
  cacheValidation: 'always' | 'on-change';  // Default: 'always'
}

Related Issues & Features

Existing issues:

• #16958 - Polling background tasks causes repeated file reads
• #19583 - Tools not being explicitly cached
• #11532 - File content ephemeral vs persistent storage
• #21693 - File modification reminders consume excessive context

Related Anthropic features:

• Memory Tool (API) - Persistent cross-session storage (September 2025)
• Context Management (API) - Context Editing for pruning tool results (September 2025)
• Claude Code Memory - Static CLAUDE.md files and auto-compact

Detailed Specification

For a comprehensive technical specification including code examples, testing strategy, and migration path, see: Ephemeral File Context Proposal

---

Summary

This is the 80/20 optimization for Claude Code context efficiency:

• High impact: 30-40% context reduction, 1.5-2x longer sessions
• Medium effort: 2-3 weeks implementation
• Addresses: The single biggest source of context bloat

Why This Matters

For Claude Code users: This is critical because Claude Code lacks the API's Memory Tool and Context Editing features. Ephemeral file context is the primary way to reduce context bloat and extend session length.

For API users: This complements existing features by preventing duplication upstream, while Memory Tool handles cross-session knowledge and Context Editing handles downstream pruning.

The Upstream Problem

Memory Tool and Context Editing solve important problems, but they don't prevent files from being read multiple times into the conversation history. This proposal addresses the root cause:

• Before: Read app.py 10x → 20k tokens in conversation → Context Editing removes 8x → 4k tokens remain
• After: Read app.py 10x → 2k tokens total (cached) → Context Editing not needed for files → 90% reduction

File operations dominate context consumption in coding sessions. This proposal directly targets that inefficiency with a clean architectural solution that leverages existing Anthropic prompt caching capabilities and prevents duplication at the source.