Bug: Post-compact context remains at ~65% instead of expected ~20%

Description

After automatic compaction (/compact), context usage remains at approximately 65% (130k of 200k tokens) instead of the expected ~15-20%. This causes:

1. Sessions quickly become "suffocated" with little room for new work
2. Frequent cascade compactions
3. Loss of user productivity

Environment

• Claude Code Version: 2.1.29
• OS: Linux (WSL2 on Windows 11)
• Node: v20.x
• Installation: npm install -g @anthropic-ai/claude-code

Reproduction Steps

1. Start a Claude Code session with CLAUDE.md + imports (~20k tokens of Memory Files)
2. Work until context reaches compaction threshold (~87%)
3. Execute /compact (or let auto-compact trigger)
4. Run /context to check usage
5. Observe: Context shows ~65% usage instead of expected <30%

Expected Behavior

Post-compact context should be approximately:

summary (~10-15k) + memory_files (1x, ~20k) + hooks (~5k) = ~40k tokens (~20%)

Actual Behavior

Post-compact context is:

summary (20k) + memory_files (1x, 20k) + memory_files (DUPLICATED, 20k) + hooks (5k) = ~65k tokens (~65%)

Root Cause Analysis

After investigating the minified cli.js (v2.1.29), we identified 4 bugs in the compaction system:

Bug 1: Duplicate Attachments (CRITICAL)

Location: Function Ua() at byte ~8672727

// CURRENT (buggy):
function Ua(A){
  return[
    A.boundaryMarker,
    ...A.summaryMessages,
    ...A.messagesToKeep??[],
    ...A.attachments,      // ← NO DUPLICATE FILTERING!
    ...A.hookResults
  ]
}

Problem: Ua() includes attachments without checking for duplicates. Memory Files that were already in context are re-read by sNY() and included again.

Impact: +20k tokens (Memory Files duplicated)

Bug 2: Memory Files Re-read After Compact

Location: Function sNY() at byte ~8674000

Problem: sNY() always re-reads Memory Files from filesystem during compaction, even though they're already in context. Combined with Bug 1, this causes duplication.

Impact: Redundant I/O + duplicate tokens

Bug 3: Fixed Summary Size Limit

Location: Constant gd1=20000 at byte ~25584

Problem: Summary limit is fixed at 20k tokens regardless of available space. It doesn't consider:

• How much space Memory Files occupy
• How much space hooks occupy
• What the total post-compact budget should be

Impact: Summary can consume ALL available space, leaving no room for conversation

Bug 4: No Post-Compact Budget Validation

Location: Function KM1() at byte ~8672700

Problem: KM1() orchestrates compaction but never validates the total size of the result. It just returns whatever was assembled, even if it's 65k tokens when it should be 30k.

Impact: Unpredictable post-compact context size

Proposed Fix

Fix for Bug 1 (Minimal, Safe)

Filter attachments by unique path before including:

// PROPOSED FIX:
function Ua(A){
  let seen = new Set();
  let filtered = A.attachments.filter(a => {
    let key = a?.content?.[0]?.source?.path || JSON.stringify(a);
    if(seen.has(key)) return false;
    seen.add(key);
    return true;
  });
  return[
    A.boundaryMarker,
    ...A.summaryMessages,
    ...A.messagesToKeep??[],
    ...filtered,           // ← FILTERED attachments
    ...A.hookResults
  ]
}

Architectural Fix (Recommended)

Implement a Budget Controller that:

1. Calculates available space BEFORE generating summary
2. Dynamically adjusts gd1 based on Memory Files + hooks size
3. Validates total result size BEFORE returning
4. Truncates summary if over budget (summary has lowest priority)

Additional Context

Code Map (cli.js v2.1.29)

| Function | Byte Position | Responsibility |
|----------|---------------|----------------|
| KM1 | 8672700 | Orchestrates compaction |
| Ua | 8672727 | Assembles result messages |
| sNY | ~8674000 | Re-reads Memory Files |
| aNY | 8675890 | Generates summary (max gd1) |
| AVY | 8679676 | Reads agents/tasks |

Relevant Constants

| Constant | Value | Description |
|----------|-------|-------------|
| gd1 | 20000 | Max tokens for summary |
| EVY | 20000 | Token margin |
| zEA | 13000 | Compaction threshold |

Suggested Environment Variables

For users who want more control, consider adding:

• CLAUDE_SUMMARY_MAX_TOKENS - Override gd1
• CLAUDE_POST_COMPACT_BUDGET - Total budget after compact
• CLAUDE_SKIP_MEMORY_REREAD - Skip re-reading Memory Files

Workaround

Currently, users can set DISABLE_AUTO_COMPACT=true and manually manage context, but this is not ideal.

Impact

This bug affects every user with Memory Files (CLAUDE.md, imports, etc.). The larger the Memory Files, the worse the problem. For users with ~20k tokens of Memory Files, they lose ~35k tokens per session to duplication.

---

Submitted by: Raphael Mendes da Cunha Neto (PPGCC/UFJF) + Claude Opus 4.5
License: MIT (for any contributed code)