[BUG] Session files grow to multi-GB due to normalizedMessages duplication in subagent progress entries

Found 3 possible duplicate issues:

1. https://github.com/anthropics/claude-code/issues/18905
2. https://github.com/anthropics/claude-code/issues/18682
3. https://github.com/anthropics/claude-code/issues/16470

This issue will be automatically closed as a duplicate in 3 days.

• If your issue is a duplicate, please close it and 👍 the existing issue instead
• To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

Additional data point from #19887:

| Metric | Value |
|--------|-------|
| Session file | 270 MB |
| Progress entries | 3,540 |
| Progress data | 250 MB (93% of file) |
| Subagents spawned | 23 |
| User-visible messages | 72 |

Trigger: ESC ESC → restore conversation (without code) during active session with compaction.

Symptoms:

• Terminal goes completely blank
• Process orphaned (persists after killing terminal)
• New terminal with claude -c also hangs
• High CPU, no error output

---

Workaround: Slim progress entries by replacing normalizedMessages with just the count:

import json
from pathlib import Path

def slim_progress(filepath):
    with open(filepath, 'r') as f:
        lines = f.readlines()
    output = []
    for line in lines:
        line = line.strip()
        if not line:
            continue
        obj = json.loads(line)
        if obj.get('type') == 'progress':
            data = obj.get('data', {})
            if 'normalizedMessages' in data:
                data['normalizedMessages_count'] = len(data['normalizedMessages'])
                del data['normalizedMessages']
        output.append(json.dumps(obj, separators=(',', ':')))
    with open(filepath, 'w') as f:
        f.write('\n'.join(output) + '\n')

# Usage: slim_progress(Path('~/.claude/projects/<project>/<session>.jsonl'))

Result: 257 MB → 17 MB, session resumes successfully. Metadata (agentId, prompt, timestamps) preserved.

Workaround Script

Here's an updated version of the workaround from the thread that works as both a manual script and a Claude Code hook.

Script

Save as ~/.claude/hooks/slim-session.py:

#!/usr/bin/env python3
"""
Slim Claude Code session files by removing normalizedMessages from progress entries.

Workaround for https://github.com/anthropics/claude-code/issues/19040

Usage:
1. Manual: slim-session.py <session-file.jsonl>
2. Stop hook: Add to settings.json or skill frontmatter
"""

import json
import sys
from pathlib import Path


def slim_progress(filepath: Path) -> tuple[int, int]:
    """Remove normalizedMessages from progress entries, keeping only the count."""
    original_size = filepath.stat().st_size

    with open(filepath, 'r') as f:
        lines = f.readlines()

    output = []
    for line in lines:
        line = line.strip()
        if not line:
            continue

        obj = json.loads(line)

        if obj.get('type') == 'progress':
            data = obj.get('data', {})
            if 'normalizedMessages' in data:
                data['normalizedMessages_count'] = len(data['normalizedMessages'])
                del data['normalizedMessages']

        output.append(json.dumps(obj, separators=(',', ':')))

    with open(filepath, 'w') as f:
        f.write('\n'.join(output) + '\n')

    return original_size, filepath.stat().st_size


def format_size(size_bytes: int) -> str:
    for unit in ['B', 'KB', 'MB', 'GB']:
        if size_bytes < 1024:
            return f"{size_bytes:.1f} {unit}"
        size_bytes /= 1024
    return f"{size_bytes:.1f} TB"


def run_as_hook() -> None:
    """Run as a Claude Code Stop hook."""
    try:
        data = json.load(sys.stdin)
    except json.JSONDecodeError:
        print(__doc__)
        sys.exit(1)

    transcript_path = data.get('transcript_path')
    if not transcript_path:
        sys.exit(0)

    filepath = Path(transcript_path)
    if not filepath.exists():
        sys.exit(0)

    # Only slim if file is over 10MB (adjust threshold as needed)
    if filepath.stat().st_size < 10 * 1024 * 1024:
        sys.exit(0)

    original, new = slim_progress(filepath)
    print(f"📦 Slimmed session: {format_size(original)} → {format_size(new)}", file=sys.stderr)
    sys.exit(0)


def run_as_script() -> None:
    """Run as manual script."""
    if len(sys.argv) != 2:
        print(__doc__)
        sys.exit(1)

    filepath = Path(sys.argv[1]).expanduser()
    if not filepath.exists():
        print(f"Error: File not found: {filepath}")
        sys.exit(1)

    print(f"Processing: {filepath}")
    original, new = slim_progress(filepath)
    saved = original - new
    percent = (saved / original) * 100 if original > 0 else 0
    print(f"Original:  {format_size(original)}")
    print(f"Slimmed:   {format_size(new)}")
    print(f"Saved:     {format_size(saved)} ({percent:.1f}%)")


if __name__ == '__main__':
    if len(sys.argv) > 1:
        run_as_script()
    elif not sys.stdin.isatty():
        run_as_hook()
    else:
        print(__doc__)
        sys.exit(1)

Usage

Manual cleanup:

chmod +x ~/.claude/hooks/slim-session.py
~/.claude/hooks/slim-session.py ~/.claude/projects/<project>/<session-id>.jsonl

As a global Stop hook (runs after every conversation turn):

Add to ~/.claude/settings.json:

{
  "hooks": {
    "Stop": [
      {
        "matcher": "",
        "hooks": [
          {
            "type": "command",
            "command": "python3 $HOME/.claude/hooks/slim-session.py"
          }
        ]
      }
    ]
  }
}

As a skill-specific hook (add to skill frontmatter):

hooks:
  Stop:
    - matcher: "*"
      hooks:
        - type: command
          command: "python3 $HOME/.claude/hooks/slim-session.py"

Results

Tested on a 343MB session file bloated by parallel subagent skills:

• Original: 343.1 MB
• Slimmed: 4.8 MB
• Saved: 338.2 MB (98.6%)

Session still loads and resumes correctly after slimming.

Conversation JSONL files in .claude/conversations/ grow without any limit. In my case, 79 conversation files totaled 2.5GB, with a single file reaching 1.1GB.

Root causes:

• Conversation files are append-only logs that store every tool call and its full response
• File reads are stored in full every time — no deduplication when the same file is read multiple times across turns
• When context summarization triggers, the LLM's working memory is compressed, but the on-disk transcript keeps every byte
• Command output (git diff, git status, build logs, etc.) is stored verbatim with no truncation in the transcript
• There is no maximum file size, no per-session cap, and no automatic cleanup of old sessions

Impact:

• Disk space consumed silently — users may not notice until they run out of space
• Large projects with long sessions are especially affected
• No warning is shown when conversation files grow large

Expected behavior:

• A configurable size cap per conversation file (e.g. stop appending raw output after X MB)
• Automatic pruning/cleanup of old conversation files (e.g. age-based or total size budget)
• Deduplication of repeated file reads within the same session
• Truncation of large tool outputs in the stored transcript (the LLM already works with truncated output anyway)
• At minimum, a warning when conversation storage exceeds a threshold

Environment:

• OS: Linux (Ubuntu)
• Claude Code: latest

This is one of the biggest contributors to session bloat — normalizedMessages duplication in subagent progress entries compounds fast.

I built Cozempic to deal with this. The progress-collapse strategy specifically targets consecutive progress ticks (which account for 40-48% of file size), and document-dedup catches the repeated normalizedMessages blocks:

pip install cozempic
cozempic current --diagnose    # Shows exact breakdown of what's bloating
cozempic treat current -rx standard --execute  # Apply with auto-backup

@Isoceth's workaround script is great for manual cleanup — Cozempic does something similar but with 13 composable strategies and automatic backups. Typical savings are 50-65% on subagent-heavy sessions.

Real-world impact: normalizedMessages bloat drove a 271 GB task output file

Adding a data point from v2.1.70 (macOS 15.7.5, M3 Max).

This issue's root cause — normalizedMessages duplication in progress entries — directly contributed to a 271 GB single .output file in /private/tmp/claude-501/. Details in my comment on #26911.

The chain reaction

1. Session had heavy subagent usage (multiple Agent tool calls with hooks)
2. Each subagent fires ~15 PostToolUse hooks per tool call
3. Each hook generates a progress entry in JSONL
4. Progress entries include full normalizedMessages (per this issue)
5. A background task concatenated all subagent outputs into one .output file
6. Result: unbounded growth at ~2 GB/min → 271 GB in ~2 hours → disk full

The normalizedMessages duplication described here is the force multiplier that turns a moderate-sized task output into a catastrophic disk-filler. Without the duplicated history in every progress entry, the same session would likely have produced single-digit GB of output.

Supporting the proposed fix

Strongly agree with the proposal to not include normalizedMessages in progress events. Reconstructing from subagent files on demand is the right pattern. Even just storing a normalizedMessages_count field (as shown in the workaround script) would reduce file sizes by 93%+.

This is not a theoretical concern — it nearly bricked a 1 TB MacBook Pro.

We've hit this too. Heavy subagent user — our workflow regularly spawns 3-6 parallel Agent tool calls per investigation cycle across ~200 sessions on the same project. Session files on Windows grow noticeably over long sessions.

Our mitigation has been architectural rather than patching the symptom:

1. Aggressive compaction awareness — we have a PreCompact hook that snapshots active work state to a separate JSON file before compaction. This means we don't fight compaction (which helps control session file growth) but instead embrace it with state preservation.

1. Subagent discipline — keeping subagent prompts focused and short-lived. The worst bloat comes from long-running subagents that accumulate large conversation histories with many progress updates. We've moved toward spawning more agents with smaller scopes rather than fewer agents with larger scopes.

1. Session rotation — for 8+ hour work sessions, we rotate to a fresh session rather than extending indefinitely. The PreCompact snapshot + MEMORY.md files make this nearly seamless.

But these are all workarounds. The OP's analysis is spot-on — storing full normalizedMessages on every progress entry is the root cause. Delta-only progress entries would fix this at the source. The exponential growth pattern (each progress entry contains all previous messages plus new ones) means a single long-running subagent can generate more data than the entire rest of the session.

A Stop hook can clean up oversized session files:

CLAUDE_DIR="$HOME/.claude"
find "$CLAUDE_DIR" -name "*.json" -size +100M -exec ls -lh {} \; 2>/dev/null | while read line; do
    FILE=$(echo "$line" | awk '{print $NF}')
    SIZE=$(echo "$line" | awk '{print $5}')
    echo "Large session file: $FILE ($SIZE)" >&2
done
find "$CLAUDE_DIR" -name "*.json" -size +50M -mtime +7 -delete 2>/dev/null
exit 0

CLAUDE_DIR="$HOME/.claude"
find "$CLAUDE_DIR" -name "*.json" -mtime +30 -delete 2>/dev/null
find "$CLAUDE_DIR" -name "*.log" -mtime +14 -delete 2>/dev/null
SIZE=$(du -sh "$CLAUDE_DIR" 2>/dev/null | awk '{print $1}')
echo "~/.claude size after cleanup: $SIZE"

{
  "hooks": {
    "Stop": [{"hooks": [{"type": "command", "command": "bash ~/.claude/hooks/session-cleanup.sh"}]}]
  }
}

The Stop hook removes oversized old sessions. Weekly cron prevents long-term disk bloat.

Multi-GB session files from normalizedMessages duplication in subagent progress is exactly what progress-collapse in Cozempic v1.4.1 targets. It removes all progress tick messages, which include these duplicated normalized entries. Measured 92% savings on a session with 2,100+ progress ticks.

pip install cozempic && cozempic init

The guard daemon runs this automatically. Also file-history-dedup removes duplicate file-history-snapshot entries that compound the problem.

This was fixed in v2.1.16 — Subagent progress entries no longer duplicate the full message history into the session file, preventing multi-GB transcript growth. If you're still seeing this in the latest version, please comment with your version and repro and we'll reopen.

This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.