External compaction tools: JSON whitespace in compact_boundary entry silently breaks K48 scanner on resume

Resolved 💬 3 comments Opened Mar 28, 2026 by Emanz1 Closed May 4, 2026

Summary

When an external tool writes a compact_boundary entry to a session JSONL file using a language whose JSON serializer defaults to spaced output (e.g., Python's json.dumps()), the K48 streaming scanner silently fails to find the boundary on resume. The session loads the entire file instead of skipping pre-boundary content, negating the compaction entirely.

Repro steps

  1. Write an external tool (Python, Go, etc.) that appends a compact_boundary entry to a session JSONL file
  2. Use the language's default JSON serializer (e.g., Python json.dumps(entry) without explicit separators)
  3. The entry is written as: {"type": "system", "subtype": "compact_boundary", ...} (note space after every colon)
  4. Resume the session with claude --resume <session-id>
  5. The session file must be >5MB (required to trigger K48 streaming scan; files <=5MB are fully parsed and work fine)

Expected behavior

K48 finds the compact_boundary entry, discards pre-boundary content, and loads only the post-boundary messages. Resume is fast and context-efficient.

Actual behavior

K48 never finds the boundary. The loader falls through to reading the entire file. All pre-compaction content is loaded, negating the compaction. No error or warning is emitted.

Root cause

K48 uses a byte-level prefix check before searching for "compact_boundary":

FJK = Buffer.from('{"type":"system"')  // fast prefix — no space after colon

Python's json.dumps() defaults to separators=(', ', ': '), producing {"type": "system" (space after colon). At byte position 8, the scanner sees 0x20 (space) instead of 0x73 (s). The line is skipped without ever checking for the "compact_boundary" needle.

The same issue affects the attribution-snapshot prefix check.

Suggested fix (Claude Code side)

Add whitespace tolerance to the prefix check:

const FJK = Buffer.from('{"type":"system"');
const FJK_SPACED = Buffer.from('{"type": "system"');
// In line-check: line.startsWith(FJK) || line.startsWith(FJK_SPACED)

Or normalize by stripping spaces before prefix check. Or skip the prefix check and rely solely on the "compact_boundary" needle search (already requires JSON parse for verification).

Workaround (tool side)

json.dumps(entry, separators=(',', ':'))  # compact separators, no spaces

Impact

  • Silent data integrity issue — compaction appears to succeed but is bypassed on resume
  • Affects any external tool writing compact_boundary entries in Python, Go, Ruby, or any language with spaced JSON defaults
  • Does NOT affect Claude Code's own /compact (Node.js JSON.stringify defaults to compact output)
  • Users debugging "why is my compacted session still using full context?" get no indication whitespace is the cause

Discovered during development of rocketlabs-ai/infinite-context.

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗