External compaction tools: JSON whitespace in compact_boundary entry silently breaks K48 scanner on resume
Summary
When an external tool writes a compact_boundary entry to a session JSONL file using a language whose JSON serializer defaults to spaced output (e.g., Python's json.dumps()), the K48 streaming scanner silently fails to find the boundary on resume. The session loads the entire file instead of skipping pre-boundary content, negating the compaction entirely.
Repro steps
- Write an external tool (Python, Go, etc.) that appends a
compact_boundaryentry to a session JSONL file - Use the language's default JSON serializer (e.g., Python
json.dumps(entry)without explicitseparators) - The entry is written as:
{"type": "system", "subtype": "compact_boundary", ...}(note space after every colon) - Resume the session with
claude --resume <session-id> - The session file must be >5MB (required to trigger K48 streaming scan; files <=5MB are fully parsed and work fine)
Expected behavior
K48 finds the compact_boundary entry, discards pre-boundary content, and loads only the post-boundary messages. Resume is fast and context-efficient.
Actual behavior
K48 never finds the boundary. The loader falls through to reading the entire file. All pre-compaction content is loaded, negating the compaction. No error or warning is emitted.
Root cause
K48 uses a byte-level prefix check before searching for "compact_boundary":
FJK = Buffer.from('{"type":"system"') // fast prefix — no space after colon
Python's json.dumps() defaults to separators=(', ', ': '), producing {"type": "system" (space after colon). At byte position 8, the scanner sees 0x20 (space) instead of 0x73 (s). The line is skipped without ever checking for the "compact_boundary" needle.
The same issue affects the attribution-snapshot prefix check.
Suggested fix (Claude Code side)
Add whitespace tolerance to the prefix check:
const FJK = Buffer.from('{"type":"system"');
const FJK_SPACED = Buffer.from('{"type": "system"');
// In line-check: line.startsWith(FJK) || line.startsWith(FJK_SPACED)
Or normalize by stripping spaces before prefix check. Or skip the prefix check and rely solely on the "compact_boundary" needle search (already requires JSON parse for verification).
Workaround (tool side)
json.dumps(entry, separators=(',', ':')) # compact separators, no spaces
Impact
- Silent data integrity issue — compaction appears to succeed but is bypassed on resume
- Affects any external tool writing
compact_boundaryentries in Python, Go, Ruby, or any language with spaced JSON defaults - Does NOT affect Claude Code's own
/compact(Node.jsJSON.stringifydefaults to compact output) - Users debugging "why is my compacted session still using full context?" get no indication whitespace is the cause
Discovered during development of rocketlabs-ai/infinite-context.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗