A lone UTF-16 surrogate (from truncated/persisted tool output) in the session transcript bricks the session with a repeating `400 ... no low surrogate in string` on every turn

Environment

• Claude Code: 2.1.170
• Model: claude-opus-4-8 (1M-context variant [1m])
• Platform: macOS (darwin)
• Context: long-running autonomous CLI session

Summary

When a tool output containing a non-BMP character (e.g. an emoji, encoded as a UTF-16 surrogate pair) is persisted/truncated mid-pair, the transcript ends up holding a lone surrogate — a high surrogate (\uD800–\uDBFF) with no following low surrogate (\uDC00–\uDFFF). From then on, every request body the harness assembles is invalid JSON, and the API rejects it:

API Error: 400 The request body is not valid JSON: no low surrogate in string: line 1 column 559887 (char 559886)

The error recurs on every subsequent turn (instant fail), because the lone surrogate is baked into the saved conversation and each request re-sends it. The session is bricked.

Severity: High (for autonomous / long-running agents)

There is no user error here — unlike the thinking-block brick, the model did nothing wrong. A harness-side truncation of a large tool output split a surrogate pair, and a single resulting half-character bricks the worker on every turn. In a fleet of autonomous agents, this silently takes a worker down until an operator intervenes.

Steps to reproduce

1. Run Claude Code 2.1.170 on claude-opus-4-8[1m].
2. Have a tool produce a large output that contains a non-BMP character (an emoji, e.g. 🧠 / 🤖 / 🦾, all in the U+1F900–U+1F9FF block → \uD83E\uDDxx). The output is large enough to be persisted and previewed/truncated (<persisted-output> … </persisted-output>).
3. The truncation boundary falls between the two halves of the surrogate pair, leaving a lone high surrogate (\uD83E) in the transcript text.
4. On the next request the assembled JSON body contains the lone surrogate → 400 ... no low surrogate in string.
5. Every subsequent turn re-hits the same 400 (the bad char is in saved context). Session bricked.

Expected

Truncation of tool output should be codepoint-aware (never cut between surrogate halves), and/or the request serializer should sanitize lone surrogates (replace with U+FFFD) before sending — so a half-emoji can never invalidate the request body.

Actual

Every turn fails instantly with the same 400. The corrupt half-character persists in history, so there is no self-recovery from inside the bricked session.

Evidence (real transcript)

Edge worker, session fae4b736-… (JSONL transcript), 2026-06-14:

• Transcript line 268, a user / tool_result message inside a </persisted-output> block, contained \uD83E (a lone high surrogate) with no following low surrogate — the trailing half of an emoji that was truncated at the persisted-output boundary.
• This mapped to char 559886 of the assembled request body → 400 ... no low surrogate in string: line 1 column 559887 (char 559886), repeated on every turn (instant "Cooked/Sautéed for 0s").

Root-cause hypothesis

The persisted-output preview/truncation indexes by byte or UTF-16 code unit, not by Unicode codepoint, so it can cut a surrogate pair in half. The surviving lone surrogate is then written into the transcript and re-serialized into every later request body, which is no longer valid JSON.

Recovery (notable: context-preserving — unlike the thinking-block brick)

Unlike the deferred-tool thinking-block brick (restart-only, all context lost — sibling report anthropic-bug-deferred-tool-thinking-block-brick-2026-05-29.md), this one is recoverable WITHOUT context loss by external transcript surgery:

1. Kill the live claude child process (it holds the corrupt context in RAM and is appending to the .jsonl).
2. In the transcript .jsonl, replace only the lone surrogate escape(s) with the U+FFFD escape (�) — same 6-char length, so byte offsets and JSON structure are preserved; never touch valid pairs (back up first).
3. claude --resume <session-id> → the cleaned transcript loads with full context intact.

This works, but it requires an operator with filesystem access; the harness should not require it.

Suggested fixes (any one mitigates)

1. Codepoint-aware truncation — never split a surrogate pair when persisting/previewing/truncating tool output.
2. Sanitize the request body — replace any lone surrogate with U+FFFD at serialization time, so a malformed transcript can never produce an invalid request body.
3. In-session recovery — when a request 400s on no low surrogate, repair the offending content client-side (replace lone surrogates) so a retry / /compact recovers, instead of bricking every turn.

Current workaround

• Recovery: the context-preserving kill → clean-transcript (\uXXXX lone-surrogate → �) → claude --resume recipe above. Internal recipe: memory reference_transcript_surrogate_brick_fix; failure-ledger class transcript-surrogate-brick.