Lone UTF-16 surrogate in tool input permanently bricks session (HTTP 400 'no low surrogate')

Open 💬 4 comments Opened Jun 14, 2026 by duwnsskan-alt

Summary

A lone UTF-16 surrogate codepoint emitted by the model inside a tool-call input field permanently disabled a long-running Claude Code session. Every subsequent API turn was rejected with HTTP 400 - The request body is not valid JSON: no low surrogate in string, with the error pointing at the same byte offset on every retry. The session log grew to ~52 MB / 6,545 records and could not be recovered.

Environment

  • Claude Code CLI on macOS (Darwin 25.5.0), zsh
  • Korean-language session. BMP Hangul (U+AC00–U+D7A3) lives adjacent to the surrogate block (U+D800–U+DFFF), which appears to make this failure mode more likely.

Reproduction (observed)

  1. During a long session, the model called AskUserQuestion with several Korean options.
  2. One option's description contained \ud99c — a lone high surrogate with no paired low surrogate. Surrounding context was "현\ud99c 보유현황..."; intended text was 현재 (U+C7AC).
  3. The tool call was accepted locally and stored in conversation history.
  4. ~33 minutes later, once cumulative request body crossed ~2.2 MB, the Messages API began rejecting every turn with:

``
API Error: 400 The request body is not valid JSON: no low surrogate in string:
line 1 column 2226149 (char 2226148)
``

  1. All 13 subsequent retries failed at the identical offset (char 2226148), confirming the corruption was in conversation history rather than new input. The session became unusable.

Impact

  • The session is irrecoverable from within Claude Code. /clear discards the full context; retrying re-sends the same poisoned history.
  • Substantial in-progress work was lost when the session had to be abandoned.
  • The same byte offset / same error class was hit in a separate earlier session by the same user (~6 hours prior), indicating this is reproducible rather than a one-off.

Root cause (suspected)

A multi-byte Korean codepoint was corrupted into a lone high surrogate somewhere in the model output or tool-call serialization path. Per RFC 8259 §8.2, JSON cannot contain unpaired surrogates, so once one lands in assistant.content[*].input (tool_use), every subsequent request is rejected.

Suggested fixes

  1. Client-side guard (highest leverage): Claude Code should validate tool-call input strings before persisting them to the session transcript. If an unpaired surrogate is detected, either re-prompt the model for that turn or replace the codepoint with U+FFFD and log a warning. This prevents the irrecoverable state.
  2. Server-side recovery path: Expose the offending byte offset in a structured form and allow clients to repair the specific record in-place, rather than requiring the whole session be discarded.
  3. Model-side: Investigate why unpaired surrogates are emitted for Korean text near the BMP surrogate boundary — likely a tokenizer or detokenizer edge case.

Workaround

I have installed a local PreToolUse hook that scans tool inputs for unpaired surrogates and blocks the call before it lands in history. Happy to share the script if useful (~50 lines of Python).

Evidence

  • Session jsonl: ~/.claude/projects/<project>/f81aa194-c1eb-48e4-902c-7dc4f6bcd83b.jsonl (52 MB, 6,545 records)
  • First poisoned record: index 6496, timestamp 2026-06-13T07:31:49.622Z, tool_name=AskUserQuestion, codepoint 0xd99c
  • First 400 error: 2026-06-13T08:05:57.428Z
  • Last assistant turn before abandonment: 2026-06-13T14:19:20.178Z (13th identical 400)

View original on GitHub ↗

This issue has 4 comments on GitHub. Read the full discussion on GitHub ↗