JSON.stringify produces lone surrogates in API request body (400 invalid high surrogate)
Bug
Claude Code occasionally sends API requests containing lone Unicode surrogates (U+D800–U+DFFF), causing the Anthropic API to reject them with:
API Error: 400 {"type":"error","error":{"type":"invalid_request_error",
"message":"The request body is not valid JSON: invalid high surrogate in string: line 1 column 411126 (char 411125)"}}Root Cause
Node.js strings are internally UTF-16. When in-memory strings contain lone surrogates (from file reads, terminal output, or web content), JSON.stringify() serializes them as \uD8xx without a following low surrogate — which is invalid JSON per RFC 8259.
The stored JSONL files are clean (verified: no surrogates in the JSONL, subagent files, or tool-result cache). The lone surrogate is introduced transiently during request body construction in Node.js memory.
Common triggers on Windows
- Emoji in code/comments read from files
- Box-drawing characters from terminal/build output
- Scraped web content with malformed UTF-16
- Any
Bashtool result containing non-BMP characters
Suggested Fix
Sanitize strings before JSON.stringify() when constructing the API request body. Node.js 20+ has a one-liner:
const safe = str.toWellFormed() // replaces lone surrogates with U+FFFDFor older Node.js:
str.replace(/[\uD800-\uDBFF](?![\uDC00-\uDFFF])/g, '\uFFFD')
.replace(/(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/g, '\uFFFD')Impact
- Session becomes stuck (the failed response is recorded, retrying may work but isn't guaranteed)
- Loss of in-progress work context if user has to start a new session
- More likely on Windows due to cp1252/UTF-16 encoding mismatches
Environment
- Windows 11 Pro
- Claude Code VS Code extension
- Node.js (whatever version ships with Claude Code)
- Python 3.14 confirmed the JSONL files themselves are valid UTF-8
Reproduction
Happens consistently when copying structured output (bullet points, chat bubbles, etc.) from another chat window into the Claude code VS extension chat field and sending it.
7 Comments
Found 3 possible duplicate issues:
This issue will be automatically closed as a duplicate in 3 days.
🤖 Generated with Claude Code
Additional reproduction data: clean session with zero supplementary Unicode still fails
I'm hitting this consistently on Windows 11 Pro, Claude Code v2.1.94 (CLI), Opus 4.6.
Key finding: content-independent reproduction
I analyzed 4 bricked sessions from the same day. The critical one is a fresh session started after
/clearthat contained:It still failed at column 110200:
This eliminates all content-based theories (emoji, web scraping, terminal escape sequences, file encoding issues). The JSONL on disk is verifiably clean — the corruption is purely transient during request body construction.
Likely trigger: large system prompt assembly
Column 110200 falls within the system prompt region of the request, not user content. My setup injects substantial context per request:
The larger the assembled system prompt, the more buffer boundaries exist during string concatenation, increasing the probability of splitting a multi-byte UTF-8 sequence and producing a lone surrogate in the JS UTF-16 string.
Session cascade data
| Session | Start | Content | Error column | Trigger |
|---------|-------|---------|-------------|---------|
| 3e2de9c5 | 00:32 | 360 lines, 809KB, 12 WebFetch (had emoji) | 337749 | Normal facienda + research |
| 6bd978f4 | 01:27 | 296 lines, 598KB | (same) | Pasted error from above |
| 65852305 | 01:12 | 315 lines, 901KB | (same) | Pasted error from above |
| 40836368 | 09:29 | 79 lines, 230KB, zero emoji | 110200 | Fresh post-/clear facienda |
Environment
Suggested fix (echoing OP)
str.toWellFormed()(Node 20+) on all strings beforeJSON.stringify()in the API request path. This is the standard mitigation for transient lone surrogates in Node.js UTF-16 strings.Adding a macOS + VS Code extension data point for this issue.
### Environment
claudeCLI on the same machine: works normally### Symptom
Every message via the VS Code extension fails with 400
invalid high, even on brand-new sessions with trivial input likesurrogate
測試:API Error: 400 {"type":"error","error":{"type":"invalid_request_error",
"message":"The request body is not valid JSON: invalid high surrogate in
string:
line 1 column 2979 (char 2978)"},"request_id":"req_011Ca224F35QVUmZf322vT1e"}
Other request IDs observed:
req_011Ca21bQMQpTCwQXyTJ6ced,req_011Ca21s1mJdKR7nuuFnghcg.### Confirming content-independence (matches @gregmarkowitz-gif's finding)
.jsonlfiles under~/.claude/projects/—zero lone UTF-16 surrogates (raw or escaped)
— all valid UTF-8, no lone surrogates
--continue/--resume### Why likely system-prompt-size related
My VS Code environment loads a substantial assembled system prompt per
request:
auto-suggestions (hooks fire on every
UserPromptSubmit)deferred tools
~/.claude/projects/*/memory/MEMORY.md) loaded everyturn
CLAUDE.md(~2KB)The terminal CLI on the same machine, same account, same settings works —
which fits the theory that the bug triggers above some assembled-prompt size
threshold, and the VS Code extension packages more context than the CLI.
### Reproduction
Trivially reproducible — any message in the VS Code extension fails. Disabling
the Vercel plugin alone did not resolve it on my end.
### Request
Please prioritize the
str.toWellFormed()fix suggested by OP — this isblocking VS Code extension usage entirely for users with heavier plugin/MCP
setups, even though the CLI remains usable.
👎'd the duplicate-close bot to keep this open.
Closing for now — inactive for too long. Please open a new issue if this is still relevant.
Still reproducing on v2.1.x. Adding a concrete, deterministic repro path that may help, since the existing reports mostly describe paste/terminal sources.
Trigger: byte-boundary truncation in a SessionStart hook.
A
SessionStarthook that injects file content into context usedhead -c 1500 "$file".head -ccuts on a byte boundary, not a character boundary. When the cut lands in the middle of a multi-byte UTF-8 character (e.g.✅=E2 9C 85), only the leading byte(s) survive. That partial byte sequence ends up as a lone surrogate (\udce2in our case), which is injected into context every session, persisted into the.jsonltranscript, and then re-serialized into the request body when a new window loads that transcript — producing:Isolated proof: byte-cut at offset 15 of
"...emoji ✅..."yields trailing bytes20 e2→ exactly the\udce2we found in 7 stored transcripts.Two observations:
.toWellFormed()beforeJSON.stringify()of the request body would make this class of corruption non-fatal regardless of source (paste, terminal, web, hooks, tool output). The session becomes permanently un-openable otherwise, since the bad surrogate is now baked into the saved transcript.head -c/ byte-slice text destined for context. Truncate on character boundaries (e.g. decode UTF-8 witherrors='ignore'then slice by characters). We also had to sanitize already-corrupted transcripts: parse each JSONL line, strip lone surrogates from the parsed string values (keeping valid pairs), and write back withjson.dumps(..., ensure_ascii=False).Given the saved-transcript persistence makes a session unrecoverable, this seems worth reopening — sanitizing at serialization time would prevent the unrecoverable state for all of the trigger sources listed above.
The existing reports here trace this to bash tool output, binary file reads, or plugin content with emoji — all external sources of lone surrogates. I found a different trigger path: the model's own generated text inside a tool_use input can itself contain a lone surrogate.
Concretely, in a session that started hitting
no low surrogate in stringon every subsequent turn, I traced the corrupted character back to anAskUserQuestiontool_use call where one of the option descriptions (Korean text) contained\ud831with no matching low surrogate — sitting in place of what should have been a normal character. Several nearby option labels in the same session also showed similar-but-wrong Hangul syllables, suggesting some kind of generation-time corruption rather than a truncation/copy-paste source.This path isn't covered by the v2.1.166 fix (which scoped to MCP server descriptions and classifier side-queries), since the corrupted text originates from the model's own tool-call arguments rather than an external string being embedded. Once it lands in the session transcript, every following turn re-sends it and fails the same way — only a fresh session recovers. Wanted to flag this as a distinct repro path in case it's useful for scoping a more general fix (sanitizing any string before
JSON.stringify(), not just the two patched call sites).This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.