Unpaired Unicode surrogates from tool results corrupt session history, causing all subsequent API calls to fail
Description
When a tool (particularly Playwright/browser automation) returns content containing unpaired Unicode surrogates, that data gets stored in the conversation history. All subsequent API requests for that session then fail with:
API Error: 400 {"type":"error","error":{"type":"invalidrequesterror","message":"The request body is not valid JSON: no low surrogate in string: line 1 column 222905 (char 222904)"}}
Once this happens, the session is permanently corrupted - every future request fails because the malformed history is included in each API call.
Steps to Reproduce
- Start a Claude Code session
- Use Playwright to navigate to a website with malformed UTF-16 encoding (e.g.,
sonnysfarm.com/products/one-dozen-eggs) - The tool result containing bad Unicode gets stored in history
- All subsequent prompts in that session fail with the "no low surrogate" error
Expected Behavior
Tool results should be sanitized before being stored in conversation history. Unpaired surrogates (U+D800-U+DBFF not followed by U+DC00-U+DFFF, or vice versa) should be replaced with the Unicode replacement character (U+FFFD) or stripped.
Actual Behavior
Raw tool output is stored directly in history without sanitization, corrupting the session permanently.
Suggested Fix
Add Unicode sanitization when storing tool results in conversation history:
function sanitizeUnicode(str) {
if (typeof str !== "string") return str;
return str
.replace(/[\uD800-\uDBFF](?![\uDC00-\uDFFF])/g, "\uFFFD")
.replace(/(?<![\uD800-\uDBFF])[\uDC00-\uDFFF]/g, "\uFFFD");
}
Environment
- Claude Code CLI
- Observed when using Playwright MCP for web scraping
- The problematic characters typically come from websites with malformed HTML/encoding
Workaround
Currently the only workaround is to start a fresh session, losing all conversation context. Users cannot recover a corrupted session.
Impact
This is particularly disruptive for long-running sessions where significant context has been built up. One bad website scrape can destroy an entire session's worth of work.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗