[BUG] U+FFFD corruption in Cyrillic text — both streaming output and Edit/Write tool
Bug description
Cyrillic characters are silently replaced with U+FFFD (��) in two places:
- Streaming output — model responses in terminal contain replacement characters
- Edit/Write tool — files written by Claude contain corrupted characters
Environment
- Windows 11, Claude Code 2.1.85, model Opus 4.6
- File encoding: UTF-8
- Affected:
.mdfiles with Russian text
Reproduction
- Ask Claude Code (in Russian) to edit a
.mdfile containing Cyrillic text - Observe the output and the resulting file
Input: контейнера, после, настройки, формулировки, Лучше
Output: конте��нера, пос��е, на��тройки, формулиров��и, ��учше
The corruption is intermittent — it depends on where SSE chunk boundaries fall within multi-byte UTF-8 sequences.
Root cause
Cyrillic characters are 2-byte UTF-8 sequences. When an SSE chunk boundary splits a character between its bytes, each orphaned byte is decoded independently and replaced with U+FFFD.
The SSE line decoder in the Node.js CLI uses TextDecoder without the { stream: true } option, which would carry incomplete byte sequences across chunks instead of replacing them.
Proposed fix
In the SSE streaming decoder, change:
const decoder = new TextDecoder();
to:
const decoder = new TextDecoder('utf-8', { stream: true });
This is a one-line fix. The { stream: true } option tells TextDecoder to buffer incomplete multi-byte sequences until the next chunk arrives, instead of emitting U+FFFD.
Related issues
- #1716 — UTF-8 Corruption Bug (open 10 months, Cyrillic)
- #40574 — Garbled characters since v2.1.86 (CJK)
- #43746 — TextDecoder missing
{ stream: true }(CJK, detailed diagnosis)
This issue adds a concrete Cyrillic reproduction case and confirms the same root cause applies to 2-byte UTF-8 sequences (Cyrillic), not only 3-byte (CJK).
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗