Garbled Chinese characters (U+FFFD) in streaming output near Markdown formatting markers
Resolved 💬 6 comments Opened Mar 26, 2026 by TonyLTalentexe Closed May 16, 2026
Description
Chinese characters occasionally render as ��� (U+FFFD replacement characters) in Claude Code's streaming text output. The corruption appears specifically when multi-byte UTF-8 Chinese punctuation is adjacent to Markdown formatting markers.
Reproduction
- Ask Claude Code (Opus 4.6) to produce a long Chinese analysis with heavy Markdown formatting (bold, tables, headers)
- Observe the output for garbled characters
Specific example
Expected output:
这篇论文本身质量不错,但含金量主要来自 Zitnik 实验室的平台势能和方向卡位,而非 Pengwei Sui 个人的独立学术能力。
Actual output:
这篇论文本身质量不错,但含金量主要来自 Zitnik 实验室的平台势能和方向卡位,而非 Pengwei Sui 个人的独立学术能力���
The 。 (U+3002, 3-byte UTF-8: E3 80 82) at the end was corrupted into ���.
Analysis
- The underlying tokens are correct (verified by the user via API inspection)
- Third-party rendering platforms show the same issue, ruling out client-side rendering bugs
- The issue appears to be in the streaming output pipeline — likely a token boundary splitting a multi-byte UTF-8 sequence mid-character
- Chinese punctuation marks (3-byte UTF-8) near Markdown formatting closers (
**,*, ```) seem most prone to this
Environment
- Claude Code CLI (macOS, Darwin 25.3.0)
- Model: claude-opus-4-6 (1M context)
- Language: Chinese (zh-CN)
- Shell: zsh
Expected behavior
All UTF-8 characters should be delivered as complete byte sequences in the streaming output, regardless of token boundaries or proximity to Markdown formatting markers.
This issue has 6 comments on GitHub. Read the full discussion on GitHub ↗