Garbled Chinese characters (U+FFFD) in streaming output near Markdown formatting markers

Resolved 💬 6 comments Opened Mar 26, 2026 by TonyLTalentexe Closed May 16, 2026

Description

Chinese characters occasionally render as ��� (U+FFFD replacement characters) in Claude Code's streaming text output. The corruption appears specifically when multi-byte UTF-8 Chinese punctuation is adjacent to Markdown formatting markers.

Reproduction

  1. Ask Claude Code (Opus 4.6) to produce a long Chinese analysis with heavy Markdown formatting (bold, tables, headers)
  2. Observe the output for garbled characters

Specific example

Expected output:

这篇论文本身质量不错,但含金量主要来自 Zitnik 实验室的平台势能和方向卡位,而非 Pengwei Sui 个人的独立学术能力。

Actual output:

这篇论文本身质量不错,但含金量主要来自 Zitnik 实验室的平台势能和方向卡位,而非 Pengwei Sui 个人的独立学术能力���

The (U+3002, 3-byte UTF-8: E3 80 82) at the end was corrupted into ���.

Analysis

  • The underlying tokens are correct (verified by the user via API inspection)
  • Third-party rendering platforms show the same issue, ruling out client-side rendering bugs
  • The issue appears to be in the streaming output pipeline — likely a token boundary splitting a multi-byte UTF-8 sequence mid-character
  • Chinese punctuation marks (3-byte UTF-8) near Markdown formatting closers (**, *, ` ``) seem most prone to this

Environment

  • Claude Code CLI (macOS, Darwin 25.3.0)
  • Model: claude-opus-4-6 (1M context)
  • Language: Chinese (zh-CN)
  • Shell: zsh

Expected behavior

All UTF-8 characters should be delivered as complete byte sequences in the streaming output, regardless of token boundaries or proximity to Markdown formatting markers.

View original on GitHub ↗

This issue has 6 comments on GitHub. Read the full discussion on GitHub ↗