Multi-byte UTF-8 characters corrupted in Write tool and Bash tool output
Description
When using Claude Code with Japanese text (3-byte UTF-8 characters), characters are frequently corrupted to U+FFFD replacement characters. This has become noticeably more frequent in recent sessions.
Environment
- Claude Code: v2.1.117
- Model: Claude Opus 4.6 (1M context)
- OS: WSL2 (Linux 6.6.87.2-microsoft-standard-WSL2)
- Shell: fish
Reproduction
Two distinct corruption patterns observed:
Pattern 1: Write tool — mid-text corruption
When the Write tool outputs a long Japanese markdown file (~160 lines), random multi-byte characters in the middle of the text are corrupted.
Example:
- Expected:
として正常ハンドリングする - Actual:
と���て正常ハンドリングする
The corrupted character ��� is U+FFFD, indicating an incomplete UTF-8 byte sequence. In a single Write call, 16 characters across 164 lines were corrupted.
Pattern 2: Bash tool — trailing character truncation
When passing Japanese text as a CLI argument (e.g., gh issue create --title "..."), the final multi-byte character is truncated.
Example:
- Expected:
Phase 1 残りツール実装(11 ツール) - Actual:
Phase 1 残りツール実装(11 ツール���
The closing full-width parenthesis ) (3 bytes: EF BC 89) loses its final byte.
Likely Cause
The output streaming buffer appears to split at arbitrary byte boundaries rather than UTF-8 character boundaries. Since Japanese characters are 3 bytes each, a chunk boundary falling mid-character produces incomplete byte sequences that are replaced with U+FFFD.
This is consistent with:
- Longer texts having more corruptions (more chunk boundaries)
- Corruption occurring at seemingly random positions
- Only multi-byte characters being affected (ASCII is never corrupted)
Workaround
- Re-read the file after Write and manually fix corrupted characters
- For Bash
--titlearguments, append a trailing ASCII character (space) to avoid end-of-buffer truncation
Expected Behavior
All UTF-8 characters should be correctly preserved when passed to tool parameters, regardless of text length or character byte width.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗