Multi-byte UTF-8 characters corrupted in Write tool and Bash tool output

Resolved 💬 3 comments Opened Apr 22, 2026 by ftsmasaki Closed Apr 22, 2026

Description

When using Claude Code with Japanese text (3-byte UTF-8 characters), characters are frequently corrupted to U+FFFD replacement characters. This has become noticeably more frequent in recent sessions.

Environment

  • Claude Code: v2.1.117
  • Model: Claude Opus 4.6 (1M context)
  • OS: WSL2 (Linux 6.6.87.2-microsoft-standard-WSL2)
  • Shell: fish

Reproduction

Two distinct corruption patterns observed:

Pattern 1: Write tool — mid-text corruption

When the Write tool outputs a long Japanese markdown file (~160 lines), random multi-byte characters in the middle of the text are corrupted.

Example:

  • Expected: として正常ハンドリングする
  • Actual: と���て正常ハンドリングする

The corrupted character ��� is U+FFFD, indicating an incomplete UTF-8 byte sequence. In a single Write call, 16 characters across 164 lines were corrupted.

Pattern 2: Bash tool — trailing character truncation

When passing Japanese text as a CLI argument (e.g., gh issue create --title "..."), the final multi-byte character is truncated.

Example:

  • Expected: Phase 1 残りツール実装(11 ツール)
  • Actual: Phase 1 残りツール実装(11 ツール���

The closing full-width parenthesis (3 bytes: EF BC 89) loses its final byte.

Likely Cause

The output streaming buffer appears to split at arbitrary byte boundaries rather than UTF-8 character boundaries. Since Japanese characters are 3 bytes each, a chunk boundary falling mid-character produces incomplete byte sequences that are replaced with U+FFFD.

This is consistent with:

  • Longer texts having more corruptions (more chunk boundaries)
  • Corruption occurring at seemingly random positions
  • Only multi-byte characters being affected (ASCII is never corrupted)

Workaround

  • Re-read the file after Write and manually fix corrupted characters
  • For Bash --title arguments, append a trailing ASCII character (space) to avoid end-of-buffer truncation

Expected Behavior

All UTF-8 characters should be correctly preserved when passed to tool parameters, regardless of text length or character byte width.

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗