Korean (CJK) characters corrupted to U+FFFD in Claude Code responses on macOS + VSCode

Description

Korean text in Claude Code responses is intermittently corrupted — some characters are replaced with ��� (U+FFFD replacement character). This happens in the model's output text itself, not in file I/O or terminal input.

Example observed output:

최근' ��� 규칙량이 에이전트 프'

Where the ��� replaces valid Korean characters, and surrounding text is truncated mid-word.

Environment

• OS: macOS (Darwin 25.3.0)
• Claude Code version: 2.1.86
• Interface: VSCode native extension
• Model: Claude Opus 4.6 (1M context)

Analysis

Korean characters are 3 bytes each in UTF-8 (e.g., 를 = bytes 0xEB 0xA5 0xBC). The U+FFFD pattern indicates byte-level truncation splitting a multibyte character, producing an incomplete byte sequence that gets replaced.

Related closed issues (same root mechanism)

• #16583 — Panic: byte index 22 is not a char boundary; it is inside '를' (bytes 20..23)
• #16490 — Panic: byte index 12 is not a char boundary; it is inside '다' (bytes 10..13)
• #16244, #16112 — Same Rust panic on Korean UTF-8 multibyte truncation

These were fixed in January 2026 by addressing the panic (crash). However, the current symptom suggests the fix converted the panic into a silent replacement (U+FFFD) rather than fixing the underlying byte-boundary truncation logic. The crash is gone, but data corruption persists.

Suspected code paths

• Context compaction — when prior messages are compressed to fit context limits, string truncation may still use byte offsets
• System prompt assembly — combining CLAUDE.md, hooks, skills, and rules may hit a length limit that truncates at byte boundaries
• Streaming chunk boundaries — SSE response chunks may split mid-character

Ruled out

• Vercel plugin hook injection — verified safe: uses Buffer.byteLength() with full include/exclude (never truncates mid-content)
• User's hook scripts — inspected all 4 scripts, no encoding issues
• Terminal/input encoding — corruption is in Claude's output, not user input

Reproduction

• Timing: Irregular — no consistent correlation with conversation length
• Frequency: Intermittent but recurring across sessions
• Content: Affects Korean characters in Claude's generated responses
• Context: Heavy Korean-language project (CLAUDE.md, rules, skills, memory all contain Korean)

Expected behavior

All Korean (and other CJK/multibyte UTF-8) characters should render correctly in Claude Code responses, regardless of context size or compaction state.

Suggested fix

Audit all string truncation paths in the Rust core for character-boundary safety. In Rust, use .char_indices() or .floor_char_boundary() (stable since Rust 1.82) instead of raw byte indexing when truncating strings.