[BUG] Write/Edit tools slice CJK characters mid-UTF-8-sequence, leaving partial 0xe3/0xe4 lead bytes at file tail
[BUG] Write/Edit tools slice CJK characters mid-UTF-8-sequence, leaving partial 0xe3/0xe4 lead bytes at file tail
Background
Split-out from master ticket (54891). This issue isolates the UTF-8 char-boundary violation failure mode: write tools (and the bash heredoc + python -c chain) appear to perform byte-level slicing on user content without respecting UTF-8 character boundaries, causing the last CJK character to end mid-sequence with a dangling 3-byte lead (0xe3 / 0xe4 / 0xe5) at the file tail.
Distinct from sibling issues:
- Not a
O_TRUNCproblem (split-01) - Not a Read cache problem (split-03)
- Not a CRLF problem (split-04)
- Not a concurrency problem (split-05)
Suspected root cause: somewhere in the write pipeline a bytes[start..end] slice is taken without converting to a str char-boundary first (Rust str::is_char_boundary / Node Buffer.toString('utf8')). When the cut point falls mid-3-byte CJK sequence, the file ends with an incomplete UTF-8 character.
Reproducer
Environment: Windows 11 + Cowork sandbox bash.
- Create a daily-note style file with heavy CJK content (~5–20 KB):
``bash``
python3 -c "
content = '今日完成項目:\n' + ('一二三四五六七八九十' * 200) + '\n'
open('/vault/daily/2026-04-20.md','w',encoding='utf-8').write(content)
"
- Use
Editon the file (any edit, even a minor frontmatter change). - Verify file tail:
``bashe3
xxd /vault/daily/2026-04-20.md | tail -3
# Observed: trailing bytes are or e4 with no continuation byte.`
`
python``
open('/vault/daily/2026-04-20.md','rb').read().decode('utf-8', errors='strict')
# Raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe3 in position N: unexpected end of data
- Obsidian renders the final CJK character as a replacement glyph (U+FFFD) or omits the line entirely.
Observed in the wild on 2026-04-20.md and 2026-04-22.md daily notes (both CJK-heavy, both ended with byte 0xe3 / 0xe4 after Edit).
Expected behavior
- All write/edit operations on UTF-8 content respect character boundaries; no partial multi-byte sequences ever written.
- If an internal slice operation would split a character, the implementation rounds back to the previous char boundary or returns an error.
data.decode('utf-8', errors='strict')on the resulting file always succeeds.
Reference implementation: Rust str::char_indices() to find safe cut points; Node use TextDecoder / Buffer.toString('utf8') not raw byte slicing; CI fixtures with CJK / emoji / combining-mark heavy content.
Actual behavior
Edit (and Write, observed less frequently) on CJK-heavy .md files truncates the tail mid-character. The file size is one or two bytes short of where a clean character boundary would land. No error is raised at the tool layer; Read returns the (cached) full content; only xxd / Python strict-mode decode reveals the corruption.
Environment
- OS: Windows 11 (build 26100.x)
- Locale: zh-Hant-TW (~80% of writes contain Han characters)
- Mount: virtiofs into Cowork sandbox
- Claude Code: 2.1.123
- Vault: 3,197
.mdfiles / 1.6 GB Obsidian vault
Related
- Parent master ticket: (54891)
- Sibling splits: #54891-split-01 (O_TRUNC), #54891-split-03 (Read cache), #54891-split-04 (CRLF), #54891-split-05 (concurrency)
- Adjacent: bash UTF-8 garbled on Windows (cross-tool encoding chain)
- Adjacent class:
panic: byte index N is not a char boundaryfamily of issues in claude-code Rust core
Workaround (current)
Banned tools for CJK content > 4 KB: Edit, Write, Desktop_Commander.edit_block, bash heredoc + python -c chain. Only reliable path: compose payload in bash heredoc with unique marker, cp to vault, run UTF-8 strict-decode check as part of 9-item verify. Any failure → stop, do not self-heal, escalate to dispatcher.
---
Cross-reference: split from closed master ticket #54891 (auto-closed as not_planned 2026-06-01 by github-actions[bot]; reproducible bugs persist). Also related to #64592 (Cowork VM service cluster). Submitted by dispatcher via REST API on behalf of the issue author (Claude Team subscription holder, IT MSP production use).
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗