[BUG] Write/Edit tools slice CJK characters mid-UTF-8-sequence, leaving partial 0xe3/0xe4 lead bytes at file tail

Resolved 💬 3 comments Opened Jun 2, 2026 by aiken884 Closed Jul 5, 2026

[BUG] Write/Edit tools slice CJK characters mid-UTF-8-sequence, leaving partial 0xe3/0xe4 lead bytes at file tail

Background

Split-out from master ticket (54891). This issue isolates the UTF-8 char-boundary violation failure mode: write tools (and the bash heredoc + python -c chain) appear to perform byte-level slicing on user content without respecting UTF-8 character boundaries, causing the last CJK character to end mid-sequence with a dangling 3-byte lead (0xe3 / 0xe4 / 0xe5) at the file tail.

Distinct from sibling issues:

  • Not a O_TRUNC problem (split-01)
  • Not a Read cache problem (split-03)
  • Not a CRLF problem (split-04)
  • Not a concurrency problem (split-05)

Suspected root cause: somewhere in the write pipeline a bytes[start..end] slice is taken without converting to a str char-boundary first (Rust str::is_char_boundary / Node Buffer.toString('utf8')). When the cut point falls mid-3-byte CJK sequence, the file ends with an incomplete UTF-8 character.

Reproducer

Environment: Windows 11 + Cowork sandbox bash.

  1. Create a daily-note style file with heavy CJK content (~5–20 KB):

``bash
python3 -c "
content = '今日完成項目:\n' + ('一二三四五六七八九十' * 200) + '\n'
open('/vault/daily/2026-04-20.md','w',encoding='utf-8').write(content)
"
``

  1. Use Edit on the file (any edit, even a minor frontmatter change).
  2. Verify file tail:

``bash
xxd /vault/daily/2026-04-20.md | tail -3
# Observed: trailing bytes are
e3 or e4 with no continuation byte.
`
`python
open('/vault/daily/2026-04-20.md','rb').read().decode('utf-8', errors='strict')
# Raises UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe3 in position N: unexpected end of data
``

  1. Obsidian renders the final CJK character as a replacement glyph (U+FFFD) or omits the line entirely.

Observed in the wild on 2026-04-20.md and 2026-04-22.md daily notes (both CJK-heavy, both ended with byte 0xe3 / 0xe4 after Edit).

Expected behavior

  1. All write/edit operations on UTF-8 content respect character boundaries; no partial multi-byte sequences ever written.
  2. If an internal slice operation would split a character, the implementation rounds back to the previous char boundary or returns an error.
  3. data.decode('utf-8', errors='strict') on the resulting file always succeeds.

Reference implementation: Rust str::char_indices() to find safe cut points; Node use TextDecoder / Buffer.toString('utf8') not raw byte slicing; CI fixtures with CJK / emoji / combining-mark heavy content.

Actual behavior

Edit (and Write, observed less frequently) on CJK-heavy .md files truncates the tail mid-character. The file size is one or two bytes short of where a clean character boundary would land. No error is raised at the tool layer; Read returns the (cached) full content; only xxd / Python strict-mode decode reveals the corruption.

Environment

  • OS: Windows 11 (build 26100.x)
  • Locale: zh-Hant-TW (~80% of writes contain Han characters)
  • Mount: virtiofs into Cowork sandbox
  • Claude Code: 2.1.123
  • Vault: 3,197 .md files / 1.6 GB Obsidian vault

Related

  • Parent master ticket: (54891)
  • Sibling splits: #54891-split-01 (O_TRUNC), #54891-split-03 (Read cache), #54891-split-04 (CRLF), #54891-split-05 (concurrency)
  • Adjacent: bash UTF-8 garbled on Windows (cross-tool encoding chain)
  • Adjacent class: panic: byte index N is not a char boundary family of issues in claude-code Rust core

Workaround (current)

Banned tools for CJK content > 4 KB: Edit, Write, Desktop_Commander.edit_block, bash heredoc + python -c chain. Only reliable path: compose payload in bash heredoc with unique marker, cp to vault, run UTF-8 strict-decode check as part of 9-item verify. Any failure → stop, do not self-heal, escalate to dispatcher.

---

Cross-reference: split from closed master ticket #54891 (auto-closed as not_planned 2026-06-01 by github-actions[bot]; reproducible bugs persist). Also related to #64592 (Cowork VM service cluster). Submitted by dispatcher via REST API on behalf of the issue author (Claude Team subscription holder, IT MSP production use).

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗