›_ Claude Code Issues
Claude Code Issues / #6903

Unicode Codepoint Corruption During Write Operation

Resolved 💬 3 comments Opened Aug 31, 2025 by ariccio Closed Sep 3, 2025

Environment

  • Platform (select one):
  • [ ] Anthropic API
  • [ ] AWS Bedrock
  • [ ] Google Vertex AI
  • [X] Other: Claude Code
  • Claude CLI version: v1.0.98
  • Operating System: macOS 15.6
  • Terminal: zsh 5.9 (arm64-apple-darwin24.0), macOS terminal Version 2.14 (455.1)

Bug Description

VERY intermittent unicode corruption through the Write tool. I spent several hours analyzing it - and am UNABLE to reproduce. Extensive analysis points out that some parts of the mechanism make sense - somehow the box drawing character (a single codepoint) gets split at the byte level (inside the codepoint!) and then gets further garbled as string processing zeros out the orphaned surrogate pairs.

This occurred when executing the /init command, with an empty CLAUDE.md. A few lines got corrupted. The first of those was ├. The current state of the larger section of CLAUDE.md (Lines 12-20) is:

\^\\^@\^@ cli.js              # CORRUPTED: Should be ├──
\^\\^@\^@ sdk.mjs             # CORRUPTED: Should be ├──
\^\\^@\^@ sdk.d.ts            # CORRUPTED: Should be ├──
\^\\^@\^@ sdk-tools.d.ts      # CORRUPTED: Should be ├──
\^\\^@\^@ yoga.wasm           # CORRUPTED: Should be ├──
\^T\^@\^@ vendor/             # CORRUPTED: Should be └──
   \^\\^@\^@ claude-code.vsix # CORRUPTED: Should be ├──
   \^\\^@\^@ claude-code-jetbrains-plugin/ # CORRUPTED: Should be ├──
   \^T\^@\^@ ripgrep/        # CORRUPTED: Should be └──

I also - for some annoying reason - ran into an issue with the /bug tool, that simply said Error submitting bug report.

Steps to Reproduce

???

Expected Behavior

Write tool correctly writes unicode to files.

Actual Behavior

Claude (generates UTF-8 ├── cli.js: E2 94 9C .....)
↓
Write Tool API (receives UTF-8)
↓ (verified in the jsonl logs that the Write tool was indeed passed uncorrupted text)
?????
↓ [❌ CORRUPTION HAPPENS HERE]
File System Write
↓ [❌ Corrupted data]
Disk (stores corrupted bytes: 1C 00 00: control char + NULLs)

Additional Context

After several hours of claude code accelerated javascript reverse engineering and testing, here's one possible mechanism:

  1. UTF-8 Sequence Breakdown:

├ = E2 94 9C (3 bytes)
└ = E2 94 94 (3 bytes)
─ = E2 94 80 (3 bytes)

  1. Buffer Splitting?:

During diff processing, buffers get sliced at arbitrary positions. This can break UTF-8 sequences, creating invalid byte sequences

  1. Masking Operation:

Broken UTF-8 bytes get processed with & 0x7F operation

  • 0x94 & 0x7F = 0x14
  • 0x9C & 0x7F = 0x1C
  • 0x80 & 0x7F = 0x00
  1. Null Padding:

The corrupted bytes get null-padded, creating:

  • 14 00 00 (from 0x94)
  • 1C 00 00 (from 0x9C)
  • 00 00 00 (from 0x80)
  1. Result:

Box drawing characters become corruption patterns that match the exact hex patterns seen in the actual corrupted files.

Nicely formatted by claude, here's a table:
| Unicode Character | UTF-8 Encoding | Corrupted Output | Analysis |
|------------------|----------------|------------------|----------|
| ├ (U+251C) | 0xE2 0x94 0x9C | 0x1C 0x00 0x00 | Low byte + padding |
| └ (U+2514) | 0xE2 0x94 0x94 | 0x14 0x00 0x00 | Low byte + padding |
| ─ (U+2500) | 0xE2 0x94 0x80 | 0x00 0x00 0x00 | 0x80 overflow → 0x00 |

There's a ton of similar issues already open, most unresolved:

unicode encoding

utf-8 encoding

chinese

multiedit related?

specifically the write tool

suggests it's hook related

suggests the file has to be NON-UTF-8 beforehand (worth trying somehow)

says something about low surrogates?

unicode reference

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗