Claude Desktop: systematic UTF-8 corruption when writing .md files (Japanese text, Dropbox)

Resolved 💬 1 comment Opened Mar 20, 2026 by jmylot Closed May 24, 2026

Summary

Claude Desktop (Windows) systematically corrupts UTF-8 Japanese text when writing .md files stored on Dropbox (Smart Sync enabled). The corruption pattern is not random — it is a repeatable, systematic encoding bug that replaces specific multi-byte UTF-8 sequences with U+FFFD (replacement character) + ASCII E (0x45).

Environment

  • OS: Windows 11 Pro
  • Claude Desktop: Latest (as of 2026-03-20)
  • Storage: Dropbox with Smart Sync (all files have ReparsePoint attribute via Dropbox filesystem filter driver)
  • Locale: Japanese (CP932 system codepage)
  • File type: .md files containing Japanese UTF-8 text

Corruption Pattern

Primary pattern: EF BF BD 45 (U+FFFD + 'E')

The Write tool replaces certain 3-byte UTF-8 sequences with 4 bytes (EF BF BD 45), causing:

| Original | UTF-8 bytes | Corrupted | Corrupted bytes |
|----------|-------------|-----------|-----------------|
| ホ | E3 83 9B | �E | EF BF BD 45 |
| ( | EF BC 88 | �E | EF BF BD 45 |
| ページ | 9 bytes | ペ�Eジ | 10 bytes |
| テーマ | 9 bytes | チE�Eマ | 10 bytes |

Key evidence this is systematic, not random:

  • In one 61KB file, 1,807 replacement characters were found across 305 of 874 lines (34.9%)
  • Of 1,067 U+FFFD + ASCII pairs, 1,019 (95.5%) had ASCII E (0x45) as the trailing byte
  • The pattern is consistent and reproducible across multiple files

Secondary issues also observed

| Issue | Files affected | Description |
|-------|---------------|-------------|
| UTF-8 BOM insertion | 17 files | EF BB BF prepended to files |
| Mixed line endings | 7 files | Both CRLF and LF in same file |
| Single bit-flip | 1 file | 0x9F0x7F (bit 7 flipped) in a UTF-8 continuation byte |

Reproduction scenario

  1. Store UTF-8 .md files with Japanese text in a Dropbox-synced folder on Windows
  2. Use Claude Desktop to edit/write these files via the Write or Edit tool
  3. Files containing certain Japanese characters (katakana, full-width parentheses, etc.) get corrupted
  4. The corruption appears to happen during the write operation, not during read

Impact

  • Data loss: 1,807 characters irrecoverably corrupted in a single critical documentation file
  • Silent corruption: No error or warning is shown to the user
  • Cumulative: Multiple sessions compound the damage as each write potentially corrupts more characters

Likely root cause hypothesis

The corruption pattern suggests an issue in the file-writing pipeline where:

  1. Certain UTF-8 byte sequences are misinterpreted during an intermediate processing step
  2. The 0x45 ('E') byte appearing 95.5% of the time after U+FFFD points to a specific code path, not random I/O errors
  3. Dropbox's filesystem filter driver (ReparsePoint / SparseFile attributes) may interact poorly with the Write tool's file I/O

Workaround

Created a PostToolUse hook that strips BOM and normalizes CRLF→LF after every Write/Edit operation. However, this does not prevent the U+FFFD corruption itself — it only addresses BOM/line-ending issues.

Request

Please investigate the Write/Edit tool's file I/O path on Windows, specifically:

  1. How UTF-8 multi-byte sequences are handled during write operations
  2. Interaction with Dropbox filesystem virtualization (ReparsePoint attribute)
  3. Whether an intermediate encoding conversion (e.g., through Node.js Buffer) could explain the U+FFFD 45 pattern

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗