Edit tool replaces Turkish UTF-8 characters with ASCII equivalents

Found 3 possible duplicate issues:

1. https://github.com/anthropics/claude-code/issues/7335
2. https://github.com/anthropics/claude-code/issues/13080
3. https://github.com/anthropics/claude-code/issues/7134

This issue will be automatically closed as a duplicate in 3 days.

• If your issue is a duplicate, please close it and 👍 the existing issue instead
• To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

This is also broken for Unicode quotes (‘, ’, “, ”) being replaced by ASCII quotes (', ").

● Yes, I understand:

  - For term quotations: U+2018 (LEFT SINGLE QUOTATION MARK) and U+2019 (RIGHT SINGLE QUOTATION MARK)
  - For parsed terms: U+201C (LEFT DOUBLE QUOTATION MARK) and U+201D (RIGHT DOUBLE QUOTATION MARK)

  The issue is that when I write edits, they may be getting converted to ASCII quotes (U+0027 or U+0022). Let me try writing with explicit Unicode escapes or see if I can preserve them.

Such a capable model held back by such garbage software.

I should note I am using GNU/Linux - this is not just a Windows issue.

Consolidated Impact Analysis - This Needs Priority Attention

I've analyzed this issue along with at least 6 related open issues that all stem from the same root cause: Claude Code's Edit/Write tools have fundamental UTF-8 encoding bugs.

Related Issues (All Still Open/Affecting Users)

| Issue | Description | Status |
|-------|-------------|--------|
| #13939 | Turkish UTF-8 characters → ASCII | Open |
| #13080 | UTF-8 multi-byte chars → NULL bytes | Open |
| #7134 | Windows-1252 encoding corruption | Open |
| #7332 | Chinese characters garbled | Open |
| #7335 | Swedish å,ä,ö → replacement char | Recently closed (inactivity, not fixed) |
| #6246, #5518, #3416 | Earlier duplicates | Various |

Why This Is High Priority

1. Affects ALL non-English users - Turkish, German, French, Spanish, Portuguese, Chinese, Japanese, Nordic languages, and anyone using Unicode symbols

1. Silent data corruption - No warning is given. Users discover corruption after the fact, often after committing to git

1. 100% reproducible - This isn't a flaky edge case. It happens every single time

1. Core functionality is broken - Edit and Write are fundamental tools. If they corrupt data, Claude Code cannot be trusted for international codebases

1. Token waste - Users spend tokens asking Claude to fix the corruption it caused (see comment in #7134: "fixing the errors generated by this issue consumes a lot of tokens")

1. Competitive disadvantage - Users in #7332 note "Gemini CLI seems to be free of this problem"

Technical Pattern Identified

From #13080's excellent analysis, the bug appears to be truncating UTF-8 multi-byte sequences to single bytes:

• Character ─ (U+2500) should be e2 94 80 (3 bytes)
• Actually written as 00 (just codepoint & 0xFF)

This suggests the encoding pipeline is treating Unicode codepoints as single bytes somewhere in the Write/Edit implementation.

Request

Could this be escalated internally? The volume of duplicate issues (spanning 6+ months) and the fundamental nature of the bug suggest this deserves engineering attention before more features are added.

cc @dicksontsai (who acknowledged this pattern in #7335)

---
🤖 Generated with Claude Code

Root Cause Analysis: The Problem is Deeper Than the Edit Tool

I have been debugging this issue systematically and found that the corruption happens at the model generation level, not in the Edit/Write tool implementation.

Experiment

I tested by having Claude write a file with explicit Unicode quote characters. When Claude tries to write:

1. Left single quote: ‘
2. Right single quote: ’

The actual bytes written are all ASCII 0x27, not Unicode U+2018/U+2019.

Tracing the Code Path

I examined the pi-coding-agent infrastructure (which uses the same Anthropic SDK):

1. Write tool: Simply calls fsWriteFile(path, content, "utf-8") - passes content through unchanged
2. Anthropic provider: Tool arguments come directly from the API JSON response via standard JSON.parse
3. No normalization happens in the infrastructure code

Conclusion

The model itself cannot generate Unicode curly quotes in tool call parameters. When Claude generates what appears to be curly-quoted text, the actual tokens produced are ASCII apostrophes (0x27), not Unicode LEFT/RIGHT SINGLE QUOTATION MARKs (U+2018/U+2019).

This is a tokenization or generation constraint at the Anthropic model level, not a bug in Claude Code's Edit/Write implementation. The infrastructure faithfully writes exactly what the model generates - but the model generates ASCII where Unicode was intended.

Workaround

Use bash with explicit hex escapes:

printf '\xe2\x80\x98test\xe2\x80\x99' > file.txt  # Writes ‘test’ with curly quotes

Implications

This suggests the fix needs to happen in:

1. Model training/tokenization (so Claude can generate Unicode tokens), or
2. A post-processing layer that converts ASCII quotes to Unicode based on context

The Edit/Write tool code appears correct - it is faithfully transmitting what the model produces.

This does not only effect non-English users. We have a lot of code with typographic quotation marks, e.g. the line

``log.error(s"$link specifies StudyOID “$oid”. Cannot find it in database.")``
gets transformed into

``log.error(s"$link specifies StudyOID "$oid". Cannot find it in database.")``
breaking the code (does not compile). When I tell the agent, the straight quotation marks just get escaped. The code works again but the output does not look as intended.

After many loops of "sorry, but my edit tool cannot tell the difference" and trying various workarounds, I finally get the result. But each time (context window) I start from scratch. Please fix this.

Just to add here another "me too". I have suffered from this kind of corruptions since day one of using Clade Code. And even with precise instructions to workaround this (for instance, using some script to convert from ANSI to UTF8 before editing the files) Claude some times forgots to do this. And fixing this errors as some other users have said costs tokens or a lot of time to fix something that was not broken on the first place.

Anthropic confirmed to me - after much delay and hoop jumping - when I had Priority support that they have no intention of prioritising this issue. I cancelled my subscription and encourage everyone to do the same.

Closing for now — inactive for too long. Please open a new issue if this is still relevant.

This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.