[Bug] Edit tool corrupts Unicode typographic characters in non-ASCII text

This issue has been inactive for 30 days. If the issue is still occurring, please comment to let us know. Otherwise, this issue will be automatically closed in 30 days for housekeeping purposes.

This is still happening for me – not just French, but any text with proper curly quotation marks.

In the meantime I made a script to hunt quotes with a --fix option so that any time claude make this mistake (it happens all day long) it calls the script. This is the only workaround I have for now

i now have this in my CLAUDE.md lol

**If you accidentally replace curly quotes with straight quotes**, use Python
with unicode escapes to fix it. For example, to fix the string "you'd":

python3 -c "
with open('path/to/file.js', 'r') as f:
    content = f.read()
content = content.replace(\"you'd\", \"you\u2019d\")  # \u2019 is the curly apostrophe
with open('path/to/file.js', 'w') as f:
    f.write(content)
"

Note: The `sed` tool and the Edit tool struggle with curly quotes because
they get converted to straight quotes in transit.

Here is my script that targets french langage patterns for the apostrophe.
The script searches accross all your codebase, ignoring the gitignored files.
It has a --fix option to auto fix quotes.
Pretty sure you can adapt the script to any language.
i told claude to use the script when he struggles with a quote.
quote-hunter.ts

Adding two failure modes involving escape sequences, beyond the direct Unicode→ASCII downgrade described above:

\uXXXX escape sequences in new_string are written as literal text:

python3 -c "
with open('/tmp/unicode-test.md', 'w', encoding='utf-8') as f:
    f.write('She said, \u201cHello.\u201d\n')
"

Then Edit with new_string='She said, \u201cHi there.\u201d' — the file receives the literal characters \u201c and \u201d (6 ASCII chars each), not the Unicode code points. So escape sequences don't provide a workaround.

\uXXXX in old_string doesn't match actual Unicode:

old_string='She said, \u201cHello.\u201d'

Returns "String to replace not found in file." The escape sequence is treated as literal text, so the model can't match existing Unicode characters via this route either.

Result: Both the direct-Unicode path (downgraded to ASCII) and the escape-sequence path (literal text / no match) fail. There is no route through the Edit tool that preserves typographic quotes.

Mixed-encoding note: When a region is edited, the affected lines get ASCII quotes while untouched lines retain Unicode — leaving the file with inconsistent quote styles, which is arguably worse than uniform corruption.

Context: This affects prose/markdown files, not just code with French strings. Any fiction or documentation workflow that depends on typographic quotes hits this on every edit. Still present in Claude Code 2.1.45 on macOS (Darwin 25.2.0).

Workaround: A pre-Edit hook that detects Unicode in the target file and blocks the Edit tool, plus a drop-in replacement script (uedit) that reads JSON edits from stdin where \uXXXX escapes are properly decoded by Python's json.loads.

Gist with both files and setup instructions: https://gist.github.com/thejud/802c31016698f1d78dbcecbde2b5575a

The hook adds a CLAUDE.md instruction telling the agent to use uedit instead, so it redirects automatically. Not a fix, but it's been reliable for daily use on a prose-heavy project.

Can confirm the fix in v2.1.47 works. Tested with a file containing French characters (é, è, ç, «, »), curly quotes (" " ' '), curly apostrophes ('), and em dashes (—). All survived an Edit round-trip with correct UTF-8 bytes verified via xxd.

Retiring my workaround hook from the gist above — leaving the files in place in case of regression.

I think this is a more general issue with the models or a step applied for cleanup after the raw model output before returned from serverside.

Using the chat UI to ask "What apostrophes exist in unicode?" gets it to list them, with U+2019 replaced with U+0027 (verified by checking codepoint).

Tested both Sonnet 4.6 and Opus 4.6.

I also had this break code before, by changing a .replace("[U+2019]", "[U+0027]") to .replace("[U+0027]", "[U+0027]"), making it a nop.

Your description is spot on — the correct characters are already in the files, and Claude Code’s Edit operation actively destroys them. This is especially painful for French (every apostrophe is affected) but impacts any language using typographic marks.

I’ve filed a feature request proposing a /curly option: instead of blind normalization, a lightweight agent verifies whether each curly character is a code delimiter (fix it) or textual content (preserve it). This would solve both the original bug (#29786, curly in code) and the reverse problem you describe.

→ [FEATURE] /curly option: context-aware curly/straight character handling in Write/Edit — #31482

I don't think your feature request makes sense as this isn't a claude code bug, but a more general claude one. I don't think such a toggle would make sense either.

(also, please consider writing your own comments)

Incredible, a human!

No, my proposal isn't just a button, but an agent (AI or otherwise) that can be activated, specialized in detecting curved characters, and therefore capable of correcting Claude Code.

Furthermore, I offer explanations that I wish I had received from Anthropic, and I also offer skills for those who are interested.

Regarding the automation of my messages by AI, this stems from the numerous issues I've opened as a human, to which the only response I've received is a bot that automatically closes the conversations.

Closing for now — inactive for too long. Please open a new issue if this is still relevant.

This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.