Edit tool silently corrupts Unicode characters (curly quotes, em dashes, etc.)

Edit tool silently corrupts Unicode characters (curly quotes, em dashes, etc.)

Summary

The Edit tool cannot write non-ASCII Unicode characters. When the model's new_string contains Unicode characters like curly quotes (U+201C, U+201D), they are silently downgraded to ASCII equivalents (straight quotes U+0022). When the model tries to use \uXXXX escape sequences, they are written as literal text rather than being interpreted as Unicode code points.

This makes the Edit tool unusable for any file containing typographic quotes, em dashes, or other non-ASCII characters — which includes most prose/fiction, many markdown documents, and code with non-ASCII string literals.

Reproduction

Setup

Create a file with Unicode curly quotes:

python3 -c "
with open('/tmp/unicode-test.md', 'w', encoding='utf-8') as f:
    f.write('She said, \u201cHello.\u201d\n\nHe replied, \u201cGoodbye.\u201d\n')
"

Test 1: Edit replaces curly quotes with straight quotes

Read the file, then use Edit to change "Hello" to "Hi there":

Edit(file_path="/tmp/unicode-test.md",
     old_string='She said, "Hello."',
     new_string='She said, "Hi there."')

Result: The edit succeeds (old_string matches via normalization), but the output line now has straight ASCII quotes (") where the file originally had curly quotes (\u201c / \u201d). The untouched line retains its curly quotes. The file now has mixed quote styles.

Before:

She said, \u201cHello.\u201d        <- curly
He replied, \u201cGoodbye.\u201d    <- curly

After:

She said, "Hi there."               <- straight (corrupted)
He replied, \u201cGoodbye.\u201d    <- curly (untouched)

Test 2: Unicode escape sequences written as literal text

Edit(file_path="/tmp/unicode-test.md",
     old_string="plain text",
     new_string='she said, \u201cplain text\u201d')

Result: The file contains the literal characters \u201c and \u201d (six ASCII characters each), not the Unicode code points they represent.

Test 3: Escape sequences don't match in old_string either

Edit(file_path="/tmp/unicode-test.md",
     old_string='He replied, \u201cGoodbye.\u201d',
     new_string='He replied, \u201cFarewell.\u201d')

Result: "String to replace not found in file." The \u201c in old_string is treated as literal text, not as a Unicode character reference.

Impact

• Silent data corruption: Edits succeed but silently downgrade Unicode to ASCII in the affected region. No warning is produced.
• Mixed encoding: Untouched lines keep their Unicode characters while edited lines get ASCII, creating inconsistent files.
• No workaround via the Edit tool: Neither direct Unicode characters nor \uXXXX escape sequences produce the intended output.
• Affects all non-ASCII characters: Curly quotes, em dashes, accented characters, CJK characters, etc.

Current workaround

Use Python via the Bash tool for any edits involving non-ASCII characters:

python3 -c "
with open('file.md', 'r', encoding='utf-8') as f:
    content = f.read()
content = content.replace('old text', 'new text')  # Python preserves Unicode
with open('file.md', 'w', encoding='utf-8') as f:
    f.write(content)
"

This works but defeats the purpose of having a dedicated Edit tool — it's slower, harder to review, and bypasses the Edit tool's diff display.

Expected behavior

The Edit tool should preserve Unicode characters exactly as they appear in the file. If old_string matches a region containing curly quotes, and new_string contains curly quotes (whether via direct Unicode or escape sequences), the output should contain curly quotes.

Environment

• Claude Code CLI
• macOS (Darwin 25.2.0)
• Model: claude-opus-4-6