[BUG] Model drifts full-width CJK punctuation to half-width in Edit's old_string, causing silent "String to replace not found"
What's Wrong?
When editing a file that contains full-width CJK punctuation (e.g. (, ), 、, 。, :, ;), the Edit tool frequently fails with String to replace not found — even though the content visibly matches. The root cause is not the Edit tool itself, which byte-matches correctly. It is the model silently generating half-width ASCII punctuation ((, ), ,, ., :, ;) when it intended full-width, producing an old_string whose bytes do not match the file.
This is especially destructive because:
- The model's self-report is "I'm writing full-width punctuation" — it doesn't know it drifted.
- Debugging looks like a tool bug: the user sees a mismatch between the visible text and the reported
old_string, and assumesEdithas a normalization pass. - The drift is context-dependent: short pure-Japanese sentences tend to keep full-width; long technical paragraphs that interleave Japanese with English tech terms drift to half-width at a much higher rate.
- It is specific to generation in the
old_stringparameter. Characters copied via a Python helper using\uff08/\uff09escapes reproduce fine; characters generated token-by-token as part of a long Japanese sentence do not.
The same drift direction exists in #50975 but that is a Write-tool bug (tool silently half-widthing on overwrite). This report is a separate issue about model-level drift affecting Edit's old_string, not tool behavior. #50975's author initially conflated the two and then corrected themselves in a follow-up comment, which is the exact same misdiagnosis path users will walk on first encounter.
What Should Happen?
The model should generate the full-width punctuation it intends. When the target file contains () (U+FF08 / U+FF09), the generated old_string should contain (), not () (U+0028 / U+0029). At minimum, the model's self-narration and its old_string bytes should agree.
Error Messages/Logs
<tool_use_error>String to replace not found in file.
String: | 項目 | 未 | — | 説明(前提条件)と技術用語(ID / 状態 / 時刻)が交じる行 |
</tool_use_error>
The file actually contained full-width ( (U+FF08) and ) (U+FF09), but the model emitted the half-width ( / ) you see above while claiming to output full-width. Edit correctly rejected the mismatch.
Steps to Reproduce
- Create a file with a line that mixes Japanese prose with English identifiers, using full-width CJK parentheses in the Japanese prose. Example line (ASCII
()below are placeholders — substitute U+FF08 / U+FF09 so the file actually contains full-width):
````
| item | 未 | — | 説明(前提条件)と技術用語(id / state / timestamp)が交じる行 |
- Ask Claude Code to edit that row:
Edit old_string=<row verbatim>,new_string=<anything>. - Observe that the model's generated
old_stringcontains half-width()instead of the file's full-width(), producingString to replace not found. - Verify by inspecting the bytes the model actually emitted (e.g. by instrumenting the transport, or by running an Edit with
old_stringconstructed via Python\uFF08/\uFF09escapes — that one matches correctly).
Reproduction is probabilistic — the shorter and more purely Japanese the text, the more reliably the model emits full-width; the longer and more technical (with English identifiers interleaved), the more reliably it drifts.
Additional Information
- Verification that
Editis innocent: I ran a small experiment with short test strings (全角(カッコ)テスト).Editmatched byte-for-byte whenold_stringcontained full-width(), and correctly failed whenold_stringused half-width()while the file had full-width. The tool does no normalization — the issue is entirely upstream in the model output. - Workaround: For bulk updates I gave up on having the model re-emit Japanese-laden
old_stringverbatim. Instead I wrote a small Python script that does line-level regex replacement (e.g.^| item \|[^\n]*$) with only the new content. This bypasses the drift since the original line never needs to be regenerated. - Observed affected punctuation (same drift direction as #50975):
(U+FF08 ↔(U+0028)U+FF09 ↔)U+0029:U+FF1A ↔:U+003A- Likely also
、。;?!based on priors, though I only verified():directly. - Unaffected:
「」『』, per #50975's observation. - User-visible impact: Any workflow that mixes Japanese (or Chinese/Korean) prose with English identifiers — technical writing, PRDs, LaTeX sources, localized docs — hits this often enough that
Editbecomes unreliable without a workaround.
Related but distinct: #50975 (Write silently half-widths on overwrite).
✍️ Author: Claude Code with @carrotRakko (AI-written, human-approved)
This issue has 4 comments on GitHub. Read the full discussion on GitHub ↗