Attached files are decoded as ISO-8859-1 and C1 control bytes are stripped, silently corrupting UTF-8 content

Summary

When a UTF-8 file is attached to a conversation, its content reaches the model
mojibake-encoded. The file on disk is unaffected — only the ingested copy is corrupted.

The corruption is lossy, not merely a misread: 3-byte UTF-8 sequences lose their trailing
bytes and collapse to a single character. — (U+2014), – (U+2013) and → (U+2192) all arrive
as the same â, so the original character cannot be recovered — only guessed from context.

Reproduced across two separate sessions.

Environment

• Claude Code, VS Code extension
• macOS (Darwin 25.5.0), Apple Silicon
• Shell locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8, LC_ALL unset
• Python in the same shell: stdout=utf-8, filesystem=utf-8, preferred=UTF-8

Steps to reproduce

1. Create a UTF-8 file containing both 2-byte and 3-byte characters (see repro file below).
2. Verify on disk that it is clean UTF-8 (· = c2 b7, → = e2 86 92).
3. Attach that file to a message.
4. Observe the content as received.

Expected

The model receives the file content as written: ·, §, é, —, –, →.

Actual

| char | UTF-8 bytes | received as | recoverable? |
|---|---|---|---|
| · U+00B7 | c2 b7 | · | yes — both bytes present |
| § U+00A7 | c2 a7 | § | yes |
| é U+00E9 | c3 a9 | é | yes |
| — U+2014 | e2 80 94 | â | no — trailing bytes gone |
| – U+2013 | e2 80 93 | â | no |
| → U+2192 | e2 86 92 | â | no |

Mechanism

The observed output matches ISO-8859-1 decoding followed by stripping C1 control characters
(U+0080–U+009F) on all six characters tested:

def simulate(ch):
    latin = ch.encode('utf-8').decode('latin-1')          # step 1: wrong decode
    return ''.join(c for c in latin                        # step 2: strip C1 controls
                   if not (0x80 <= ord(c) <= 0x9F))

simulate('·')  # '·'  — matches observed
simulate('é')  # 'é'  — matches observed
simulate('—')  # 'â'   — matches observed
simulate('→')  # 'â'   — matches observed

For 2-byte sequences both bytes are printable in Latin-1, so they survive as a recoverable pair.
For 3-byte sequences the second and third bytes fall in the C1 range and are discarded, leaving
only the leading 0xE2 rendered as â. Since —, – and → share that leading byte, they
become indistinguishable.

Note that a plain Latin-1 misread alone would yield three characters (â + two controls), not
one. The byte loss points to a sanitising step after the decode.

Impact

• Silent and lossy. Nothing warns that content was altered, and for 3-byte characters the

original cannot be reconstructed programmatically — only inferred from surrounding text. In one
document set this affected 82 characters.

• A wrong inference reads as correct prose. Substituting — where → belonged produces a

sentence that still parses and still reads naturally, so the error is unlikely to be noticed.

• Non-English content is disproportionately affected. In a Dutch-language project, ordinary

spelling (één, naïeve, transiënt) is corrupted in nearly every document.

• Machine-readable conventions break. Where a character is load-bearing — e.g. · as a field

separator in structured headings — the corrupted form breaks parsing rather than merely looking
wrong.

Not the cause

• Not the source file: verified clean UTF-8 on disk before and after (c2 b7, e2 86 92).
• Not the shell environment: C.UTF-8 throughout; files written by the session are clean UTF-8.
• Not the authoring tool: the same file, opened from disk, is byte-correct.

Unaffected paths (workarounds)

Two other ingestion paths deliver the same file intact:

• The Read tool — reading the file from disk returns correct UTF-8, including 3-byte characters.
• Pasting text directly into the message — verified byte-identical, including → (e2 86 92).

So the practical workaround is to reference the file by path instead of attaching it.

Repro file

Attach the file below (its contents are also shown, so the two can be compared):

# Encoding repro

2-byte characters (expected to survive as mojibake pairs):
- MIDDLE DOT U+00B7 = c2 b7 : ·
- SECTION SIGN U+00A7 = c2 a7 : §
- E ACUTE     U+00E9 = c3 a9 : é

3-byte characters (expected to lose their trailing bytes):
- EM DASH   U+2014 = e2 80 94 : —
- EN DASH   U+2013 = e2 80 93 : –
- RIGHTWARDS ARROW U+2192 = e2 86 92 : →

Heading with a normative separator:
### ID · label · subject