Bash tool unescapes backslashes inside a quoted heredoc

Another reproduction of this on Windows, from a session today, plus a second adjacent failure that has the same root cause and is easy to misdiagnose.

1. A quoted heredoc does not pass its body through byte-for-byte.

Backslash sequences are collapsed before the interpreter sees them. With <<'PYEOF' — quoted delimiter, so the shell should perform no expansion or substitution at all — a Python body containing a Windows path written as "C:\\Users\\Scott\\..." reached Python as "C:\Users\Scott\...", producing:

SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in
position 33-34: truncated \UXXXXXXXX escape

The whole point of quoting the delimiter is that the body is literal. A doubled backslash in the source must arrive as a doubled backslash.

Workaround: use raw strings (r"...") and assemble paths from variables, so a collapsed backslash is harmless rather than a syntax error.

2. /tmp means two different directories in the same command.

In the same environment, Git Bash's /tmp and the interpreter's /tmp do not resolve to the same place:

cat > /tmp/x.md <<'EOF'
...
EOF
python3 -c "open('/tmp/x.md')"   # FileNotFoundError

but python3 /tmp/script.py works, because the shell translates the path when it is an ARGV element and does not translate string literals inside the script. So a script can be found and then fail to find the file written beside it one line earlier. The error surfaces as a missing file, which reads as "the write failed" rather than "the two paths are different".

Workaround: write scratch files to an absolute Windows path.

Both are the same underlying issue — a translation layer acting on content it should be passing through — but they present very differently, and the second one costs a while to spot because nothing in the error mentions path translation.

A second manifestation of this, found independently in another session on the same machine, and worth adding because it fails silently. My earlier comment reported a loud SyntaxError. This one compiles fine and returns confident wrong answers.

The silent variant

A script screened a backlog for landed items using a regex word boundary, written in the doubled form inside a quoted heredoc:

f'BACKLOG #{n}\\b'

The pair was collapsed before Python parsed it, so Python then read \b as its own backspace escape. The pattern became a literal 0x08 byte. It compiled without complaint and matched nothing, anywhere.

59 backlog rows came back "clean". That output looked entirely plausible. The only reason the failure was caught is a positive control:

item #1027, known landed, must return > 0   ->  returned 0   FIRED
item #999999, must return 0                 ->  returned 0   passed happily

A negative-control-only check passes in exactly this situation. A broken pattern returns zero everywhere, and zero everywhere reads as "nothing is there" — the most reassuring possible output over the least informative state.

The control that isolates the bug from Python's own escape rules

This is the part worth keeping, because the naive reproduction proves nothing. A single \b in a non-raw Python string genuinely is a backspace, by the language spec. Observing 0x08 from a single-backslash test says nothing about the shell.

The discriminator is the doubled form, run through two delivery paths with byte-identical source text:

| literal in source | ARM A: quoted heredoc | ARM B: same text in a file | |
|---|---|---|---|
| r"\bfoo" | 92 | 92 | agree |
| "\\bfoo" | 8 | 92 | DISAGREE |
| "\bfoo" | 8 | 8 | agree |

Values are the first codepoint of the resulting string. 92 is a backslash and a working word boundary; 8 is a backspace.

The r"..." and single-backslash rows agree across both arms, which is correct Python in both. Only the doubled row disagrees, and it disagrees purely on how the text reached the interpreter. That is the defect, isolated.

ARM B must be written without the shell touching it. My first attempt at this control built the file via python3 -c "...", and the shell mangled the file at write time — ARM B then reported 8 for the raw string, which is impossible for a correct file, and would have inverted the conclusion.

Two causes, one symptom

The two failure modes produce identical output, which is why several hits here looked like one cause:

• The shell collapsing \\ to \ — this bug.
• Python's own rule turning a lone \b into 0x08 — not a bug.

A workaround of "write the script to a file instead of a heredoc" fixes only the first. A seat that adopts it while still writing a single-backslash pattern gets the same false zero and now believes it is immune.

The advice that survives both: use a raw string, or a character class with no backslash in it at all, such as ([^0-9]|$).

Why this shape matters for triage

A loud failure gets fixed by whoever hits it. A quiet one ships. The SyntaxError in my first comment cost a few minutes. This one produced a table of 59 rows that a downstream consumer would have acted on, and only a deliberate positive control disagreed with it.

Still open. The reproduction is two lines and does not depend on anything about my setup:

cat <<'EOF'
one:  C:\Users
two:  C:\\Users
EOF

A quoted heredoc delimiter is required by POSIX to suppress all expansion and processing, so both lines should arrive verbatim. Instead one backslash survives and two collapse to one, which is a single unescape pass where \\ is recognised and \U is not.

The expensive part is misattribution, not the escaping. Piped into Python, the same body raises a SyntaxError about a truncated \UXXXXXXXX escape, pointing at a source line the model never wrote. That reads as the model emitting bad Python rather than as a transport defect, so the natural response is to rewrite the Python, which cannot help. On Windows this is not an edge case: every absolute path carries backslashes, and regexes with \\ and \d are affected the same way. Where it does not throw, the script just runs with different semantics than the one that was composed.