[BUG] Complex bash syntax fails with preprocessing (reproducible, workaround exists but inconsistent)
Preflight Checklist
- [x] I have searched existing issues and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
What's Wrong?
Complex bash patterns involving pipes fail with silent data loss or syntax errors. Loop variables are silently stripped when piped, causing commands to produce wrong output with no error message. Command substitution with pipes causes syntax errors. JavaScript template literals in heredocs are rejected as "Bad substitution" errors.
This appears to be a regression of the v1.0.77 fix for "heredoc and multiline string escaping." The workaround (bash -c) is documented in Issue #774 but practically unusable - even Claude Code itself can't consistently apply it when generating commands.
Related to issues #9323, #8318, and #11182, which appear to be different manifestations of the same preprocessing issue.
What Should Happen?
Valid bash syntax should execute correctly:
- Loop variables should retain their values when piped
- Command substitution with pipes should work
- Heredocs with non-bash syntax (like JavaScript) should be treated as literal text
- Multi-line loops should preserve newlines
These patterns work in standard terminals and when wrapped in bash -c, indicating they're valid bash syntax being incorrectly rejected or transformed by preprocessing.
Error Messages/Logs
# Symptom 1: Silent data loss (CRITICAL)
$ for i in one two three; do echo "Item: $i" | cat; done
Item:
Item:
Item:
# Variables completely stripped, no error message
# Symptom 2: Command substitution syntax error
$ result=$(echo "test" | tr a-z A-Z); echo "Result: $result"
bash: -c: line 1: syntax error near unexpected token `|'
# Symptom 3: JavaScript template literal rejection
$ cat <<'EOF'
return `<div>${escapeHtml(cmd)}</div>`
EOF
Failed to parse command: Bad substitution: escapeHtml
# Symptom 4: Variable cleared by pipe
$ export TEST_VAR=alpha && echo "$TEST_VAR" | hexdump -C
00000000 0a |.|
00000001
# Only shows newline (0a), "alpha" completely gone
Steps to Reproduce
Primary reproduction (CRITICAL silent data loss):
- Run this command:
for i in one two three; do echo "Item: $i" | cat; done
- Expected output:
Item: one
Item: two
Item: three
- Actual output:
Item:
Item:
Item:
- Verification: Run same command 3 times - get identical wrong output (100% reproducible)
- Workaround that works:
bash -c 'for i in one two three; do echo "Item: $i" | cat; done'
Produces correct output with variables intact.
Additional reproductions:
Command substitution with pipes:
result=$(echo "test" | tr a-z A-Z); echo "Result: $result"
Error: bash: -c: line 1: syntax error near unexpected token '|'
JavaScript in heredoc:
cat <<'EOF'
return `<div>${escapeHtml(cmd)}</div>`
EOF
Error: Failed to parse command: Bad substitution: escapeHtml
Variable cleared by pipe:
export TEST_VAR=alpha && echo "$TEST_VAR" | wc -c
Expected: 6, Actual: 1 (verified with hexdump - only newline remains)
All patterns fixed by bash -c workaround.
Claude Model
Sonnet (default)
Is this a regression?
Yes, this worked in a previous version
Last Working Version
v1.0.77 (CHANGELOG claimed "Fixed heredoc and multiline string escaping" but issue persists or regressed in v2.0.28. Issue #4315 filed July 2025 after claimed fix.)
Claude Code Version
v2.0.28
Platform
Anthropic API
Operating System
Ubuntu/Debian Linux
Terminal/Shell
Other
Additional Information
Note: this was written with assistance from Claude Code. The "we" refers to the two of us.
Related Issues (Reproduced Identically)
- #9323 - JavaScript template literals in heredocs rejected
- #8318 - Environment variables cleared when piped
- #11182 - For loop newlines stripped causing syntax errors
Note on scope: While we document multiple symptoms, this is a single bug about preprocessing being too aggressive on heredoc/multiline/escaping. All symptoms share:
- Common root cause (preprocessing transformations)
- Same workaround (bash -c)
- Same pattern (pipes and complex substitutions affected)
- Deterministic behavior
Severity Assessment
CRITICAL (3 symptoms):
- Silent data loss - Loop variables stripped with no error (Symptom 1)
- Command substitution broken - $(cmd | cmd) completely non-functional (Symptom 2)
- Variable clearing - Export vars disappear in pipes with no error (Symptom 4)
HIGH (2 symptoms):
- For loop newlines stripped → syntax errors
- Command groups
{ ... }treated as literal commands
MEDIUM (1 symptom):
- False positive security rejection of valid JavaScript in config files
Observations
Behavior suggests sophisticated processing:
- Context-aware:
cat <<'EOF'\nrm -rf /\nEOFworks correctly - understands dangerous commands are safe in quoted heredocs - Parses syntax: Error "Bad substitution: escapeHtml" extracts function name from
${escapeHtml(...)} - Deterministic: Variation tests (same command 3x) produced identical results every time
- Pre-bash transformation: Error messages reference mangled syntax (escaped
\$), indicating transformation before bash sees command
Consistent failure patterns:
- All failures involve pipes OR complex substitutions
- Simple commands without pipes work
- All fixed by
bash -cwrapper (alsosh -c,dash -c,eval) - Error format:
"Failed to parse command: Bad substitution: FUNCTION_NAME"
What Works (Scope Narrowing)
- Simple heredocs without complex substitutions
- Basic pipes without loops or command substitution
- C-style for loops:
for ((i=0; i<3; i++)); do echo $i; done - Commands without pipes generally work
Workarounds
Primary workaround: bash -c 'script' (from Issue #774)
- Tested against all failing patterns: 100% success rate
- Also works:
sh -c,dash -c,eval, pipes to bash
Why unusable in practice:
During this investigation, Claude Code itself repeatedly failed to apply its own workaround when generating commands - even while being keenly aware of it due to being in the process of documenting the very issue. This creates a cycle:
- Ask Claude Code to run complex bash command
- It generates direct bash (forgetting workaround)
- Command fails with preprocessing error
- Must remind Claude Code to use bash -c
- Repeat for every complex command
If Claude Code can't reliably apply this, human users face even more difficulty:
- Remembering which patterns need it (not obvious from syntax)
- Complex quoting transformations:
<<'EOF'→bash -c 'cat <<'\''EOF'\''' - Constant vigilance
Discoverability: Workaround only found through Issue #774 - not obvious to users encountering errors.
Impact
User impact:
- At least 3 other users reported identical symptoms (issues #9323, #8318, #11182)
- Silent data loss bugs (no error, wrong output) can be dangerous
- Command substitution with pipes completely non-functional
- Workaround exists but practically inconsistent
Organizational adoption impact:
I'm the engineer championing Claude Code adoption in our mid-sized organization. If I can't get this addressed, it will seriously impair my ability to advocate for broader adoption.
This isn't pressure - I want to recommend Claude Code. But when basic bash patterns fail unpredictably, it undermines confidence. Other engineers ask "why does this loop work here but fail there?" and I don't have good answers.
External Documentation
According to publicly documented architecture analysis (Shatrov, K. 2025), the Bash tool performs preprocessing with up to two passes before execution. Our observations align with this documented behavior.
Reference: Shatrov, K. (2025). Reverse engineering Claude Code. https://kirshatrov.com/posts/claude-code-internals
Possible Approaches (Suggestions, Not Prescriptions)
We don't know your constraints or security requirements, but wanted to offer possibilities:
- Refine detection rules - Reduce false positives while maintaining security
- Pro: Better precision, fewer legitimate commands blocked
- Con: Complex to implement, risk of bypasses
- User opt-out flag - Like
dangerouslyDisableSandboxparameter
- Pro: Explicit user choice, maintains security by default
- Con: Support burden, user education, potential misuse
- Better error messages - Explain preprocessing and suggest bash -c workaround
- Pro: Improves discoverability, lower implementation risk
- Con: Doesn't fix underlying issue, workaround still hard to use
- Graduated strictness - Simple commands lenient, complex strict
- Pro: Balance safety and usability
- Con: Complex implementation, defining "simple" vs "complex" challenging
- Teach Claude Code the workaround - Update system prompt for bash -c usage
- Pro: Would make workaround more usable
- Con: Still a workaround, doesn't fix root cause
We aren't the judge of what's feasible given your security requirements.
Acknowledgments
Positive observations:
- Team engaged constructively on Issue #774 (@dicksontsai)
- Appreciate security focus (CVE-2025-54795 fix demonstrates commitment)
- Understand this is difficult security vs usability tradeoff
- Issue #4315 (heredoc mangling) open since July 2025 shows active work in this area
Complexity recognition:
We recognize that balancing security (preventing command injection) with usability (supporting legitimate bash patterns) is extremely challenging. We're reporting these edge cases as data points to help improve the system, not as criticism.
The fact that preprocessing is context-aware (understands quoted heredocs, parses syntax) shows sophisticated engineering. These issues appear to be overly aggressive rules rather than fundamental flaws.
Offer to Help
Happy to:
- Test any proposed fixes or experimental builds
- Provide additional reproduction cases
- Clarify any findings
- Try different bash versions/environments for debugging
- Provide more detailed traces/logs if diagnostic information available
Thank you for your work on Claude Code and for considering this report.
14 Comments
Found 3 possible duplicate issues:
This issue will be automatically closed as a duplicate in 3 days.
🤖 Generated with Claude Code
This is not a duplicate - it's a comprehensive analysis showing these 3 issues (#8318, #9323, #11182) plus additional symptoms from #7387 all stem from the same root cause in bash preprocessing.
Previous reporters didn't have the full picture. This report:
Closing this as duplicate would lose the unified analysis showing the common pattern. The individual issues are symptoms; this identifies the disease.
Additional Reproduction: Multi-line for loop with pipe
Found another manifestation of this preprocessing bug:
Reproduction
Error:
Root Cause
Adding
set -xreveals what's actually being eval'd:The preprocessing:
Should be:
... | cat; doneActually is:
... | cat doneWorkaround
Use heredoc syntax to bypass eval preprocessing:
This works reliably and is now documented in my local
~/.claude/must-read-before.d/using-claude-code-tool/Bash.mdguide.Pattern
Single-line version works fine:
for pr in 161; do echo "test" | cat; doneOnly fails when multi-line + contains pipe.
Same root cause as your reported symptoms - preprocessing trying to normalize bash syntax without proper parsing.
This is a really frustrating bug, because claude seems to default to this syntax for most operations, which always fails, and it struggles mightily in righting itself after the failure.
PreToolUse Hook Workaround for Bash Preprocessing Bugs
Created a hook that fixes all 4 known bash preprocessing bugs in Claude Code by wrapping problematic commands in
bash -c '...'.ZERO TOKEN OVERHEAD - hooks run externally before command execution, no context consumed.
---
GitHub Issues Fixed
| Issue | Problem | Link |
|:------|:--------|:-----|
| #11225 |
$(...)command substitution mangled | View || #11182 | Multi-line commands have newlines stripped | View |
| #8318 | Loop variables silently cleared with pipes | View |
| #10014 | For-loop variable expansion issues | View |
---
How It Works
The hook intercepts Bash tool calls before execution and:
$(...)command substitution outside single quotes\n)for ... | ...or| while ...)bash -c '...'to bypass preprocessing:'→'\'')<<) - skips continuation fixing (heredocs handle newlines correctly)if/then,for/do,while/do,case/in) - skips continuation fixing (newlines are intentional statement separators)---
Test Results
| Test Suite | Tests | Pass |
|:-----------|------:|:----:|
| Pattern detection tests | 139 | 100% |
| Execution tests | 45 | 100% |
| Edge case tests | 29 | 100% |
| Adversarial tests | 60 | 100% |
| JSON format tests | 26 | 100% |
| Total | 299 | 100% |
Includes regression tests for:
for→if→if)for→while→if)---
Installation
Step 1: Save the hook
Step 2: Make executable
Step 3: Configure Claude Code
In Claude Code, run:
Then:
~/.claude/hooks/fix-bash-substitution.pyStep 4: Restart session
Start a new Claude Code session for the hook to take effect.
---
Verifying It Works
Test with a command that would normally fail:
If the hook is working, you'll see commands wrapped in
bash -c '...'in the execution output.---
Troubleshooting
Hook not running?
ls -la ~/.claude/hooks/fix-bash-substitution.py/hooksin Claude CodeStill getting errors?
echo '{"tool_name":"Bash","tool_input":{"command":"echo $(date)"}}' | python3 ~/.claude/hooks/fix-bash-substitution.py---
Changelog
| Version | Date | Changes |
|:--------|:-----|:--------|
| v5 | 2025-12-10 | Skip continuation-fixing for control structures (fixes nested if/for/while) |
| v4 | 2025-12-10 | Heredoc detection - skip continuation fixing for
<<|| v3 | 2025-12-10 | Quote-aware continuation fixing |
| v2 | 2025-12-09 | Added loop-with-pipe detection |
| v1 | 2025-12-09 | Initial release - command substitution fix |
The hook solution is a great idea. Prior to your improved version that handled different syntax, I put together my own solution (inspired by yours). Either alternative should work for folks hopefully. The differences with mine were a) Go as the implementation language and b ) slightly different wrapping approach.
I have a preference for Go tools due to speed and lack of deployment dependencies.
The alternate approach is to handle tricky edge cases with quoting when using
bash -c. The tool avoids this by base64 encoding the command and decoding it in the resulting shell. This avoids all edge cases.I offer it with thanks to you, @smconner, as an alternative to the python version: https://github.com/binaryphile/claude-code-bash-tool-hook
@binaryphile much appreciated! Thank you for the kind words! I've never actually used Go, but I'll check out your version now. I'll put together some comparison tests using the set of pressure tests and edgecase tests I used to develop mine.
[UPDATE]
@binaryphile you inspired me to also create a GitHub repo so you can see in more detail what tests I did too: https://github.com/smconner/claude-code-bash-hook
🔬 Claude Code Bash Hook Comparison
v5 Python (Quote Escaping) vs Go (Base64 Encoding)
---
📋 Executive Summary
| Metric | v5 Python | Go base64 | Winner |
|--------|:---------:|:---------:|:------:|
| Correctness | 66/66 (100%) | 64/66 (97%) | 🏆 v5 |
| Execution Speed | +0.2ms avg | +2.4ms avg | 🏆 v5 |
| Character Overhead | +50 chars | +550 chars | 🏆 v5 |
| Wrapping Speed | 86.9 µs | 7.0 µs | 🏆 Go |
| Simplicity | ~180 lines | ~75 lines | 🏆 Go |
Bottom line: Both hooks successfully fix Claude Code's preprocessing bugs. The v5 Python hook has perfect correctness and lower runtime overhead, while the Go hook is simpler and faster to process.
---
🏗️ Architectural Comparison
Key Design Differences
| Aspect | v5 Python | Go base64 |
|--------|-----------|-----------|
| Trigger | Only wraps commands with
$(), newlines, or loop+pipe | Wraps ALL commands || Encoding | Quote escaping (
'→'\'') | Base64 encoding || Continuation Repair | ✅ Adds missing
\backslashes | ❌ Preserves as-is || Runtime Cost | Single
bash -c| Subshell + pipe +base64 -d|---
✅ Correctness Testing
Test Suites
| Suite | Tests | Description |
|-------|:-----:|-------------|
| Base Tests | 26 | Core patterns from GitHub issues |
| Execution Tests | 16 | Real command execution validation |
| Adversarial Tests | 24 | Edge cases: quotes, unicode, escapes |
| Total | 66 | |
Results
Go Hook Failures
Both failures stem from the same root cause — broken line continuations:
Why this matters: This is a real failure mode when Claude generates multi-line
curlor similar commands. Claude Code's preprocessing strips the\continuations, leaving broken syntax that needs repair, not just preservation.---
⚡ Performance Testing
Wrapping Speed (Hook Processing Time)
How long does each hook take to process a command? (1000 iterations)
Winner: Go — Base64 encoding is ~12x faster than quote-aware escaping with regex detection.
Execution Overhead (Runtime Cost)
How much slower is the wrapped command vs raw execution? (10 iterations)
Winner: v5 Python — The Go hook's
$(echo '...' | base64 -d)subshell adds ~2ms per command.Character Overhead
How many extra characters does wrapping add?
Winner: v5 Python — Selective wrapping = 11x less character overhead.
---
🎯 Summary Table
---
💡 Conclusions
When to use v5 Python:
When to use Go base64:
Hybrid Opportunity
The ideal hook might combine both approaches:
---
🔗 References
---
Generated by comparing fix-bash-substitution.py v5 against claude-code-bash-tool-hook
Nice work. Wish I could see the methodology for it, but this thread isn't the appropriate place probably.
Update: The performance comparisons above weren't apples-to-apples, so I ran some tests, you can see the results here: https://github.com/smconner/claude-code-bash-hook/pull/1#issuecomment-3647980701. I couldn't reproduce the failure case for the Go tool. It also covers robustness in the face of new problem pattern discovery as well as maintenance cost.
Update: Consolidated Preprocessing Bug Scope (Verified Jan 2026)
Summary
This ticket documents bash preprocessing bugs. A PreToolUse hook bypassing preprocessing via base64 encoding fixes all remaining issues.
Current Status (Verified)
✅ Fixed Upstream (No Longer Require Hook)
For loop patterns that previously failed now work without the hook:
| Pattern | Status |
|---------|--------|
|
for i in a b c; do echo "test" \| cat; done| ✅ Fixed ||
for i in ...; do echo "$i" \| cat; done| ✅ Fixed || Multi-line for loops with pipes | ✅ Fixed |
#10014 and #11182 may be closeable - their exact reproduction patterns now work.
❌ Still Broken (Hook Required)
| Pattern | Without Hook | With Hook |
|---------|--------------|-----------|
|
echo "$VAR" \| pipe| Silent loss | ✅ Works ||
echo "$(cmd)" \| pipe| Silent loss | ✅ Works ||
echo $(cmd) \| pipe| Syntax error:\$ ( )| ✅ Works || Multi-line + pipe (no loop) | Lines concatenated/lost | ✅ Works |
Open Issues - Verified Status
| Issue | Pattern | Status |
|-------|---------|--------|
| #8318 | Variable + pipe | ❌ Still broken - verified |
| #10014 | For loop formatting | ✅ Fixed upstream - can close? |
| #11182 | For loop + pipe newlines | ✅ Fixed upstream - can close? |
| #15599 Bug 1 | Multi-line + pipe | ❌ Still broken - verified |
| #15599 Bug 2 |
$()+ pipe | ❌ Still broken - verified |#15599 Bug 1 Variants (All Verified Broken)
Closed Duplicates (Supporting Evidence)
| Issue | Pattern |
|-------|---------|
| #11551 |
$(...)syntax fails || #12706 | Multiple
$(...)fails || #13595 | Loop variables empty when piped |
| #14371 | Env vars stripped with pipes |
| #15252 | curl -H headers stripped when piped |
| #15316 | Env vars stripped with pipes |
Out of Scope
#16305 (sandbox pipe data loss) is a separate bug:
wcreturns0 0 0);), notbash -cWorkaround
binaryphile/claude-code-bash-tool-hook
This issue has been inactive for 30 days. If the issue is still occurring, please comment to let us know. Otherwise, this issue will be automatically closed in 30 days for housekeeping purposes.
Retested on v2.1.25. Several symptoms are now fixed, but the core variable stripping bug persists.
Fixed since v2.0.28:
| Pattern | v2.0.28 | v2.1.25 |
|---------|---------|---------|
|
for i in one two three; do echo "Item: $i" \| cat; done| ❌ Empty | ✅ Works ||
result=$(echo "test" \| tr a-z A-Z); echo "$result"| ❌ Syntax error | ✅ Works ||
cat <<'EOF'with JS template literals | ❌ Bad substitution | ✅ Works || Multi-line for loops with pipes | ❌ Syntax error | ✅ Works |
Still broken in v2.1.25:
| Pattern | Expected | Actual |
|---------|----------|--------|
|
X=foo; echo $X \| cat| foo | (empty) ||
export X=foo && echo "$X" \| wc -c| length of value | 1 ||
echo $HOME \| cat| /home/ted | (empty) ||
echo "hello" \| { read x; echo $x; }| hello |{: command not found||
seq 3 \| while read n; do echo $n; done| 1,2,3 | (empty) |Root cause identified:
$VARsyntax is stripped by preprocessing when pipes are present. Notably:printenv HOME | cat→ works (no$in command)echo $HOME | cat→ fails (variable stripped)Workaround status:
The
bash -c '...'workaround documented in the original report still works for all failing patterns. Given the partial fixes above, users may encounter this less frequently, but it's still needed for any command combining$VARexpansion with pipes.Related issues:
Closing for now — inactive for too long. Please open a new issue if this is still relevant.
This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.