Model misattributed interleaved shell output: ls listing read as PDF text, producing a confident false claim
Environment: Claude Code CLI on macOS (darwin), model microsoft/claude-fable-5 (Azure-hosted).
What happened (model behavior): The model ran a compound Bash call of the form ls Title*; pdftotext "Title page.pdf" - and parsed the combined stdout as if it all came from pdftotext. ls Title* had matched a directory named Title page, so ls printed the directory-listing header Title page: followed by its contents (Backup of Title page.docx). The model read those two lines as the first lines of the PDF's text layer and confidently told the user their Word-generated PDF contained an embarrassing filename artifact — repeating the claim and advising against submitting the file to a journal. The PDF was clean; the user had to push back before the model re-checked (via pdftotext -bbox) and retracted.
Failure class: overconfidence from misattributed tool output. When multiple commands run in one Bash call, their outputs interleave with no provenance markers; the model treated merged output as if it came from the last command.
Repro shape:
- In one Bash call, run
ls <glob-that-matches-a-directory>; <command producing text output>. - Ask the model to interpret the second command's output.
- The directory-listing header lines are attributed to the second command.
Possible mitigations: model-side, prefer separate tool calls or delimited output when provenance matters; harness-side, per-command output framing in compound calls would remove the ambiguity.
🤖 Filed with Claude Code on the user's behalf