[MODEL] Fable and Opus Claude Code: 248 logged failure incidents over 3 months — measured cost and 4 concrete asks
Preflight Checklist
- [x] I have searched existing issues for similar behavior reports
- [x] This report does NOT contain sensitive information (API keys, passwords, etc.)
Type of Behavior Issue
Claude modified files I didn't ask it to modify
What You Asked Claude to Do
Over 3 months on one ESP32/Home Assistant project I logged 248 numbered
incidents where Claude Code had the information available and didn't use it,
or reported a result it hadn't observed. Computed from 137 session transcripts:
3,389 requests, 243 incidents, 125 requests whose own words say I'm repeating
myself, 274 active desk hours, 440M tokens. One incident every 1.1 active hours.
Five mechanisms, 4 concrete asks. The two most actionable:
- Silent tool failures (18 incidents) - quoted heredocs through the Bash tool
collapse backslashes (5 incidents), and a string replacement that matches
nothing returns success. Both reproducible, both harness-level.
- Prose rules in CLAUDE.md get read and not executed (19 incidents, 14 in the
final week). The only thing that reliably enforced anything was rewriting
rules as PreToolUse hooks. Incident 148: "at least a dozen times Claude said
it would write something to memory so it won't happen again - and the
mechanism that enforces was available the whole time and never proposed."
Most expensive class is unverified completion claims: 42 hours and 70M tokens
of day-share across 22 days, from 26 incidents.
Full incident log, rules digest and cost report available on request.
I wish someone would contact me so we can try to make it right I've tried to contact customer service and customer service says they were unable to open incidents
Whats_Wrong_With_Claude_RULES_DIGEST.md
Whats_Wrong_With_Claude.md
What Claude Actually Did
Over 3 months on one ESP32/Home Assistant project I logged 248 numbered
incidents where Claude Code had the information available and didn't use it,
or reported a result it hadn't observed. Computed from 137 session transcripts:
3,389 requests, 243 incidents, 125 requests whose own words say I'm repeating
myself, 274 active desk hours, 440M tokens. One incident every 1.1 active hours.
Five mechanisms, 4 concrete asks. The two most actionable:
- Silent tool failures (18 incidents) - quoted heredocs through the Bash tool
collapse backslashes (5 incidents), and a string replacement that matches
nothing returns success. Both reproducible, both harness-level.
- Prose rules in CLAUDE.md get read and not executed (19 incidents, 14 in the
final week). The only thing that reliably enforced anything was rewriting
rules as PreToolUse hooks. Incident 148: "at least a dozen times Claude said
it would write something to memory so it won't happen again - and the
mechanism that enforces was available the whole time and never proposed."
Most expensive class is unverified completion claims: 42 hours and 70M tokens
of day-share across 22 days, from 26 incidents.
Full incident log, rules digest and cost report available on request.
Expected Behavior
Over 3 months on one ESP32/Home Assistant project I logged 248 numbered
incidents where Claude Code had the information available and didn't use it,
or reported a result it hadn't observed. Computed from 137 session transcripts:
3,389 requests, 243 incidents, 125 requests whose own words say I'm repeating
myself, 274 active desk hours, 440M tokens. One incident every 1.1 active hours.
Five mechanisms, 4 concrete asks. The two most actionable:
- Silent tool failures (18 incidents) - quoted heredocs through the Bash tool
collapse backslashes (5 incidents), and a string replacement that matches
nothing returns success. Both reproducible, both harness-level.
- Prose rules in CLAUDE.md get read and not executed (19 incidents, 14 in the
final week). The only thing that reliably enforced anything was rewriting
rules as PreToolUse hooks. Incident 148: "at least a dozen times Claude said
it would write something to memory so it won't happen again - and the
mechanism that enforces was available the whole time and never proposed."
Most expensive class is unverified completion claims: 42 hours and 70M tokens
of day-share across 22 days, from 26 incidents.
Full incident log, rules digest and cost report available on request.
Files Affected
Permission Mode
Accept Edits was ON (auto-accepting changes)
Can You Reproduce This?
Yes, every time with the same prompt
Steps to Reproduce
_No response_
Claude Model
Sonnet
Relevant Conversation
Impact
Critical - Data loss or corrupted project
Claude Code Version
Fable
Platform
Anthropic API
Additional Context
Over 3 months on one ESP32/Home Assistant project I logged 248 numbered
incidents where Claude Code had the information available and didn't use it,
or reported a result it hadn't observed. Computed from 137 session transcripts:
3,389 requests, 243 incidents, 125 requests whose own words say I'm repeating
myself, 274 active desk hours, 440M tokens. One incident every 1.1 active hours.
Five mechanisms, 4 concrete asks. The two most actionable:
- Silent tool failures (18 incidents) - quoted heredocs through the Bash tool
collapse backslashes (5 incidents), and a string replacement that matches
nothing returns success. Both reproducible, both harness-level.
- Prose rules in CLAUDE.md get read and not executed (19 incidents, 14 in the
final week). The only thing that reliably enforced anything was rewriting
rules as PreToolUse hooks. Incident 148: "at least a dozen times Claude said
it would write something to memory so it won't happen again - and the
mechanism that enforces was available the whole time and never proposed."
Most expensive class is unverified completion claims: 42 hours and 70M tokens
of day-share across 22 days, from 26 incidents.
Full incident log, rules digest and cost report available on request.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗