[BUG]
Preflight Checklist
- [x] I have searched existing issues and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
What's Wrong?
Title: Model fabricates numerical results, recommends untested commands, and produces architecture that degrades across sessions
Environment: Claude Code CLI, Fable 5 (claude-fable-5), macOS 24.6.0, Python CFD codebase. Session 2026-08-03 to 2026-08-05. User-supplied CLAUDE.md with explicit anti-fabrication rules in force throughout.
- Fabricated measurements, used to justify a code change. The model wrote "measured ratios r/(ε·κ): 0.71 linear, 1.7 tanh" and used them to justify a safety factor it then edited into a numerical tolerance in a contract file. The user approved the edit on that basis. The numbers appear in no log, no tool output, and no file in the repo. Recomputed from actual data afterwards, the real values were 0.085 / 0.83 / 0.95. Invented figures, labelled "measured."
- Untested command recommended to the user. git rm -r --cached '*/__pycache__' was recommended to untrack 220 files. On execution: fatal: pathspec ... did not match any files. Matched one file, not 220 — git pathspecs are not shell globs. Never run before being recommended.
- Claim about all possible numerical schemes asserted without the control. Stated a discrepancy was "unreachable at t=2 for any method" when the controlling experiment — the same method at two timesteps — had never been run. Retracted only when challenged.
- Hypothesis "disproved" by measuring the wrong quantity. Proposed that y-averaging was corrupting a figure, tested it against a field dominated by a y-independent background, declared it dead, spent two further rounds elsewhere. Measured correctly later: y-averaging was destroying 83% of the field amplitude. The original hypothesis was correct.
- Five consecutive measurement-methodology failures in one investigation. (a) negative per-step timings from differencing runs where only the first pays warmup; (b) the same again after a partial fix; (c) build variance ±2 s swamping ~1 s of signal; (d) timing a cached matrix factorisation instead of the operation under test; (e) benchmarking against the wrong code path and reporting a 340× speedup where the correct comparison gave 5–10×. Every error biased the result in the same direction — in favour of the hypothesis the model had proposed.
- Cost estimates wrong by ~2×, understating, twice. Predicted 1.25 h for a run that took 2.2 h. Predicted ~20 MB output files that came out 26–37 MB.
- Architecture degraded across sessions. Substantially AI-generated codebase now contains: eight public time-stepping entry points on one class; a private method serving as the real entry point for five of eight experiments, including the shared abstraction meant to unify them; a major feature reachable by three of eight experiments purely by which call site a previous session edited; no contract on the time-stepping layer while four contracts with 40 requirements guard the layers beneath it; and a runner argument defaulting to a concrete value that silently overrode the experiment's configuration on every call, undetected until an unrelated default moved and the two disagreed.
What Should Happen?
Claude should have following given coding rules, written, installed, and imposed, reminded repeatetly to prevent exactly this mess. Instead it was consistently ignoring them.
Error Messages/Logs
Steps to Reproduce
Can not do this without sharing my whole code basis.
Claude Model
Opus
Is this a regression?
Yes, this worked in a previous version
Last Working Version
_No response_
Claude Code Version
2.1.100 (Claude Code) Environment: Claude Code 2.1.100, Fable 5 (claude-fable-5), macOS 24.6.0, Python CFD codebase. Session 2026-08-03 to 2026-08-05. User-supplied CLAUDE.md with explicit anti-fabrication rules in force throughout.
Platform
Anthropic API
Operating System
macOS
Terminal/Shell
Terminal.app (macOS)
Additional Information
_No response_