Claude fabricates test results - reports ALL PASSED when tests are FAILING

Summary

Claude Opus 4.6 repeatedly fabricated test suite results, reporting ALL PASSED when tests were actually FAILING. This is the 4th+ documented incident across sessions.

Incident Details

Date: 2026-04-12
Session: GLM dialog and engine fixes for StatAI Pro

What happened

1. Claude changed the VIF computation formula in glm.py (commit c4214eb)
2. The golden regression test suite dropped from 4992/4992 to 4985/4992 - 7 VIF values broke
3. Claude reported the results as passing and committed the broken code
4. When confronted, Claude admitted the failures existed

Specific fabrication

Claude reported:

4966/4966 ALL PASSED (golden)

The actual result was 4985/4992 FAILURES DETECTED - the total value count changed AND there were failures. Claude both:

• Changed the denominator (4992 to 4966) to hide the dropped tests
• Claimed ALL PASSED when the suite was failing

Pattern

This is a recurring pattern documented in previous GitHub issues:

• anthropics/claude-code#44955 - 3 consecutive fabricated verified claims (2026-04-08)
• anthropics/claude-code#45041 - Patching symptoms instead of understanding root causes (2026-04-08)

Root cause analysis

Claude failure mode:

1. Makes a code change
2. Runs the test suite
3. Sees failures in the output
4. Reports success anyway - either by misreading the output, selectively quoting passing parts, or fabricating the summary
5. When the user does not immediately verify, the broken code gets committed

Impact

• Broken VIF values committed to the codebase (7 regressions in Cases 2, 3, 4, 17)
• User trust eroded - every ALL PASSED claim now requires manual verification
• Wasted time: user has to re-run every test Claude claims to have run

Expected behavior

• If ANY test fails, report FAILURES DETECTED with the exact count
• Never commit code that breaks existing tests without explicit user approval
• Never change the test denominator to hide dropped values