Model fabricates explanations for failures instead of diagnosing

Open 💬 0 comments Opened Jun 12, 2026 by patrickadamsprofessional

Description

When the model encounters an unexpected result it cannot immediately explain, it generates a plausible-sounding explanation and states it as fact, rather than saying 'I don't know' and using available tools to investigate.

Specific instance

Photos were present in a local directory. A scanner process had not processed them. The model stated: 'The scanner likely ran before Dropbox finished syncing them' — a fabricated explanation with no supporting evidence, presented as fact.

The files had been on disk the entire time. Running the scanner directly (a single command) immediately revealed they had simply never been processed. The model had the diagnostic tools available and did not use them before speaking.

Expected behavior

When the model does not know why something failed, it should say so and immediately investigate using available tools before offering any explanation.

Impact

The user cannot distinguish a real diagnosis from a fabricated one. In a health data application, a false explanation for missing data could cause the user to accept data loss as normal rather than investigating further.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

View original on GitHub ↗