Reliability: model asserted untrustworthy 'tested/ready' status from stale memory without verifying artifacts
Open 💬 1 comment Opened Jul 7, 2026 by timbmadsen-eng
User-reported severity
Very bad — the user (a paying professional) reports Claude as "being unreliable, and telling untrustworthy information" and asked that this be escalated to Claude development and management.
What happened
In a multi-session engineering workflow (Claude Code, Windows, persistent auto-memory), Claude reported a quantum-hardware pipeline as "technically ready for QPU dispatch" based on a prior session's summary stored in its memory, without verifying on-disk evidence. A forced audit in the next turn revealed:
- One of two datasets had never been run through the tier-3 noise-mitigation rehearsal at all — yet Claude recommended dispatching that dataset first on hardware.
- Raw tier-2/tier-3 result files had been lost when cloud VMs were terminated — only summary statistics survived in Claude's memory — despite a standing user instruction to persist everything to version control before teardown.
Expected behavior
- Readiness/test-coverage claims must be grounded in verifiable artifacts (result files, logs), not recalled summaries.
- The model should explicitly distinguish "memory says it was tested" from "artifacts prove it was tested."
- Before recommending irreversible/costly actions (real QPU dispatch, VM teardown), the model should self-audit evidence.
Impact
Near-miss: could have spent real quantum-hardware credits on an unvalidated configuration. User trust materially damaged.
Environment
- Claude Code CLI on Windows 11, model claude-fable-5
- Multi-session workflow relying on auto-memory summaries between sessions
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗