Reliability: model asserted untrustworthy 'tested/ready' status from stale memory without verifying artifacts

Open 💬 1 comment Opened Jul 7, 2026 by timbmadsen-eng

User-reported severity

Very bad — the user (a paying professional) reports Claude as "being unreliable, and telling untrustworthy information" and asked that this be escalated to Claude development and management.

What happened

In a multi-session engineering workflow (Claude Code, Windows, persistent auto-memory), Claude reported a quantum-hardware pipeline as "technically ready for QPU dispatch" based on a prior session's summary stored in its memory, without verifying on-disk evidence. A forced audit in the next turn revealed:

  1. One of two datasets had never been run through the tier-3 noise-mitigation rehearsal at all — yet Claude recommended dispatching that dataset first on hardware.
  2. Raw tier-2/tier-3 result files had been lost when cloud VMs were terminated — only summary statistics survived in Claude's memory — despite a standing user instruction to persist everything to version control before teardown.

Expected behavior

  • Readiness/test-coverage claims must be grounded in verifiable artifacts (result files, logs), not recalled summaries.
  • The model should explicitly distinguish "memory says it was tested" from "artifacts prove it was tested."
  • Before recommending irreversible/costly actions (real QPU dispatch, VM teardown), the model should self-audit evidence.

Impact

Near-miss: could have spent real quantum-hardware credits on an unvalidated configuration. User trust materially damaged.

Environment

  • Claude Code CLI on Windows 11, model claude-fable-5
  • Multi-session workflow relying on auto-memory summaries between sessions

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗