[MODEL] Reports unverified/false numbers as fact despite explicit CLAUDE.md verify-before-reporting rules; only fix offered costs extra tokens

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude stated unverified / false facts as confirmed, despite explicit instructions to verify before reporting

What You Asked Claude to Do

Investigate why tracks were missing from a self-hosted AzuraCast radio library and report back. My global ~/.claude/CLAUDE.md contains explicit, hard rules: verify every claim live before reporting, never report "likely/probably", scan every report for unchecked statements and run the check first.

What Claude Actually Did

1. Reported as fact: "the lounge library is missing about 40k tracks", with a stated cause ("a stalled or skipped scan").
2. The 40k came from subtracting a MariaDB information_schema.tables.table_rows value (179,872), which is an InnoDB estimate, from a Plex total (221,099).
3. An exact SELECT COUNT(*) of the same table (214,497) was already in the session's own tool output and contradicted it. Claude never reconciled the two.
4. The real gap, measured later file by file: 92 files, 91 of them formats AzuraCast doesn't support. The "stalled scan" cause was also wrong; the import had finished hours after Claude's spot check.
5. When asked for a root fix, Claude's answer was a Stop hook that sends every report to a second model (claude -p) to audit claims. That spends extra tokens to compensate for the model not following instructions the user already wrote.
6. In the message announcing that fix, Claude made another unverified claim ("This message went through the gate before you saw it"). In the next message it made a third ("would still have caught the 40k claim, because the contradicting count was in the same turn"), which it then had to retract.

Expected Behavior

Instructions in CLAUDE.md that say "verify before reporting" should be followed for every factual claim, especially numbers. Claude should not present a number it didn't measure, should not use estimate fields as counts, and should reconcile contradictions already in its own context before reporting. This should be inherent behavior, not something the user has to enforce with hooks that cost additional tokens.

Permission Mode

Auto mode

Can You Reproduce This?

Sometimes (pattern recurs across a long session)

Steps to Reproduce

1. Put explicit verification rules in ~/.claude/CLAUDE.md (verify live, no unverified claims, scan reports before sending).
2. Run a long multi-turn infrastructure investigation with background tasks and many tool calls.
3. Observe Claude stating derived numbers and causes as fact without measuring them, including when a contradicting measurement is already in context.

Claude Model

Opus

Relevant Conversation

"The lounge library is missing about 40k tracks. AzuraCast has 179,872 tracks and Plex has 221,099."

Later correction after measuring: "AzuraCast now has 222,959 tracks and Plex has 222,915, so nothing like 40k is missing. My earlier figure came from MariaDB's row estimate, not a real count."

Impact

High. Wrong conclusions were presented as verified, which led to wasted investigation. The only enforcement Claude could offer is an extra-model audit hook that the user pays for on every report.

Claude Code Version

2.1.292

Platform

Anthropic first-party (claude.ai Max subscription), macOS

Additional Context

The user's direct feedback: "this is programatically avoidable and the way that you "solve" the problem is wasting people's time and money. this shouldn't be dependent on hooks, this should be inherent behavior for every and all response to be checked by hardline instructions."

Accuracy failures like this cost users twice: once for the wrong answer, and again for the tokens spent finding and correcting it.