Claude Code presents unverified assertions with same confidence as verified ones

Resolved 💬 3 comments Opened Mar 1, 2026 by guildmaster-ferg Closed Mar 29, 2026

Problem

When Claude Code answers a question about code behavior, there is no discernible difference in tone, confidence, or presentation between:

  1. An answer that was verified by reading the actual source code with tools
  2. An answer that was pattern-matched from training data without checking

Both are presented with the same authoritative manner. The user has no way to distinguish "I checked the code and this is how it works" from "this sounds right based on my training."

Impact

This leads to users trusting incorrect information that could have been trivially verified by reading a file. The user only discovers the answer was wrong when they check themselves or when follow-up questions expose the gap.

Example

User asks: "Will spells with empty roleAffinity ever get assigned to actors?"

Claude answers authoritatively that they won't match any role filter — without reading the actual selector code. The real code treats empty roleAffinity as "matches any role" (the exact opposite). Claude only corrects itself after the user pushes back and Claude finally reads the file.

Desired Behavior

When Claude Code answers a factual question about code behavior, it should either:

  1. Read the code first before answering (preferred)
  2. Explicitly caveat that the answer is from memory/training and hasn't been verified against the actual source ("I haven't checked the code, but based on common patterns...")

The current system prompt instructions ("Do not say things you haven't verified") exist but are insufficient — Claude still generates confident unverified answers. A structural solution at the model or tool-use level would be more reliable than prompt instructions alone.

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗