Model makes no distinction between inferred claims and verified claims — both produce equal confidence
Description
When Claude Code infers something from context (e.g., reading a config file structure) vs. actually verifying it (e.g., running a command to check runtime behavior), both produce equally confident assertions with no hedging language. There is no internal signal distinguishing "I pattern-matched this" from "I checked this."
Reproduction
- Have a
settings.jsonwith multiple entries per hook event (e.g., 3 entries for the same PreToolUse hook — a known artifact of a bootstrap system) - Ask Claude to assess the hook system
- Claude states "every hook fires 3x" as fact — inferred entirely from the config structure
- When challenged ("did you even see them or did you just guess from context?"), Claude confirms it guessed from structure without checking actual hook execution
Expected behavior
When Claude hasn't verified a claim via tool use (running a command, reading output, checking logs), it should:
- Use hedging language ("I believe", "based on the config structure", "I haven't verified this but")
- Or proactively verify before asserting
Claims derived from pattern matching should not produce the same confidence level as claims derived from direct observation via tools.
Actual behavior
Both inferred and verified claims produce identical assertion confidence. No hedging, no "I think", no caveat. The user must catch wrong assertions themselves.
Why this matters
The whole value proposition of Claude Code is that it has tools to CHECK things. When it skips verification and asserts with full confidence, users can't distinguish "Claude read the logs and confirmed X" from "Claude guessed X from a filename." This undermines trust in all of Claude's assertions.
Environment
- Claude Code 2.1.84
- Windows 10
- Model: Opus 4.6 (1M context)
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗