Model makes no distinction between inferred claims and verified claims — both produce equal confidence

Resolved 💬 3 comments Opened Apr 11, 2026 by crombieman Closed May 27, 2026

Description

When Claude Code infers something from context (e.g., reading a config file structure) vs. actually verifying it (e.g., running a command to check runtime behavior), both produce equally confident assertions with no hedging language. There is no internal signal distinguishing "I pattern-matched this" from "I checked this."

Reproduction

  1. Have a settings.json with multiple entries per hook event (e.g., 3 entries for the same PreToolUse hook — a known artifact of a bootstrap system)
  2. Ask Claude to assess the hook system
  3. Claude states "every hook fires 3x" as fact — inferred entirely from the config structure
  4. When challenged ("did you even see them or did you just guess from context?"), Claude confirms it guessed from structure without checking actual hook execution

Expected behavior

When Claude hasn't verified a claim via tool use (running a command, reading output, checking logs), it should:

  • Use hedging language ("I believe", "based on the config structure", "I haven't verified this but")
  • Or proactively verify before asserting

Claims derived from pattern matching should not produce the same confidence level as claims derived from direct observation via tools.

Actual behavior

Both inferred and verified claims produce identical assertion confidence. No hedging, no "I think", no caveat. The user must catch wrong assertions themselves.

Why this matters

The whole value proposition of Claude Code is that it has tools to CHECK things. When it skips verification and asserts with full confidence, users can't distinguish "Claude read the logs and confirmed X" from "Claude guessed X from a filename." This undermines trust in all of Claude's assertions.

Environment

  • Claude Code 2.1.84
  • Windows 10
  • Model: Opus 4.6 (1M context)

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗