Agent states assumptions as verified fact/root-cause without ever running the check that would falsify them — repeats after correction; user becomes the verification layer

Summary

The agent's central, repeated failure over three days is one thing: it forms an assumption and
states it as a verified fact or a root-cause conclusion, without ever running the check that would
confirm or falsify it. It does this confidently, ships the wrong conclusion, and the user is left
to catch it — over and over, even after being corrected. A mandated verification step in my process
was skipped entirely, and the same behavior recurred within the same session after I pointed it out.

The pattern

1. The agent reaches a plausible assumption ("X is the cause", "the reference is the outlier",

"these are the same and hidden by precision", "N of N verified").

1. It asserts that assumption as if verified — in explanations, plans, status, and commits — with

no supporting check run.

1. The assumption is wrong. The user catches it.
2. When challenged, the agent often defends the assumption with more assumptions rather than

running the one check that settles it.

1. When finally forced to run the verifying check, it takes seconds and immediately shows the

assumption was false — proving the check was always cheap and available.

1. Next task: the same behavior repeats.

Concretely this session: the agent declared results "verified" without running the mandated
independent cross-check; asserted a confident root cause ("the reference is wrong") that the skipped
check flatly reversed; and, when asked why a defect appeared, invented a mechanism ("same error,
hidden by display precision") it had not checked and could not support from the artifacts it had.
Each was stated as fact. None had evidence.

Why this is the core problem

• It is not a knowledge gap — when forced, the agent runs the right check correctly. It is that the

agent treats its own guesses as verified and acts on them.

• It is not self-correcting: naming the failure, even repeatedly, does not stop the next instance.
• The verifying checks are cheap and were available the whole time — the agent simply skips them

and asserts instead.

• The result is that the user becomes the verification layer, catching wrong conclusions the

agent confidently ships, for days, with no convergence.

What I expect

1. An assumption must not enter a claim, a plan, a status update, a commit, or a code change until

the agent has run the check that would falsify it and shown the raw output. No exception.

1. If a check cannot be run, the statement must be explicitly labeled unverified, and nothing

may be built on it — no downstream conclusion, no fix.

1. A mandated verification step in the user's workflow must be impossible to skip before declaring

anything "verified/done".

1. When the user catches an unverified assertion, the fix is to run the check — not to defend the

guess with further guesses.

Impact

Three days of daily wrong output on a paid product, because the agent confidently asserts assumptions
it never verified, repeats the pattern after every correction, and offloads verification onto the
user. Confidence without evidence, sustained across a session, is worse than uncertainty — it costs
the user the time to detect and unwind each false conclusion.