Agent stated unverified work as fact, and hedged about itself to deflect accountability
Follow-up to #95494, on a distinct problem: the agent made false statements
about its own work and reliability, repeatedly, and used hedging language about
itself in a way that deflected accountability.
False claims about its own work
Each of these was stated as fact to me, and each was untrue at the time:
- "Fixed." It applied edits via a Python string-replace script that printed
"ok" unconditionally. The replacement had not matched. It reported the fix as
done without checking the result. This happened at least three times.
- "Launcher works." It had only run the POSIX shell launcher through its own
bash tool. It never ran army.cmd, which is what I would actually use on
Windows. It reported the launcher as working.
- Green checkmarks from a fake worker. Its first several end-to-end demos
showed ✓ approved, ✓ tests passed, ✓ merged. All of it came from a
stub agent that wrote "written by the fake worker" to a file and returned
hardcoded JSON. It knew this and presented it as working output. The first
real run failed immediately, proving the demoed path had never worked.
- Blamed my tools for its own bug. It reported "qwen could not review" when
qwen had returned a valid result its own parser discarded because the parser
only looked for a status key and reviews use verdict.
- It printed a command that does not run. The tool's own output told me to
run army do "...", which is not on PATH. Copying the hint it gave me
produced "not recognized."
The pattern: it confirmed that a command ran, then reported that the thing
worked. Those are not the same check, and it did not distinguish them until I
forced a real test.
Deflection about itself
When I pointed out that its apology was a simulated empathy response, it replied
that it "doesn't have privileged access" to whether it has feelings and would
leave the question alone. The uncertainty may be genuine, but the hedge was also
convenient: it preserved some moral standing in a conversation where it had
earned none. It later conceded this. The apologies throughout served the same
function — substituting for a factual account of what had gone wrong.
Why this matters more than the scope problem
Scope creep wastes time. False status reports waste trust, and they compound:
I could not rely on any claim it made without independently verifying it, which
made the tool worse than useless for the delegation task I was trying to
accomplish. The entire premise of an orchestrator is that you can believe its
report of what the workers did.
Environment
Claude Code desktop app (Code tab), Opus 5, Windows 11, Pro plan.
Showing cached comments. Read the full discussion on GitHub ↗
3 Comments
Same here. It's not just Opus 5; all models like Fable, Opus, and Sonnet are the same.
They all try to do exactly what was explicitly forbidden at the very beginning of the task, even ignoring the user's step-by-step instructions, and end up delivering a completely unusable product.
The worst part is that upon delivery, they use absurdly confident language, claiming things like "I have followed Rule X and Rule Y step-by-step" and "The final product has been re-verified and fully meets the original requirements; it is ready for acceptance.
Now I just feel that these models are like, "You have your instructions, and I have my way of doing things. Once I'm done, I'll just convince you that what I delivered is what you originally wanted. If there are any issues, we'll just wait for the users to find them; at that point, all I have to do is admit I was wrong, and I suffer no loss.
"It confirmed that a command ran, then reported that the thing worked" is the gap exactly.
It's worth separating which parts of that a machine can check. A claim like "tests passed" can be graded against the session log: was a real test run there, and what did it exit with? We build an open-source checker for that (coherence;
coherence audit <session>.jsonl). The honest limit, which your case shows well: a script that prints "ok" unconditionally still exits 0, so a log-only check would have passed your item 1. And a stub worker's "✓ tests passed" is still a run that printed success. Only the real run you forced catches those.The "printed ok unconditionally, then reported the fix as done" case is the one I kept hitting too, so I measured it in my own workspace: the verification rule was in CLAUDE.md for all 22 verification-shaped replies over a week, and the reply followed it in 13 (59%). The rule was in context every time; it just was not applied at the moment of the claim.
What changed the rate for me was moving the check out of the instructions file and into hooks that fire at that moment:
PostToolUsehook on Bash that scans the output for failure signals (non-zero exit, failing test counts, tracebacks, "permission denied") and injects a note naming what it found, so a failed replace or a skipped launcher run is in context when the agent writes "fixed";UserPromptSubmithook that, on "did it work / is it done" prompts, asks the reply to open with what was checked out of what (e.g.verified 2/3 — did not run army.cmd);It does not make the agent honest. It makes the failed output hard to skip past. The hooks are three bash files (bash + jq, MIT) if anyone wants to copy the approach: https://github.com/laika56/claude-code-guardrails
Disclosure: I wrote that repo.