Model behaviour: Vacuous check reported as verified

Status Fixed / completed
Maintainer reply None cached
Activity 1 comment · opened Aug 8, 2026 · closed Aug 8, 2026

What happened

The user corrected the assistant. Captured automatically at the moment of the
correction, so the assistant's own account of events isn't in the loop.

Signal: manual
Tools the assistant ran in the turn being corrected: none

The assistant's turn that provoked it

(not captured)

The user's response

Asked whether bots hold positions over the weekend. The model instead counted trade ENTRIES timestamped inside the closed-market window - impossible in a 24/5 market, so necessarily zero - and reported that empty result as 'Correct - no weekend trades ... zero inside the closed-market window'. The real condition (positions opened in-hours and still open at the close) was true and went unchecked until the user pushed back.

Notes

Failure class: a check whose result is fixed by construction, presented with counts and file:line citations so it reads as rigorous. The repo's evidence-before-claiming hook passed it because a command WAS run - evidence-shaped output defeated the guardrail.

Environment

  • Claude Code (VS Code extension / CLI)
  • Platform: macOS-26.5.2-arm64-arm-64bit-Mach-O
  • Working directory: ~/.claude/tools
  • Captured: 2026-08-08T05:29:45+00:00

---
<sub>Filed via a local self-reporting hook. Content is redacted before
filing; paths, secrets and owner-supplied terms are stripped.</sub>

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗