Model overclaims verification results in a review summary
Status Open
Maintainer reply None cached
Activity 0 comments · opened Sep 25, 2026
What happened
The review summary stated that things 'held up' ('no caller breaks', 'no duplication', matches the model) when only partial checks had been done.
Expected
Report only what was actually verified, and name what was not.
Context: a project CLAUDE.md plus the user's auto-memory define a mandatory, ordered code-review checklist (read every touched file, the whole diff, linked issue and thread; cite the normative source per component; grep callers; check duplication, new structure and scope; then a single short verdict). The model had the checklist loaded in context and had been reminded of it repeatedly in the same session. Reviewing a multi-thousand-line pull request.
Reported by the user; no transcript or private content included.