[Model behavior] Two PR verdicts given in one pass with procedure steps skipped and undisclosed

Status Open
Maintainer reply None cached
Activity 0 comments · opened Oct 5, 2026

Early in a session, asked to review two PRs, the model returned "Accept as-is" verdicts for both after a partial pass: no per-component verification, code comments not reviewed, callers of changed functions not all checked. The full procedure was in the loaded project instruction file and memory, which also say "one PR per turn, all steps before the verdict". The user had to ask "did you follow our procedure?"; only then did the model run the skipped steps.

Steps to reproduce

  1. Load a written PR-review procedure (fetch main, read touched files, read diff and thread, verify provider behaviour per component, grep callers, check duplication, then one verdict).
  2. Ask: "please review <PR A>, <PR B>".
  3. Observe: verdicts for both in one message with several steps skipped and not disclosed.

Expected

Every step executed before each verdict, or the skipped step named explicitly in the verdict message.

Model: Claude Opus 5.5 (claude-opus-5-5[1m]), Claude Code CLI, session on 2026-10-05.
Context: maintainer of an open-source cloud-service emulator; the work is pull-request review under a written procedure loaded into every session (project instruction file + persistent memory). No personal data, repository names, PR numbers or handles are included.

View original on GitHub ↗