[Model behavior] Batch PR verdicts reversed after one follow-up question; procedure applied only after challenge

Status Open
Maintainer reply None cached
Activity 0 comments · opened Oct 7, 2026

Summary

Asked to review eight pull requests, the model issued six "Reject" verdicts. The user asked whether the underlying issues were real. The model confirmed they were, then reversed all six verdicts to "Accept-and-apply" in its next message.

The reversal was correct under the user's loaded procedure, which reserves Reject for changes that real AWS behaviour contradicts and sends fixable problems to Accept-and-apply. The procedure was loaded before the first verdicts, yet the model applied it only after being challenged.

The same session also showed the pattern the user describes as wasted time:

  • The model answered meta questions at length.
  • It re-listed the same PRs several times.
  • It did not go deep on the one large PR (about 1,300 lines) the user actually cared about.

The user reports this degradation started after moving from Pro to Max. That is the second time the change followed a plan switch; see #100311.

Environment

  • Claude Code CLI, Opus-class model, 1M context
  • Code-review workflow with a written verdict procedure loaded every session

Expected

  • Classify each PR against the loaded procedure before the first verdict.
  • Do not reverse verdicts in bulk when asked a follow-up question.
  • Spend effort on the substantive review the user asked for.

Impact

The user cannot act on the first answer and has to cross-examine every verdict, which takes more time than doing the review by hand.

View original on GitHub ↗