code-review plugin: step-1 'already commented' guard makes re-review a silent no-op that still exits success

Resolved 💬 3 comments Opened Jul 10, 2026 by Jdubz Closed Jul 13, 2026

Summary

plugins/code-review/commands/code-review.md step 1 tells a haiku agent to stop and do not proceed if "Claude has already commented on this PR". Combined with claude-code-action's continue-on-error usage as a CI check, this makes the command unusable as a required review gate: the run exits success in ~20 seconds, posts nothing, and is indistinguishable from "reviewed, found nothing."

The skipped case is exactly the one that matters most: re-reviewing after the author has addressed the review's own feedback. By construction, Claude has commented by then.

Evidence

Using prompt: '/code-review:code-review <pr> --comment' on anthropics/claude-code-action@v1, self-hosted, OAuth token:

| run | trigger | turns | duration | posted a comment? |
|---|---|---|---|---|
| A | issue_comment | 10 | 4m49s | ✅ "No issues found." |
| B | issue_comment (after A) | 4 | 13s | ❌ |
| C | issue_comment (after A) | 3 | 47s | ❌ |
| D | issue_comment (after A) | 7 | 19s | ❌ |
| E | issue_comment (after A) | 5 | 20s | ❌ |

Every one of B–E reported conclusion: success. Three commits were pushed after A. The only reliable way we found to get a fresh review was gh pr close && gh pr reopen, which fires pull_request: reopened — at the cost of cancelling in-flight CI.

A second, distinct problem: a real review can post nothing

On a different PR with no prior Claude comment, a pull_request: reopened run performed a full review — 26 turns, 1m45s — found no issues, and posted no comment at all, despite step 7:

If --comment argument IS provided and NO issues were found, post a summary comment using gh pr comment and stop.

So even without the step-1 guard, "no comment" cannot be read as "clean". Every run in our logs also reports permission_denials_count: 1, with allowed-tools matching the command's frontmatter — possibly related, but we couldn't see which tool was denied (show_full_output: false).

Why this matters

Our repo's merge policy requires a Claude review of the commit being merged before a PR can deploy to production. With the plugin, that gate silently degrades to a no-op after the first review. We caught it only by comparing num_turns / duration_ms across runs; from the GitHub UI both look identical (green check, no comments).

Suggested fixes

  1. Drop, or make opt-out, the step-1 "already commented" guard. A re-review request is an explicit instruction to review; honoring it is the point. At minimum, don't skip when the trigger was an explicit @claude review on a commit newer than the last review.
  2. Always post a verdict, and stamp it with the reviewed commit sha (e.g. ## Claude review — <sha>). Then "clean" is positively distinguishable from "did not run", and the result is machine-checkable by a merge gate.
  3. Signal a no-op. If the command exits without producing a verdict, say so on stdout in a greppable form (or let the action surface a non-success outcome) so it can't be mistaken for a clean review.
  4. Investigate the ubiquitous permission_denials_count: 1.

Workaround

We stopped using the plugin command and inlined an equivalent prompt (parallel reviewers + a validation pass) that always posts a sha-stamped verdict, and we now gate merges on that comment's presence rather than on the check's colour.

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗