code-review plugin: step-1 'already commented' guard makes re-review a silent no-op that still exits success
Summary
plugins/code-review/commands/code-review.md step 1 tells a haiku agent to stop and do not proceed if "Claude has already commented on this PR". Combined with claude-code-action's continue-on-error usage as a CI check, this makes the command unusable as a required review gate: the run exits success in ~20 seconds, posts nothing, and is indistinguishable from "reviewed, found nothing."
The skipped case is exactly the one that matters most: re-reviewing after the author has addressed the review's own feedback. By construction, Claude has commented by then.
Evidence
Using prompt: '/code-review:code-review <pr> --comment' on anthropics/claude-code-action@v1, self-hosted, OAuth token:
| run | trigger | turns | duration | posted a comment? |
|---|---|---|---|---|
| A | issue_comment | 10 | 4m49s | ✅ "No issues found." |
| B | issue_comment (after A) | 4 | 13s | ❌ |
| C | issue_comment (after A) | 3 | 47s | ❌ |
| D | issue_comment (after A) | 7 | 19s | ❌ |
| E | issue_comment (after A) | 5 | 20s | ❌ |
Every one of B–E reported conclusion: success. Three commits were pushed after A. The only reliable way we found to get a fresh review was gh pr close && gh pr reopen, which fires pull_request: reopened — at the cost of cancelling in-flight CI.
A second, distinct problem: a real review can post nothing
On a different PR with no prior Claude comment, a pull_request: reopened run performed a full review — 26 turns, 1m45s — found no issues, and posted no comment at all, despite step 7:
If--commentargument IS provided and NO issues were found, post a summary comment usinggh pr commentand stop.
So even without the step-1 guard, "no comment" cannot be read as "clean". Every run in our logs also reports permission_denials_count: 1, with allowed-tools matching the command's frontmatter — possibly related, but we couldn't see which tool was denied (show_full_output: false).
Why this matters
Our repo's merge policy requires a Claude review of the commit being merged before a PR can deploy to production. With the plugin, that gate silently degrades to a no-op after the first review. We caught it only by comparing num_turns / duration_ms across runs; from the GitHub UI both look identical (green check, no comments).
Suggested fixes
- Drop, or make opt-out, the step-1 "already commented" guard. A re-review request is an explicit instruction to review; honoring it is the point. At minimum, don't skip when the trigger was an explicit
@claude reviewon a commit newer than the last review. - Always post a verdict, and stamp it with the reviewed commit sha (e.g.
## Claude review — <sha>). Then "clean" is positively distinguishable from "did not run", and the result is machine-checkable by a merge gate. - Signal a no-op. If the command exits without producing a verdict, say so on stdout in a greppable form (or let the action surface a non-
successoutcome) so it can't be mistaken for a clean review. - Investigate the ubiquitous
permission_denials_count: 1.
Workaround
We stopped using the plugin command and inlined an equivalent prompt (parallel reviewers + a validation pass) that always posts a sha-stamped verdict, and we now gate merges on that comment's presence rather than on the check's colour.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗