Pattern + proposal: adversarial self-review via a fresh sub-agent to counter author-blindness (a concrete fix for #32301)

Status Fixed / completed
Maintainer reply ✓ Yes — bcherny
Activity 4 comments · opened Jun 29, 2026 · closed Aug 16, 2026
💡 Likely answer: A maintainer (bcherny, collaborator) responded on this thread — see the highlighted reply below.

TL;DR

When the same agent that wrote the code reviews it, it tends to confirm its own reasoning instead of trying to falsify it — "author-blindness" (the problem reported in #32301). Adding more review instructions to context does not fix this; the same biased context recalls the rules rather than running them. What reliably works is structural independence: spawn a fresh sub-agent to review the diff. It did not write the code, so it reads it cold and tries to break it. Built entirely on existing primitives (Skill + the Agent/sub-agent tool + a PreToolUse hook + TodoWrite), this caught a large fraction of issues a same-context self-pass missed. I'm sharing the pattern and proposing first-class support.

The problem (why a self-pass under-fires)

A self-review fails for three structural reasons, none of which more prose fixes:

  1. No independence. The author-context agrees with the reasoning that produced the bug.
  2. Recall, not execution. A long checklist of "things to check" is remembered, not run; under completion-pressure the agent declares a claim verified without executing the command that would falsify it.
  3. No forcing function. The pass is optional; nothing gates "done."
The key realization: this is not a knowledge gap. The agent already "knows" the checks. It's an independence + execution + enforcement gap.

The pattern (built on existing Claude Code primitives)

Four mechanisms, one per gap:

  1. Independence → a fresh adversarial sub-agent. A Skill (/self-review) spawns a sub-agent via the Agent tool over git diff, with this contract:
  • "You did not write this code. Be adversarial — try to falsify each claim, not confirm it."
  • Run-don't-read: "for every data/numeric/behavioral claim, RUN the command that would falsify it and judge from the OUTPUT — never from reading the code." (A query can read perfectly and still return 0 rows, be 100× too slow, or collapse a total to zero — all invisible until executed.)
  • A finding-class catalogue to hunt (below) + "reason from first principles about what could break that I didn't think of."
  • Return: a verdict (GO / GO-WITH-CHANGES / NO-GO) + a numbered findings table (severity + file:line + failing scenario + required edit).

A clean-context sub-agent provides real contextual independence. (Honest ceiling: a same-model sub-agent still shares model-level blind spots, so we reserve a different-model reviewer for the highest-risk changes — but the same-model fresh agent removes the bulk of context-level author-blindness, for free, every change.)

  1. Execution over recall → a TodoWrite-driven evidence log. The skill turns the change into a TodoWrite checklist: for each touched surface × applicable finding-class, one todo = the exact falsifying command. Each is run and written to a log block (CLAIM / CMD / ENV / OUT / VERDICT / STRENGTH). Rule: if you can't name a command that would falsify a claim, it's UNVERIFIED, not verified. Pair every pass with a negative control (show it actually tripped — a test that cannot fail is a false green).
  1. Enforcement → a PreToolUse hook. On git commit, if code is staged with no recent review log, the hook nudges (or blocks) "run the review first." This is the forcing function that makes the independent pass non-skippable.
  1. Codification loop (what makes it compound). Every finding that generalizes updates the project's rule files and the finding-class catalogue and the review skill itself — so the next review is strictly stronger than the last. Review → fold → codify → the codified class fires next time → repeat.

A reusable, domain-neutral finding-class taxonomy

Mined from a corpus of ~200 independent reviews, these are the recurring classes a self-pass skips (generic — adapt per project):

  • Parity / claim-falsification — a "faithful / equivalent / unchanged" claim is a claim under attack; verify against the real source.
  • Provenance / honest-divergence — a "transparent refactor" that silently changes outputs for some inputs must be flagged as a contract change, not a refactor.
  • Determinism — unpinned tie-breaks; LIMIT 1 with no secondary sort.
  • Cross-surface divergence — the same concept computed two ways on two surfaces.
  • Enforcement vs decoration — a guard/validator/schema that doesn't actually run on the real path (or a UI gate not re-enforced server-side).
  • Concurrency seam — atomic-claim-before-write; a claimed state stranded if a step between two irreversible commits throws; reading a decision input on the wrong side of a lock; which event arms a TTL vs when the work starts; broker redelivery semantics.
  • Authorization population — grep the whole bypass/persona set, not one call site; distinguish cross-tenant scoping from entity-state gates.
  • Money/grain — sign conventions; 1:N JOIN inflation; excluding rows from an apportionment denominator vs the output grain.
  • Perf under real load — not an isolated EXPLAIN; run it at real scale.
  • Silent caps / truncation.
  • Signal/observability semantics — a health probe must model OK / BAD / UNKNOWN, never green over a failed probe.
  • Runtime-substrate-only defects (the deepest) — bugs visible only when you RUN against the real substrate: actual data distribution (a "rare edge" that's actually the majority of rows), prod topology (worker concurrency/mounts), the query planner (an index a predicate can't use), a packaged/frozen build. Static review — including a strong independent reviewer's — is necessary but not sufficient here; only running a diagnostic against real data reveals it.

Two empirical dynamics worth knowing:

  • The second-order fold law: each fix tends to spawn the next round's finding (the fix introduces a new edge). So re-review every fold, not just the original finding.
  • Convergence signature: rounds with a monotone-decreasing count of HIGH-severity findings are converging; if the HIGH count won't fall, the design is wrong, not the code.

Obstacle worth flagging

Spawning a sub-agent to do a security/correctness review can trip safety-classifier false-positives inside the sub-agent (see #64558) — which directly penalizes this otherwise-valuable multi-agent review design. Worth considering when tuning classifiers for legitimate defensive/review use inside spawned agents.

Proposal to Claude Code

  1. A first-class adversarial self-review — e.g. a built-in /self-review (or a --reviewer sub-agent mode) that spawns a clean-context sub-agent with the run-don't-read + falsify contract and returns a verdict table. The "fresh context reviewing the diff" is the load-bearing part.
  2. Ship templates: a self-review Skill + a PreToolUse commit-gate hook + a documented finding-class taxonomy, so projects adopt the pattern without rebuilding it.
  3. At minimum, document the pattern (independence > more-prompting) as guidance — it's the most reliable mitigation for #32301 we've found.

Appendix — the two rules that carry the most weight

  • Run-don't-read: a data/numeric/behavioral claim is verified by RUNNING a probe and reading its output, never by reading the code.
  • Independence beats instruction: you cannot prompt a context out of its own blind spots; you have to hand the work to a context that didn't create them.

(Mechanisms here are all standard Claude Code primitives; no proprietary or project-specific content. Happy to share sanitized skill/hook templates if useful.)

View original on GitHub ↗

3 Comments

kcarriedo · 3 months ago

This is a well-structured case for structural independence over prompt-as-fix. The run-don't-read rule is the part most likely to actually change behavior -- a long checklist in context gets recalled as a belief ("I checked this") rather than executed as a verification step, which is exactly the failure mode you are describing.

One addition worth considering for the finding-class taxonomy: state coherence across handoffs. When a parent session delegates to a sub-agent and then consumes the result, there is an implicit assumption that the sub-agent's output is self-consistent and that nothing in the parent's accumulated context contradicts it. In practice, when the sub-agent reviews code the parent wrote earlier in the session, the parent's context already contains a story about why that code is correct. A truly adversarial reviewer needs not just a fresh context window but a directive to treat the parent's framing as adversarial input, not shared ground truth -- otherwise the review inherits the author's priors through the task description.

The enforcement gap you describe (a gate that nudges rather than blocks until evidence is on file) is the structural fix that actually makes this reliable. The distinction between "things to check" and "commands that would falsify claims, run and logged" is load-bearing.

One concern worth flagging: the classifier false-positive issue (#64558) hits hardest on the reviews that matter most -- security and correctness review of production-touching code is exactly the kind of review the classifier flags. That asymmetry (the more sensitive the review, the higher the false-positive risk) is worth Anthropic addressing before this pattern gets widely adopted.

github-actions[bot] · 1 month ago

Closing for now — inactive for too long. Please open a new issue if this is still relevant.

bcherny collaborator · 1 month ago

Thanks for the detailed writeup. The core of this is available as a built-in: /code-review (alias /review) reviews your current diff (or /code-review <level> <pr#> for a PR) using fresh subagents that didn't write the code, then verifies candidate findings against the actual code before reporting. For the enforcement piece, a Stop or PreToolUse hook can require the review to have run before Claude finishes or commits.

Docs: https://code.claude.com/docs/en/code-review#review-a-diff-locally and https://code.claude.com/docs/en/hooks

🤖 Generated with Claude Code

Showing cached comments. Read the full discussion on GitHub ↗