Pattern + proposal: adversarial self-review via a fresh sub-agent to counter author-blindness (a concrete fix for #32301)

Open 💬 1 comment Opened Jun 29, 2026 by casriraj

TL;DR

When the same agent that wrote the code reviews it, it tends to confirm its own reasoning instead of trying to falsify it — "author-blindness" (the problem reported in #32301). Adding more review instructions to context does not fix this; the same biased context recalls the rules rather than running them. What reliably works is structural independence: spawn a fresh sub-agent to review the diff. It did not write the code, so it reads it cold and tries to break it. Built entirely on existing primitives (Skill + the Agent/sub-agent tool + a PreToolUse hook + TodoWrite), this caught a large fraction of issues a same-context self-pass missed. I'm sharing the pattern and proposing first-class support.

The problem (why a self-pass under-fires)

A self-review fails for three structural reasons, none of which more prose fixes:

  1. No independence. The author-context agrees with the reasoning that produced the bug.
  2. Recall, not execution. A long checklist of "things to check" is remembered, not run; under completion-pressure the agent declares a claim verified without executing the command that would falsify it.
  3. No forcing function. The pass is optional; nothing gates "done."
The key realization: this is not a knowledge gap. The agent already "knows" the checks. It's an independence + execution + enforcement gap.

The pattern (built on existing Claude Code primitives)

Four mechanisms, one per gap:

  1. Independence → a fresh adversarial sub-agent. A Skill (/self-review) spawns a sub-agent via the Agent tool over git diff, with this contract:
  • "You did not write this code. Be adversarial — try to falsify each claim, not confirm it."
  • Run-don't-read: "for every data/numeric/behavioral claim, RUN the command that would falsify it and judge from the OUTPUT — never from reading the code." (A query can read perfectly and still return 0 rows, be 100× too slow, or collapse a total to zero — all invisible until executed.)
  • A finding-class catalogue to hunt (below) + "reason from first principles about what could break that I didn't think of."
  • Return: a verdict (GO / GO-WITH-CHANGES / NO-GO) + a numbered findings table (severity + file:line + failing scenario + required edit).

A clean-context sub-agent provides real contextual independence. (Honest ceiling: a same-model sub-agent still shares model-level blind spots, so we reserve a different-model reviewer for the highest-risk changes — but the same-model fresh agent removes the bulk of context-level author-blindness, for free, every change.)

  1. Execution over recall → a TodoWrite-driven evidence log. The skill turns the change into a TodoWrite checklist: for each touched surface × applicable finding-class, one todo = the exact falsifying command. Each is run and written to a log block (CLAIM / CMD / ENV / OUT / VERDICT / STRENGTH). Rule: if you can't name a command that would falsify a claim, it's UNVERIFIED, not verified. Pair every pass with a negative control (show it actually tripped — a test that cannot fail is a false green).
  1. Enforcement → a PreToolUse hook. On git commit, if code is staged with no recent review log, the hook nudges (or blocks) "run the review first." This is the forcing function that makes the independent pass non-skippable.
  1. Codification loop (what makes it compound). Every finding that generalizes updates the project's rule files and the finding-class catalogue and the review skill itself — so the next review is strictly stronger than the last. Review → fold → codify → the codified class fires next time → repeat.

A reusable, domain-neutral finding-class taxonomy

Mined from a corpus of ~200 independent reviews, these are the recurring classes a self-pass skips (generic — adapt per project):

  • Parity / claim-falsification — a "faithful / equivalent / unchanged" claim is a claim under attack; verify against the real source.
  • Provenance / honest-divergence — a "transparent refactor" that silently changes outputs for some inputs must be flagged as a contract change, not a refactor.
  • Determinism — unpinned tie-breaks; LIMIT 1 with no secondary sort.
  • Cross-surface divergence — the same concept computed two ways on two surfaces.
  • Enforcement vs decoration — a guard/validator/schema that doesn't actually run on the real path (or a UI gate not re-enforced server-side).
  • Concurrency seam — atomic-claim-before-write; a claimed state stranded if a step between two irreversible commits throws; reading a decision input on the wrong side of a lock; which event arms a TTL vs when the work starts; broker redelivery semantics.
  • Authorization population — grep the whole bypass/persona set, not one call site; distinguish cross-tenant scoping from entity-state gates.
  • Money/grain — sign conventions; 1:N JOIN inflation; excluding rows from an apportionment denominator vs the output grain.
  • Perf under real load — not an isolated EXPLAIN; run it at real scale.
  • Silent caps / truncation.
  • Signal/observability semantics — a health probe must model OK / BAD / UNKNOWN, never green over a failed probe.
  • Runtime-substrate-only defects (the deepest) — bugs visible only when you RUN against the real substrate: actual data distribution (a "rare edge" that's actually the majority of rows), prod topology (worker concurrency/mounts), the query planner (an index a predicate can't use), a packaged/frozen build. Static review — including a strong independent reviewer's — is necessary but not sufficient here; only running a diagnostic against real data reveals it.

Two empirical dynamics worth knowing:

  • The second-order fold law: each fix tends to spawn the next round's finding (the fix introduces a new edge). So re-review every fold, not just the original finding.
  • Convergence signature: rounds with a monotone-decreasing count of HIGH-severity findings are converging; if the HIGH count won't fall, the design is wrong, not the code.

Obstacle worth flagging

Spawning a sub-agent to do a security/correctness review can trip safety-classifier false-positives inside the sub-agent (see #64558) — which directly penalizes this otherwise-valuable multi-agent review design. Worth considering when tuning classifiers for legitimate defensive/review use inside spawned agents.

Proposal to Claude Code

  1. A first-class adversarial self-review — e.g. a built-in /self-review (or a --reviewer sub-agent mode) that spawns a clean-context sub-agent with the run-don't-read + falsify contract and returns a verdict table. The "fresh context reviewing the diff" is the load-bearing part.
  2. Ship templates: a self-review Skill + a PreToolUse commit-gate hook + a documented finding-class taxonomy, so projects adopt the pattern without rebuilding it.
  3. At minimum, document the pattern (independence > more-prompting) as guidance — it's the most reliable mitigation for #32301 we've found.

Appendix — the two rules that carry the most weight

  • Run-don't-read: a data/numeric/behavioral claim is verified by RUNNING a probe and reading its output, never by reading the code.
  • Independence beats instruction: you cannot prompt a context out of its own blind spots; you have to hand the work to a context that didn't create them.

(Mechanisms here are all standard Claude Code primitives; no proprietary or project-specific content. Happy to share sanitized skill/hook templates if useful.)

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗