First-class "adversarial-verify + live-source check" review mode for multi-agent workflows
Type: enhancement · Area: multi-agent workflows / verification patterns
Motivation
A repeatedly high-value workflow shape is: produce, then adversarially verify against the source of truth. Concretely, in a codebase audit:
- A "mapper" agent extracts claims (invariants, risks, behaviors) from code + history + docs.
- A separate "verifier" agent is instructed to refute each claim by opening the actual files.
- Claims that survive are confirmed; claims marked
needs-live-dbare settled by read-only queries against the running system.
In real runs this caught defects the mapper asserted confidently, and corrected the verifier itself — a claim derived from static source (a migration file) was wrong about the live system, and only the read-only production check surfaced the truth. Neither single agent, and no static-only pass, would have caught it.
Proposed feature
A documented, first-class pattern / helper for "generate → adversarial-verify → live-source settle," including:
- a verifier role prompt template that defaults to skepticism (refute-by-default, cite
file:line), - a structured verdict schema (
confirmed/refuted/partly-true/needs-live-db), - guidance/guardrails for a read-only live-source settle step (e.g. enforced read-only transactions) so verification against a running system is safe by construction.
Why it helps everyone
"One agent confidently produces a plausible-but-wrong artifact" is the central failure mode of LLM automation. Making adversarial verification a supported, low-friction mode — rather than something each author hand-rolls — raises the floor on trustworthiness for audits, reviews, migrations, and research.
Filed from real multi-agent workflow usage.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗