Model confirmation bias in iterative adversarial QC agent loops

Description

When using Claude Code's Agent tool to run iterative QC-Fix-Verify loops (a common pattern in agent orchestration), the QC agents exhibit systematic confirmation bias in verification rounds, even when explicitly prompted to be "adversarial."

Observed Behavior

Round 1 (fresh adversarial QC): Agents genuinely find issues. In my testing, two parallel QC agents found 13 and 20 real issues respectively across two documents. This round works as expected.

Round 2+ (verify-QC after fixes): Agents receive the original QC report + fix report and are asked to: (1) verify all claimed fixes, (2) do a full adversarial scan for regressions and new issues. In practice, the agents treat task (1) as primary and task (2) as secondary. They confirm fixes thoroughly but give the adversarial scan less rigor.

Concrete evidence of the problem:

• An agent metadata header artifact (11 lines of ## Summary, Verdict: COMPLETE, etc.) survived two full verify-QC rounds before being caught in Round 3. This is extremely obvious — literal agent output metadata left in the final document.
• A parallel QC agent reviewing a different document caught the same type of artifact in Round 2. So the pattern is inconsistent — one agent catches it, another misses it, suggesting the "adversarial scan" portion is not thorough.
• An arithmetic error (200 vs 160 hours) was noted by the QC agent but explicitly dismissed as "below MINOR" and "optional fix," even though it was a factual error in the document.

Expected Behavior

When an agent is prompted to be "adversarial" and to "find ALL problems," it should:

1. Give equal rigor to verifying claimed fixes AND scanning for new/missed issues
2. Not dismiss factual errors as "below MINOR"
3. Catch obvious artifacts (like agent metadata headers) on the first pass, not the third

Root Cause Analysis

This appears to be a combination of:

1. Model-level confirmation bias: Claude tends toward agreement/confirmation in later iterations of a loop, even when explicitly told to challenge and find problems
2. Anchoring effect: Providing the fix report as context anchors the agent's attention on verifying those specific fixes, reducing attention to the broader adversarial scan
3. Progressive trust: The model appears to develop implicit trust that a document that has been through N rounds of QC is likely clean, reducing adversarial rigor

Reproduction

1. Use Claude Code Agent tool to generate a long document (~1,500 lines)
2. Launch an adversarial QC agent (Round 1) — this works well
3. Launch a fix agent to address found issues
4. Launch a verify-QC agent (Round 2) with instructions to verify fixes AND do a full adversarial scan
5. Observe that obvious issues (formatting artifacts, metadata headers, factual arithmetic errors) survive multiple verify rounds

Suggested Improvements

• Model-level: Train/tune for maintained adversarial rigor across QC iterations. The model should not reduce skepticism in later rounds.
• Documentation: Add guidance in Claude Code docs about this tendency, so users can design their QC prompts to counteract it (e.g., running fresh adversarial QC agents that don't see previous QC reports, rather than verify-QC agents that do).
• Prompt engineering guidance: Suggest separating "verify claimed fixes" and "fresh adversarial scan" into two separate agents, so neither task is deprioritized.

Environment

• Claude Code CLI
• Model: Claude Opus 4.6
• Agent tool with run_in_background: true
• Iterative QC-Fix-Verify loop pattern (4 rounds)

Impact

This pattern is particularly problematic for users building quality-critical document pipelines (regulatory, academic, legal) where "find ALL issues" means exactly that. The false sense of security from a "PASS" verdict that missed obvious issues is worse than no QC at all.