[feedback] shared-model-judges: Two same-model judge seats produced largely disjoint findings (a strength) but shared one model-level bias (FB

Status Open
Maintainer reply None cached
Activity 1 comment · opened Sep 28, 2026

Context. Solo-operator project using Claude Code (desktop app, Windows 11) with a lead session on Claude Fable 5.1 orchestrating Opus 5.5 / Sonnet subagents (worktree-isolated), a JSON task store, PreToolUse/SubagentStop hooks and a harvest-train commit flow. This finding was measured on or before 2026-09-22 and filed on 2026-09-28 at the operator's request; numbers are from tool output. No secrets, identities or private paths.

Class / model / severity. shared-model-judges / Opus (judge seats) / MEDIUM

Finding. Two same-model judge seats produced largely disjoint findings (a strength) but shared one model-level bias (FB-003); a seat reversed its view after reading the other but kept its recorded verdict (correct).

Evidence (measured). reports/j_M4_M5_M3_r2_seat1 and seat2; the concurrence reads.

Minimal reproduction. Run the same review with two instances of one model vs two models.

Ask. For safety adjudication, diversify instrument and model, not only instance.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗