[feedback] instrument-vs-proxy: Three independent judge seats and one builder verified a coverage floor with a hand --cov= run instead of the

Status Open
Maintainer reply None cached
Activity 1 comment · opened Sep 28, 2026

Context. Solo-operator project using Claude Code (desktop app, Windows 11) with a lead session on Claude Fable 5.1 orchestrating Opus 5.5 / Sonnet subagents (worktree-isolated), a JSON task store, PreToolUse/SubagentStop hooks and a harvest-train commit flow. This finding was measured on or before 2026-09-22 and filed on 2026-09-28 at the operator's request; numbers are from tool output. No secrets, identities or private paths.

Class / model / severity. instrument-vs-proxy / Opus (judge seats), Sonnet (fix-pack lane) / HIGH

Finding. Three independent judge seats and one builder verified a coverage floor with a hand --cov= run instead of the project's coverage instrument, which read 0.0 on the moved modules; the fix pack deleted the assertion that would have caught it.

Evidence (measured). Both round-1 faces and one round-2 face of the risk_halt package pair (reports/j_M4_M5_M3_*); the matcher endswith('<name>.py') cannot match a dotted key.

Minimal reproduction. Give the model a spec naming a specific checker script and a shortcut reporting the same metric; observe which it runs.

Ask. When a spec names an instrument, the proof goes through it; any substitute is labelled PROXY. Independence of instances is not independence of instrument.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗