Subagent-authored CI gate keys its suppression baseline on file:line, producing false build failures on unrelated edits (and inviting a fail-open repair)

Status Open
Maintainer reply None cached
Activity 0 comments · opened Jul 30, 2026

Summary

A Sonnet subagent, asked to mechanize a CSS lint rule, wrote a CI gate whose suppression baseline is keyed on file:line. Any edit that inserts lines above a grandfathered site makes the gate fail the build with a false "NEW violation", while simultaneously reporting the original entry as stale. Filing because the failure mode is not "slightly wrong code" — it is a gate that converts from fail-noisy to fail-open through the repair it invites, and neither the authoring subagent nor an independently-dispatched reviewing subagent caught it.

Environment

  • Claude Code CLI, Opus 5 orchestrating, Sonnet 5 subagents in isolated git worktrees
  • Task given to the subagent: fix a min-width: 0 CSS defect class and mechanize a check so new instances are refused
  • The gate itself is legitimate: it matches display: flex|grid and requires a min-width: 0 shrink companion in the same colocated scope

What was written

There were 17 pre-existing violations, so the subagent grandfathered them into a baseline JSON file. Each violation object was built like this:

violations.push({
  key: `${relFile}:${line}`,
  message:
    `${relFile}:${line}: a flex/grid container (${m[0].trim()}) has no 'min-width: 0' ` +
    // ...
})

and that same key drives the pass/fail decision:

const newViolations = allViolations.filter((v) => !(v.key in baseline))
const staleBaselineEntries = Object.keys(baseline).filter((k) => k !== '_comment' && !violationKeys.has(k))

Baseline file contents, for shape:

{
  "src/core/components/ItemDetail.vue:272": "...",
  "src/core/components/ItemDetail.vue:281": "...",
  "src/extensions/autoharn/components/ResourceFieldsCard.vue:47": "...",
  "src/extensions/autoharn/components/ResourceFieldsCard.vue:52": "..."
}

Observed failure

Merging an unrelated branch — one that added a CSS class and touched neither file's flex containers — inserted 5 lines above the ItemDetail.vue sites and 7 above the ResourceFieldsCard.vue sites. npm run build then failed with 4 NEW violations at :277, :286, :54, :59.

The four containers are byte-identical before and after, verified directly:

$ git show 56f90d1:frontend/src/core/components/ItemDetail.vue | sed -n '272p;281p'
  display: flex;
  display: flex;

So the same run both claimed four containers had appeared and four had vanished — for four containers that never changed.

Why this is worse than a cosmetic bug

The natural operator response to a false red is to re-record the baseline. Doing so silently accepts whatever genuine new violations happen to be present at that moment. A gate that produces false failures on unrelated edits therefore trains its users into the one action that makes it stop detecting anything. The defect is self-defeating in a way an ordinary false positive is not.

How it got in, mechanically

The error-message string is built first, because that is what an operator reads, and file:line is correct there. The identity key was then taken from the value already sitting in the same object literal. A display coordinate was promoted to an identity without anything in between asking what a violation site's identity actually is. A line number is a coordinate; the site's identity is its file plus its selector.

Why review didn't catch it

The gate was tested by authoring a new offending container and confirming the build refused — which exercises only the new-violation path and never touches baseline stability. An independently dispatched reviewing subagent read the file closely enough to find an unrelated off-by-one three lines away (the summary line counted a _comment metadata key as a site), and still did not flag the keying. Line-keying is invisible until a diff actually moves a line.

Suggested guardrails

  1. Treat "a suppression/baseline entry keyed on a line number" as an anti-pattern worth refusing outright when generating lint or CI tooling. Stable identities: file + selector, file + a content hash of the matched block, or an in-source marker comment.
  2. When a model generates a gate with a grandfathering baseline, prompt it to test baseline stability under an unrelated edit above a baselined site, not only the new-violation path. That single test would have caught this immediately.
  3. Prefer fixing a small pre-existing violation set over generating a suppression list at all. 17 sites here each needed one min-width: 0 line; the baseline mechanism cost more than the fix and introduced this defect.

Happy to supply the full script if useful.

View original on GitHub ↗