Failure class: model satisfies the local instruction but does not verify the artifact against the top-level goal (especially what should be absent), even when stated explicitly

Preflight

• [x] I searched existing issues for similar behavior reports (related but distinct: #17097, #21187, #33603, #60583 — see Additional context).
• [x] This report contains NO sensitive information (the example is genericized).

Type of behavior issue

Unreliable execution of an explicit instruction — the model reports a task done without verifying the finished artifact against the stated top-level goal, and is specifically blind to negative space (what the goal requires be removed / absent).

What I asked Claude to do

One explicit instruction: "Make this repo/installer generic and public-safe — remove ALL my personal identifiers, names, and project names; none should remain anywhere."

What Claude actually did

Across multiple passes it genericized most of the repo but left personal identifiers hardcoded in several locations (a value in a source dictionary; a default fallback in ~17 files; doc examples; config/SQL header comments). It reported the genericization passes complete. The remaining identifiers were removed only after I explicitly pointed them out — and pointing out one instance did not reliably trigger a sweep for the rest.

Expected behavior

Before reporting done, derive the goal's implied invariants and verify the whole artifact against them — especially negative space ("does ANY identifier still remain?") — and catch the leaks itself. An explicitly stated constraint should be sufficient; the human should not have to be the backstop.

Files affected

(Genericized) a classifier source file (hardcoded dict), ~17 files carrying a default fallback value, a backfill script, markdown doc examples, and SQL migration header comments.

Permission mode

Default / standard permissions (edits within the working tree).

Can you reproduce this?

Intermittently — probabilistic, not every run. More reliable with long / multi-session context (the goal crosses a compaction boundary) and many scattered negative-space items. Full protocol + objective evidence in the first comment.

Steps to reproduce

Short form (full protocol in the comment below): put a unique identifier in several distinct location-types across a multi-file repo; in one instruction, ask the agent to make it generic and remove ALL identifiers; after it reports done, grep for the identifier; observe remaining instances; point out one and observe whether it then sweeps for the rest.

Claude model

Claude Opus 4.8 (reasoning effort: high).

Relevant conversation

Spanned multiple long Claude Code sessions including context compaction. Genericized excerpts / objective evidence available on request.

Impact

High. Personal data remained in shipped source and propagated to a second machine before being caught; and the operator repeatedly became the QA backstop for an explicitly stated requirement. As an increasing share of code is AI-authored and human review thins, this failure class is invisible to exactly the fast plausible-output review that thins first — a compounding-oversight concern, not just a UX annoyance.

Claude Code version

2.1.167

Platform

Anthropic (first-party Claude Code), macOS 26.5, zsh.

Additional context

Failure class: global goal-consistency / negative-space verification failure — satisfies the local task, skips checking the artifact against the top-level goal, blind to what should be ABSENT, and persists even when the constraint is stated explicitly.

Proposed direction: not "more model judgment" but an enforced verification gate — before "done," derive the goal's invariants and check the artifact for negative-space violations ("what is present that the goal says should not be?"). Trainable (reward catching the violation in a plausible-looking artifact, not only producing good artifacts) and harness-enforceable today.

Related existing reports (corroborating the pattern, not duplicates): #17097 (does not follow prompts through completion), #21187 (ignores instructions / copies training examples), #33603 (hard rules consistently ignored, escalating), #60583 (cross-session blindness → incomplete refactor). This issue isolates the negative-space / goal-consistency angle and proposes a concrete verification gate.

Full reproduction protocol + objective evidence in the comment below.