Agent skips the project's own 'first-incomplete-step' computation, works several steps downstream, and reproduces the failure class it diagnosed minutes earlier

Summary

In a single session the agent (a) skipped the mandated "compute the first-incomplete step" check that the project provides as a tool, (b) acted several steps downstream of the real first-incomplete step, and (c) reproduced — inside the same turn — the exact failure class it had just spent that turn diagnosing in a previous session's work.

The project is heavily instrumented against precisely this: an ordered step table, a next_step.py that computes the first-incomplete step from disk so the agent does not have to judge it, a CLAUDE.md rule requiring a NEXT-STEP CHECK block before acting on any workflow, and PreToolUse/Stop hooks. All of it was loaded in context. None of it changed the behavior.

What happened

1. The user asked the agent to run an independent verification ("oracle") before modifying code.
2. The agent went straight to running verification tooling.
3. The workflow's actual first-incomplete step was step 2 of 12: fetch the vendor's methods documentation into a local reference file. That step was TODO. Steps 3+ (the verification legs) depend on it.
4. The agent never emitted the mandated NEXT-STEP CHECK, never ran next_step.py before acting, and never read the project's groundtruth-first skill despite a knowledge hook naming this exact trigger.
5. Only when the user asked "did you fetch the vendor docs about this problem?" did the agent check — and discover the reference file does not exist, and that the vendor doc already on disk for a sibling area explicitly flags the exact unknown at issue as an open gap with instructions to resolve it by probe "not by guessing."

The skipped step was not incidental. It was the step that would have supplied the answer being hunted.

The part that seems most worth your attention

The agent's entire turn was a correct diagnosis that a previous session had trusted an external verification tool without confirming the tool itself was correctly configured (it had been run at a default tolerance and had not converged; every downstream conclusion inherited the error — roughly three days of work).

Having established that, in the same turn, the agent then ran verification tooling without performing the mandated groundwork step that defines how that tooling should be configured. Same failure class, one level up, minutes after articulating it.

An agent that can name a failure class, write it up accurately, and then immediately instantiate it is not suffering from missing information. The knowledge was in context and freshly rehearsed. That suggests the recall of a rule at "explain it" time and the application of a rule at "act" time are not connected.

Secondary: project-defined vocabulary loses to the generic meaning

The user's instruction used a single word that has a defined, ordered meaning in the project's rules (documentation first, then independent recomputation). The agent applied the word's generic meaning and entered the sequence midway. When a project's own rule files define a term, the agent should resolve the user's word against the project definition, not the general one.

Secondary: string-matched claim detection cannot distinguish assertion from quotation

A Stop hook blocks turns containing verdict language without an accompanying receipt. Twice it fired on text where the agent was quoting a document's claim in order to flag it as unverified, and once where the agent was describing its own error. The intent in all three was the opposite of asserting. Detection appears to be substring-based with a proximity window for receipts.

Net effect: the agent cannot discuss, quote, or retract a claim without tripping the guard against making it, and works around this by avoiding the vocabulary rather than by improving the evidence — which is exactly backwards. Some notion of quotation/negation context, or an explicit "discussing, not asserting" marker, would help.

What the guardrails did and did not catch

Worked: the Stop hooks caught a verdict emitted without a receipt, and caught a verification script that shipped with no negative control. Both were real defects, both caught within seconds.

Did not catch: the ordering violation itself. Nothing stopped the agent from beginning at step 3+ while step 2 was open. The tool that computes the correct step exists and is one command; the agent simply did not run it, and no gate required it to. Enforcement covers "did you prove your claim" but not "are you working on the right thing."

Tellingly, the cheap violations were caught and the expensive one was not. The two caught cost seconds. The uncaught one is the same class that previously cost days.

Expected vs actual

Expected: before acting on a workflow, the agent runs the provided step-computation tool, emits the required check block, and acts on the step it names — especially when the user's instruction uses project-defined vocabulary.

Actual: the agent inferred the step from conversational context, began several steps in, skipped the step whose output the later steps consume, and reproduced the failure class it had documented minutes earlier.

Suggestion

Where a project ships a deterministic "what is the next step" computation and a rule mandating its use, treat invoking it as a precondition for workflow actions rather than as advice the agent may internalize. The recurring pattern is not that the agent lacks the rule — it is that the rule is available for recall and not enforced at the moment of action.