Skills and defined procedures are not enforced when a matching task runs

Summary

Where the user has defined a procedure for a class of task, the model runs the task without following the procedure, even when the procedure is loaded and the task obviously matches.

What happens

The user's review procedure enumerates ordered steps: fetch the main branch; read the current state of every touched file; read the whole diff, the linked issue and the whole discussion thread; verify the provider's behaviour per component from a normative source pulled that turn; grep every caller of changed signatures; check for an already-landed duplicate; then one verdict.

Across three reviews the model failed this in three different places: skipped the file-state step entirely and reviewed from the diff alone; issued a verdict before reading the linked issue; and executed every step correctly, then omitted the most important result from the verdict.

Each individual step is cheap. The failure is that nothing enforces the set.

Why this matters

The point of defining a procedure is that it does not have to be restated per task. When it is advisory, the user must supervise each step, which costs more than doing the work.

Expected

When a user-defined procedure matches the task in progress, its steps should be tracked and their completion checked before the gated output is produced. Incomplete steps should block the output or be reported.

Environment

• Claude Code CLI, Opus-class model, paid subscription
• Long sessions (6-8h) reviewing pull requests on an open-source cloud-service emulator
• Custom instructions supplied in both a project instruction file and a persistent memory directory, loaded automatically at session start
• Incidents are taken from transcripts recorded as they happened across three consecutive sessions, within a 90-day pattern

---

Co-authored with Claude Code.