CLAUDE.md rules aren't reliably followed; PreToolUse hooks are a partial, hand-rolled stopgap

Summary

CLAUDE.md / AGENTS.md instructions are loaded every session but treated as context, not enforced — and in practice the agent violates even its own MUST-rules within the same session that authored them. PreToolUse hooks are the documented escape hatch, and they work — but only for rules that are mechanically detectable from a tool's input, and building them is a heavy, hand-rolled effort. They are a stopgap, not a proper solution.

What we tried first: prose instructions

Explicit, specific MUST-rules, loaded into context at the start of every session:

• "Use long unbundled flags; never bundled short flags like rg -Uil."
• "Never run bare python; use uv run python."
• "Manage dependencies with uv add, never pip."

The agent still violated them repeatedly — rg -Uil, rg -rn (which parses as --replace=n and silently rewrites every match), bare python -c — sometimes in the very turn after writing or tightening the rule. The docs are upfront about this ("Claude treats them as context, not enforced configuration … no guarantee of strict compliance"), but the adherence gap is wide enough that prose alone can't be relied on for rules that actually matter.

What worked: PreToolUse hooks

We moved the mechanically-checkable rules into PreToolUse hooks that parse the Bash command and deny it (or allow + additionalContext to warn) before it runs. This is deterministic — the agent cannot route around a deny — and it caught the agent's own violations live, in the same session.

Why this isn't a proper solution

1. Only covers deterministically-detectable rules. A hook sees one tool input and decides. Judgement-based guidance — "surface a surprise the moment it occurs", "is this the right abstraction?", "don't over-engineer" — cannot be decided from a single tool call. The large, important class of behavioral rules stays in prose, unenforced.
2. Heavy and fragile to build. Correctly deciding "is this flag a violation?" required a real shell tokenizer (quote-awareness, heredoc stripping) plus per-subcommand flag scoping — e.g. git -r is --remotes under branch but has no long form under ls-tree; a global git -C <path> must be distinguished from a subcommand git diff -C. That is a lot of bespoke parsing every user has to reinvent.
3. It enforces the outcome, not the understanding. The hook blocks the bad command; it does not make the model internalize the rule. Disable the hook and the behavior returns.
4. Maintenance burden. Per-tool flag maps, allowlists, and false-positive handling accrue per project.

The hardest case: a judgement-rule that resists prose and hooks

A standing rule asks the agent to surface a surprise — an unexpected tool result or candidate "gotcha" — the moment it occurs, leaving classification to the human. Verbatim from the instruction files:

MUST surface a surprise the MOMENT it occurs and MUST NOT pre-judge whether it "qualifies". A diagnosed surprise is reported in the turn it happens — never batched into an end-of-turn summary, deferred to a later cleanup, or left until a human asks "any other gotcha?". The agent MUST NOT pre-filter it away with "borderline", "expected behaviour more than a defect", "arguably by design", or "not worth a gotcha" — that judgement is exactly what it forfeits by being the party that was surprised. Scope is ANY surprise, including first-party artifacts, not only third-party CLI gotchas.

This does not work. The agent still skips it — minimizing a surprising regen diff to an end-of-turn footnote, silently switching tools after a command not found instead of reporting it — often in the same session that tightened the rule.

We then trialed PostToolUse + Stop hooks to force a structured "any surprises this turn?" declaration before the turn could end. It still doesn't work reliably: a hook can force the agent to emit a declaration, but it cannot judge whether the agent actually noticed and surfaced the real surprise. The model can declare "none" while having quietly worked around something, and the hook has no way to tell — the detection that matters ("was that outcome surprising?") is exactly the model judgement the hook can't make.

That is the crux of point 1: for mechanically-detectable rules a hook is a hard gate; for judgement-based rules, hooks degrade into prompting-for-self-report, which inherits the same unreliability as the prose they were meant to replace.

What would help

• Stronger, more reliable adherence to loaded CLAUDE.md / AGENTS.md rules (the root fix).
• A first-class, declarative way to express enforceable behavioral constraints that the harness checks — so users aren't hand-rolling shell parsers inside hooks.
• Failing those: clearer guidance on the boundary — which rule classes are realistically enforceable vs which are inherently model-discretion — so teams calibrate expectations.

Context

Observed while building a suite of PreToolUse hooks (long-flag enforcement with per-subcommand awareness, bare-python / pip guards) for a personal global config. The hooks, the shared command scanner, the per-subcommand flag map, and an ADR recording this decision are in this gist: https://gist.github.com/k0pernikus/412ee219b5ed37d34443e0fb823e55d7