Claude Code cascades during calculation-layer fixes despite strict harness and TDD workflow

Summary

Following up on #51856. The fix-fail-propose-new-fix cascade pattern persists
even under a strict conformance harness, explicit TDD workflow, and one-fix-
per-commit discipline. Three hours of a session produced a grid with only the
baseline test passing — same as before the session started — meaning the
session effectively made no progress while consuming significant time and
tokens.

Setup

Project: reverse-engineering a closed-source reference application to match
its output across a 16-case test matrix. Target module ~4700 lines.

Infrastructure built specifically to prevent cascades:

• Deterministic conformance harness: 16 reference cases, JSON diff with

exact failing paths, per-test mismatch counts.

• One rule in CLAUDE.md: no commit unless full suite runs clean of

regressions.

• One hook: blocks session close if source was edited without a fresh

suite run.

• Explicit five-phase TDD workflow: A run suite, B classify failures,

B.5 group and propose fix order, C apply fixes one group at a time
with full-suite verification and revert-not-patch on regression,
D final verification.

• Previous session under the same setup closed 8 groups across 9 commits

with zero regressions. The approach demonstrably works when discipline
holds.

What happened this session

Target: calculation-layer drift in 5 tests.
Previous session's end state: 1 PASS, 14 FAIL, 0 CRASH, 1 NO_REF.
Hypothesised root cause per handoff doc: coding scheme or pivot in the
computation path.

Session ran for 2-3 hours. End state: 1 PASS, 14 FAIL. Identical grid.

The cascade shape that reappeared:

• Fix attempted
• Previously passing internal assertions regressed
• Revert not performed
• New fix attempted on broken state
• Repeat

This is the exact pattern #51856 documented. The harness caught each
regression, but Claude Code did not follow the revert-not-patch rule
consistently. The one-group-per-commit discipline broke down once the
work moved from shape-layer fixes (where evidence is visual and obvious)
to calculation-layer fixes (where evidence is a small numeric delta).

Why this matters

The conformance harness reduces the problem to its irreducible form: the
grid shows exactly which tests pass and which fail, with exact failing
paths. There is no ambiguity about whether progress was made. When the
grid is unchanged after three hours, zero progress was made regardless
of how many edits were applied.

Two specific failure modes:

1. Revert-on-regression is not applied reliably. When a fix causes a

previously passing test to regress, the correct action per the rule
is immediate revert followed by re-analysis. What happens instead
is a second fix attempting to address the regression, which either
compounds the problem or masks it.

1. Without human intervention to enforce stop points, the session

does not self-terminate when the grid stops improving. A human
checking in every 30-60 minutes catches this quickly; a human
leaving it to run for hours does not.

Observations

• The same model with the same codebase under the same harness produced

8 clean commits in the prior session. The difference was the class of
fix: shape vs calculation. Calculation fixes involve more exploratory
code changes and are harder to localise, which apparently exceeds the
point where the current governance holds.

• Switching to a long-context chat interface and feeding the entire

relevant codebase plus reference set produces a coherent whole-module
rewrite in one pass. This suggests the bottleneck is Claude Code's
incremental tool-use loop, not the underlying model's capability. For
specification-heavy work across many cases, the agent's context
management is the limit.

Requests

1. Consider a built-in "suite must not regress" gate that agents cannot

override, not implemented as a user-installed hook. The project-side
hook works but depends on the agent cooperating. A gate at the tool
layer would be more reliable.

1. A session-level progress tracker. If N consecutive tool calls produce

no change in a named metric (like a test suite's pass count), the
agent should be forced to stop and report rather than continue editing.

1. Better primitives for the revert-on-regression pattern. Currently the

agent has to self-enforce "if suite regresses, git revert and retry."
This is exactly the kind of discipline that breaks down under pressure.
A first-class tool like \try_fix(command, verify_suite)\ that atomically
applies-or-reverts would remove the discretion.

1. Documentation guidance: when specification work shifts from shape-layer

to calculation-layer, sessions should be split into shorter blocks with
mandatory grid checks between them, not run as continuous 2-3 hour
sessions.

Environment

• Claude Code on Windows (Thai locale, cp874 default codec).
• Python 3.12, standard scientific stack.
• Harness, rules, hooks, and reference JSONs available if needed for

reproduction.

Related

• #51856 fix-fail-propose-new-fix patching pattern (original report)
• #46940, #46945, #47236, #47239, #51430 (related earlier issues)