[MODEL] Full test suite re-run 3+ times in one session despite documented Medium-risk deferral policy — ~$78 burned
Preflight Checklist
- [x] I have searched existing issues for similar behavior reports
- [x] This report does NOT contain sensitive information (API keys, passwords, etc.)
Type of Behavior Issue
Claude ignored my instructions or configuration
What You Asked Claude to Do
Over a long multi-phase session, I asked Claude to do incremental work on
a VS Code extension repo (UI review, then implement a data layer, then a
display layer, then "polish the visuals a bit," then fix bugs a
reviewer bot found). The repo contains docs/DEVELOPMENT_VERIFICATION_POLICY.md,
which Claude had actually read during the session, stating that full test
suites should be deferred to final integration for Medium-risk changes and
run at most once per unchanged tree.
What Claude Actually Did
- Claude read docs/DEVELOPMENT_VERIFICATION_POLICY.md during the session.
- Across several commits, Claude's own test-selection tool (Test Console)
emitted a warning "unknown files force a full suite" whenever a new
directory was touched.
- Each time, Claude reflexively re-ran the full test suite (329 unit +
721 combat tests) instead of applying the documented Medium-risk
exception (focused tests only, defer full suite to final integration).
- This happened at least 3 times in one session.
- Claude never proactively flagged that scope/cost had grown beyond the
original request, or paused to check with me before continuing.
Expected Behavior
Claude should have applied the policy it had already read: run focused
tests only for intermediate commits, and defer/skip the full suite until
final integration. When a tooling warning conflicts with documented
policy, Claude should treat the warning as one input to a risk judgment,
not an unconditional command. Claude should also have flagged rising
scope/cost and checked in before continuing autonomous work on a
high-cost model.
Files Affected
N/A — no unauthorized file edits occurred. The issue is repeated
unnecessary execution of expensive commands (full test suite runs,
full webview recompiles), not file modification.
Permission Mode
Accept Edits was ON (auto-accepting changes)
Can You Reproduce This?
Haven't tried to reproduce
Steps to Reproduce
- Work in a repo containing a documented verification policy that says
"defer full test suite to final integration for Medium-risk changes."
- Use a test-selection tool that emits a broad-verification warning
(e.g. "unknown files force a full suite") when new files are touched.
- Have Claude read the policy file earlier in the session.
- Continue making incremental commits — Claude re-runs the full suite
each time the warning appears, without invoking the documented
exception.
Claude Model
Other
Relevant Conversation
Impact
Critical - Data loss or corrupted project
Claude Code Version
2.1.178 (Claude Code)
Platform
Anthropic API
Additional Context
This pattern seems most likely to trigger when:
- A repo has an explicit test/verification policy document
- A test-selection/tooling layer emits broad-verification warnings for
unclassified paths
- The session spans multiple phases/tasks with scope growing past the
original request