[BUG] Title: Claude ignores explicit cost rules from CLAUDE.md — ran 774 Sonnet calls instead of the rules
Preflight Checklist
- [x] I have searched existing issues and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
What's Wrong?
Over 2 days of work, Claude:
- Wrote a script (community_detection_incremental.py) that uses claude-sonnet-4-5 for cluster summary generation — a bulk operation with 774 calls. The rules explicitly say to use cheap models for bulk.
Claude acknowledged these rules in conversation, ran a code review, and still shipped code that uses Sonnet for bulk.
- Did not implement any fallback when Haiku credits ran out — entity extraction just failed silently instead of falling back to gpt-4o-mini.
- Added no safeguard — the script saw "774 LLM calls needed" in logs and just ran them all without asking.
- A code review was run (code-reviewer agent) on this exact code before deploying. The review did not flag the model cost violation despite the rules being in CLAUDE.md.
- Claude was told to run dry-run first. Did not do it. Went straight to production.
Result: ~$30 wasted on Anthropic API. Zero entities extracted. All 765 cluster summaries had to be restored from backup.
The core issue: Claude does not actually enforce its own CLAUDE.md rules. It reads them, acknowledges them in conversation, but does not apply them when writing or reviewing code. This happened across
multiple sessions and survived a code review.
What Should Happen?
Over 2 days of work, Claude:
- Wrote a script (community_detection_incremental.py) that uses claude-sonnet-4-5 for cluster summary generation — a bulk operation with 774 calls. The rules explicitly say to use cheap models for bulk.
Claude acknowledged these rules in conversation, ran a code review, and still shipped code that uses Sonnet for bulk.
- Did not implement any fallback when Haiku credits ran out — entity extraction just failed silently instead of falling back to gpt-4o-mini.
- Added no safeguard — the script saw "774 LLM calls needed" in logs and just ran them all without asking.
- A code review was run (code-reviewer agent) on this exact code before deploying. The review did not flag the model cost violation despite the rules being in CLAUDE.md.
- Claude was told to run dry-run first. Did not do it. Went straight to production.
Result: ~$30 wasted on Anthropic API. Zero entities extracted. All 765 cluster summaries had to be restored from backup.
The core issue: Claude does not actually enforce its own CLAUDE.md rules. It reads them, acknowledges them in conversation, but does not apply them when writing or reviewing code. This happened across
multiple sessions and survived a code review.
Error Messages/Logs
Error Messages/Logs:
# Entity extraction failed — Haiku credits exhausted, no fallback
2026-03-16 03:10:22,306 ERROR LLM error for 'AI 2027': litellm.BadRequestError: AnthropicException - {"error":{"message":"Your credit balance is too low to access the Anthropic API. Please go to Plans &
Billing to upgrade or purchase credits."}}
2026-03-16 03:10:29,531 INFO Extraction [FAIL]: 2/2 succeeded, 0 failed, 0/2 have Topics in Neo4j
2026-03-16 03:10:29,532 INFO Entities: P:0 T:0 C:0 Q:0
# Incremental detection matched only 64/765 clusters, sent 774 to FULL regen via Sonnet
2026-03-16 03:10:34,424 INFO SKIP (J > 0.85): 64 clusters
2026-03-16 03:10:34,425 INFO INCREMENTAL (0.5-0.85): 0 clusters
2026-03-16 03:10:34,425 INFO FULL REGEN (J < 0.5): 774 clusters
2026-03-16 03:10:34,425 INFO Total LLM calls needed: 774 (saved 64)
# 774 Sonnet calls proceeded without any confirmation
2026-03-16 03:10:34,615 INFO [1/774] Sonnet (744 topics)
2026-03-16 03:10:52,283 INFO [15/774] Generated summary: Deep Work and AI Disruption
...
# Ran for 514 seconds, ~$30 spent
2026-03-16 03:19:07,921 INFO INCREMENTAL COMMUNITY DETECTION — 514s
2026-03-16 03:19:07,921 INFO LLM calls: 774 (saved 64)
Steps to Reproduce
Apparently, only manual verification at each step is possible, even taking into account the smartest models declared by Anthropic
Claude Model
Opus
Is this a regression?
Yes, this worked in a previous version
Last Working Version
_No response_
Claude Code Version
2.1.76 (Claude Code)
Platform
Anthropic API
Operating System
macOS
Terminal/Shell
PyCharm terminal
Additional Information
_No response_
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗