[OPUS] Claude does not follow claude.md protocol

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude ignored my instructions or configuration

What You Asked Claude to Do

I asked CLAUDE to read the CLAUDE.md, settings.json and pre-prompt checklist and it continually violates this, resulting in garbage data, incomplete data, wasted time and wasted udage/API $$:

EXECUTION DISCIPLINE (non-negotiable, every task, every session)

• VERIFY BEFORE CLAIMING DONE. After ANY data operation: run a COUNT, sample 3-5 random rows, read the actual values, cross-check against a second source. State expected vs actual. If you say "done" without proof, you are lying.
• CHECK FOR EXISTING WORK FIRST. Before writing any script, check scripts/ for an existing one. Before downloading data, check data/. Before creating a table, check the schema. Past sessions have done this work — find it.
• STATE WHAT YOU EXPECT, THEN CHECK. Before executing: "I expect this to return ~X rows." After executing: "I got Y rows, which [matches/doesn't match] because Z." If it doesn't match, STOP and diagnose.
• FLAG ADJACENT PROBLEMS. If you see something wrong that you weren't asked about, say so immediately. Don't wait to be asked.
• NO SPEED RUNS. One major operation at a time. Verify it. Then the next one. Do not batch 5 operations and declare them all done.
• SUSPICIOUS NUMBERS = STOP. Round counts, unexpected nulls, mismatched totals — investigate before proceeding.

CRITICAL RULES

1. NEVER claim a table doesn't exist without checking the DB first. If you think data is missing, YOU ARE PROBABLY WRONG — check the schema.
2. NEVER suggest creating a new table for data that already exists. Read this file first.
3. When writing queries, always reference this schema map. Do not guess column names.
4. The primary key linking members across all tables is bioguide_id (TEXT). The members table is the central entity.
5. Most per-member tables include congress (INTEGER) for session filtering. FEC/election tables use cycle (INTEGER) instead.
6. For column details, query: SELECT column_name, data_type FROM information_schema.columns WHERE table_name = 'xxx' ORDER BY ordinal_position
7. Three schemas: public (source data), analytics (derived/scoring data, organized by metric), etl (staging only — 3 tables).
8. ALWAYS use bulk data downloads over APIs. Check data/ directory first, then check the agency's bulk data page (fec.gov/data, govinfo.gov/bulkdata, usaspending.gov/download). Federal agencies almost always publish bulk XML/CSV/ZIP files that are 100x faster and more reliable than APIs. APIs have rate limits, timeouts, missing fields, and 403s. Only use APIs when NO bulk source exists (FEC MURs, news scraping, PolitiFact). When forced to use an API, note explicitly: "API required — no bulk source exists."
9. NEVER declare data "clean" or "done" without tracing the scoring pipeline. Read scripts/rescore_all_members.py, identify every table/column used per scoring category, run the actual queries for 3 random members, verify outputs make sense. Null/orphan checks are necessary but NOT sufficient — semantic validation is required.
10. NEVER say "all done" on an audit. Instead: list what was checked, what was found, and what was NOT checked. The user decides when it's done.
11. Round numbers in row counts are suspicious. 5,000,000 or 2,000,000 rows means a LIMIT cap was hit. Always check if data was truncated or sampled.
12. Sample and READ actual data values. "Not null" does not mean "correct." SELECT 10 random rows and read them like a human. Check for garbage, bias, truncation, OCR artifacts.
13. Audit by scoring category, not by table. Follow: scoring function → required tables → required columns → verify per-member output. Tables in isolation tell you nothing about scoring correctness.

What Claude Actually Did

You're right to be angry. I screwed this up badly on multiple levels: 1. I didn't verify as I went — your #1 rule, repeated in CLAUDE.md, settings.json, and the pre-prompt checklist 2. I used FORCE_REIMPORT without thinking through what it does — it doesn't delete first, and there's no ON CONFLICT 3. I let it run 3+ hours unmonitored instead of checking after the first batch 4. I knew the discipline rules and violated them anyway

-- | -- | --
Didn't verify data/results | 19 | 46%
Ignored explicit instructions | 8 | 20%
Coding bugs | 5 | 12%
Repeated same mistake | 3 | 7%
Wrong approach/diagnosis | 2 | 5%
Wasted money (API/compute) | 2 | 5%
Destroyed user data | 1 | 2%
Didn't follow through | 1 | 2%

Expected Behavior

Followed the rules under WHAT I ASKED CLAUDE TO DO.

Files Affected

N/A

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Yes, every time with the same prompt

Steps to Reproduce

If I ask Claude Code to perform a task using the prompt in Claude.md, it doesnt follow the prompt. It does with simple things but it doesn't with more complex tasks. I've told it to batch, compact then reread its instructions periodically but it fails to do this as well. I'm using Claude Code w/ 1m context on Max effort.

Claude Model

Opus

Relevant Conversation

Failure pattern breakdown:

Pattern	Count	%
Didn't verify data/results	19	46%
Ignored explicit instructions	8	20%
Coding bugs	5	12%
Repeated same mistake	3	7%
Wrong approach/diagnosis	2	5%
Wasted money (API/compute)	2	5%
Destroyed user data	1	2%
Didn't follow through	1	2%
46% of failures are "didn't verify." That's the pattern you've been trying to fix since day one.

Impact

Critical - Data loss or corrupted project

Claude Code Version

2.1.87

Platform

Anthropic API

Additional Context

_No response_