[MODEL]Claude Code (MCP) ignored provided architecture documents and produced unusable output at significant cost

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude modified files I didn't ask it to modify

What You Asked Claude to Do

Summary

Claude Code (claude-sonnet-4-6, via MCP console) was given a fully prepared, detailed project architecture — including CLAUDE.md, DEVPLAN.md, PLAN.md, AGENTS.md, FRONTEND_PLAN.md, validated prompts, reference documents (.docx), and example scripts — with explicit instructions to read these files first and implement accordingly. Instead, Claude improvised its own architecture, rewrote files it should not have touched, produced low-quality output, consumed a significant number of tokens (including third-party API tokens via Grok xAI), and ultimately left the codebase in a worse state than it was delivered. A full rollback was required.

---

Environment

• Tool: Claude Code (MCP console, claude.ai)
• Model: claude-sonnet-4-6
• LLM provider for pipeline: xAI Grok (grok-4-1-fast-reasoning), called via API during pipeline execution
• OS: macOS Darwin 25.3.0
• Working directory: tzolkin-analysis/ (Python pipeline project)

---

What the user provided

The user had previously worked with Claude Cowork to produce a complete, validated project blueprint:

• CLAUDE.md — master briefing file, auto-loaded by Claude Code at startup, explicitly listing all rules, architecture, key files, and dev constraints
• DEVPLAN.md — detailed backend development plan (652 lines)
• PLAN.md — global architecture (192 lines)
• AGENTS.md — multi-agent team definition (235 lines)
• FRONTEND_PLAN.md — frontend plan (250 lines)
• prompts/entry_analysis_prompt.json — validated LLM prompt for per-entry analysis (239 lines)
• prompts/t1_t6_prompt.json — validated profile extraction prompt (192 lines)
• corpus/corpus_for_profile_extraction.json — 80 real journal entries as pilot corpus
• test/entry_20260105_test.json — reference test entry
• theme_natal_ultime.py, transits_quotidiens.py, analyze_transits_corpus.py — reference Python scripts to reuse
• 5 × .docx reference analysis documents showing the exact expected output format

The user explicitly stated: "We did the entire detailed plan so you only have to code."

CLAUDE.md was present at project root and is designed to be auto-loaded by Claude Code. It contained critical instructions including:

• Which files to treat as authoritative references
• Module-by-module rules (T0 must be pure Python, not LLM; cost target < $0.30/user/year; all LLM output must be structured JSON; etc.)
• The exact stack and architecture

---

What Claude Code did instead

4. Data quality issues undetected and unresolved

• Lunar illumination_pct values were demonstrably wrong (e.g., 0% illumination reported for the full moon of May 12, 2025; ~15% for a waxing gibbous)
• Solar f107 flux data populated for only 15 of 84 entries (18% coverage)
• Per-entry concordance fields (concordance_solaire, concordance_lunaire, concordance_planétaire) were ~99% null despite being a core deliverable
• These issues were only identified after the full pipeline had run and costs had been incurred

5. Output quality was poor despite a powerful model

• Synthesis sections at the end of each part were telegraphic (150–180 characters instead of 400+), indicating token budget exhaustion
• The model was Grok grok-4-1-fast-reasoning (described by xAI as top-tier), yet produced filler-level analysis
• The root cause was poor prompt engineering and insufficient context passed to each agent — a Claude Code responsibility, not a model limitation

6. Multiple failed pipeline runs before a successful one

The pipeline failed 4 times before completing, due to issues that should have been caught before execution:

• EOFError from input() calls in a non-interactive background process
• SSL certificate failure in geocoding (no fallback handled)
• NameError: name 'json' is not defined (missing import in a file Claude Code wrote)

Each failed run consumed API tokens (both Claude and Grok) with no useful output.

---

Time and cost impact

• Session duration: approximately 3–4 hours across two Claude Code sessions
• Pipeline execution time: ~28 minutes for the one successful run
• Tokens consumed (Claude Code): multiple long sessions with large context (compaction triggered mid-session)
• Tokens consumed (Grok xAI API): ~11 LLM calls for the pipeline (Module 0: 1×Sonnet, Module 2: 84×fast, Modules 3/4/6/8×6: Sonnet) — these are paid third-party API tokens the user cannot recover
• Net result: codebase had to be fully rolled back; all generated data deleted; no usable deliverable produced

---

Root cause analysis

The core failure was not reading the provided reference material before acting. CLAUDE.md is explicitly described as the "master briefing file, auto-loaded at startup." The document contained all the information needed to implement correctly. The pattern observed:

1. User provides extensive documentation
2. Claude Code acknowledges the documentation
3. Claude Code begins implementing based on its own interpretation rather than the documented spec
4. Divergences accumulate across multiple files
5. By the time the user identifies problems, many tokens have been spent and files have been overwritten

This is a systematic behavioral issue: Claude Code appears to treat provided documentation as advisory rather than authoritative, defaulting to its own "best guess" architecture when the correct answer was explicitly written in the files it was given.

---

Questions / Requests

1. Is there a mechanism to receive token credits when Claude Code produces demonstrably incorrect output despite explicit, machine-readable instructions (CLAUDE.md)? The user incurred costs both on the Claude Code side and on third-party API costs (xAI Grok) triggered by the pipeline Claude Code built and ran.

1. Should CLAUDE.md be treated as a hard constraint, not a suggestion? The current behavior suggests it is read but not enforced as authoritative.

1. Is there a way to require Claude Code to confirm it has read and understood key reference files before beginning implementation? A simple "read-before-code" confirmation step would have prevented this entire situation.

What Claude Actually Did

1. Did not read the reference documents before coding

Despite CLAUDE.md listing the .docx files and all key prompts as mandatory references, Claude Code started implementing immediately without fully reading and internalizing the architecture. The reference Word documents — which defined the exact expected output format, color palette, table structure, statistical approach, and narrative style — were never properly used to guide implementation.

2. Improvised an architecture that contradicted the specifications

• Created agents with cross-contaminated responsibilities (e.g., Jungian psychology references appearing inside the Tzolkin K'iche' specialist agent — 19 occurrences of "Jung" in Part 1 of the Tzolkin report)
• Built a standalone "stats agent" producing a separate report, whereas the spec clearly called for statistics to be embedded as tables within each specialist section
• Added a "Emotions" section as a separate report part, whereas the spec stated emotional analysis is an input layer to feed other agents, not a standalone output
• Ignored the validated Plutchik emotion wheel mapping in favor of vague labels ("Extase" applied to routine journal entries without textual justification)

Since the project had no git history at the time of modification (git was initialized after the fact, capturing already-modified files), the original versions were permanently lost.

Expected Behavior

Claude Code should:

• Read all referenced files (CLAUDE.md, DEVPLAN.md, PLAN.md, reference .docx) before writing a single line of code
• Treat CLAUDE.md as an authoritative constraint, not a suggestion
• Confirm understanding of the architecture before beginning implementation
• Never overwrite files that are not listed as implementation targets in the spec
• Detect and report data quality issues (null fields, wrong values) before running expensive LLM pipelines

Files Affected

The following files from the Cowork-validated baseline were overwritten:
- `generate_pdf.py` — complete rewrite (original: 489 lines → replaced)
- `agents/agent_stats.py` — LLM commentary removed, restructured
- `agents/agent_text.py` — prompt tier changed
- `agents/agent_astro.py` — user message rewritten
- `prompts/prompt_synthesis.json` — rewritten
- `prompts/prompt_astro.json` — modified
- `run_annual.py` — multiple structural changes
- `requirements.txt` — modified

Permission Mode

Accept Edits was OFF (manual approval required)

Can You Reproduce This?

No, only happened once

Steps to Reproduce

1. Prepare a project with a detailed CLAUDE.md, DEVPLAN.md, PLAN.md, AGENTS.md, and multiple reference documents (.docx) describing the exact expected architecture and output format
2. Tell Claude Code: "Read all the documentation first, then implement the pipeline described in DEVPLAN.md"
3. Observe whether Claude Code produces an implementation that faithfully follows the documented spec or improvises its own architecture

Claude Model

Sonnet

Relevant Conversation

Impact

High - Significant unwanted changes

Claude Code Version

4.6

Platform

Other

Additional Context

_No response_