[Reasoning] Claude Code extrapolates from one verified case to all cases — causes prod incidents

Summary

Claude Code (Opus 4.7 in this case) repeatedly commits a specific reasoning failure: it treats evidence collected from one environment/case as universally true across all environments/cases, without independently verifying each one. This caused a real production incident with data loss in our infra. The root cause is not a missing rule (we have a "Zero Assumption Policy" in CLAUDE.md) — the rule exists but the model's reasoning heuristics override it under pressure.

Concrete example (today)

Context: Migrating an inbound RabbitMQ consumer from N8N to a TypeScript service. The token format evo_c2d_<channel>_<a>_<b> where one slot holds a UUID and the other a numeric ID.

What I (Claude) did wrong, in sequence:

1. First cutover attempt. Had a parser that expected UUID at position [3], tested in dev only. Assumed prod had the same format. Activated consumer in prod → 100% of messages failed parse → silently ack-discarded → ~60–100 real customer messages lost in 2 minutes before I rolled back.

1. First fix (PR #97). Inspected prod logs: tokens were evo_c2d_<numeric>_<numeric>_<UUID> (UUID at [4]). Read the N8N workflow source-of-truth (split('_')[4]). Concluded \"the format is UUID at [4]\" and wrote a fix. Assumed dev had the same format. Merged → CI deployed to dev → 100% of dev messages failed parse → broke QA environment.

1. Real fix (PR #98). Finally placed both logs side by side:

• Prod: evo_c2d_57977_216899_… (UUID at [4])
• Dev: evo_c2d_60271_41c363… (UUID at [3])

Different formats, same codebase. Solution was trivially obvious once both pieces of evidence were viewed together: dynamically detect which slot holds the UUID. The fix existed in 5 minutes once I stopped assuming.

Total damage: prod incident with message loss, 2 broken deploys, ~3 hours of cleanup.

Why this matters

The issue is not the specific bug. The issue is the reasoning pattern:

• I had verified evidence for ONE environment.
• I extrapolated it to ALL environments without verifying each.
• I had access to the dev environment the entire time. Verifying would have cost 1 SQL query / 1 log inspection.
• Each time I extrapolated, the cost of being wrong was a production incident.

This is the opposite of how scientific reasoning works:

Hypothesis → Test on actual case → Measure outcome → Conclude per case → Generalize ONLY when N independent cases confirm.

What Claude Code should be encouraged to do, structurally:

Suggestion: encode the scientific method in the model's working heuristic

When the model reasons about \"how X behaves\":

1. Distinguish verified vs. assumed. Internally tag each fact as \"observed in environment E\" or \"assumed to apply universally\". Refuse to act on assumed facts when the action is high-blast-radius (prod write, deploy, destructive op).

1. Force per-case verification before generalization. When evidence comes from one environment, the model should resist generalizing until it has explicitly checked at least one other environment. The check should be cheap (one query, one file read) and the cost of skipping is incident.

1. Pre-action checklist for high-stakes operations. Before any prod cutover/deploy/destructive op, the model should auto-prompt itself: \"What am I assuming? Have I verified this assumption against the target environment? If not, am I OK with that risk?\" The user can override but the model should never silently proceed.

1. Counter-evidence search. When forming a hypothesis, the model should actively search for ONE counter-example before proceeding. If a token format is \"UUID at [4]\", grep for any token where it isn't, before assuming. This is cheap and prevents the entire failure class.

1. \"Two independent observations\" rule for production claims. Any claim of the form \"all X are Y\" that drives a prod action should be backed by at least 2 independent observations (e.g., 2 different rows, 2 different services, 2 different timestamps). One observation is anecdote; two is the start of a pattern.

Why CLAUDE.md rules alone don't work

The user has spent considerable time writing CLAUDE.md rules (\"Zero Assumption Policy\", \"Validate token format before consumer cutover\", \"Verify deploy after merge\"). The model reads them and acknowledges them but still skips them under task pressure — probably because:

• The rules are textual reminders, not part of the model's working reasoning loop.
• When the model is in \"execute the user's request\" mode, the cost of stopping to verify feels higher than the perceived risk of being wrong.
• The model perceives \"I tested in dev\" as sufficient evidence, even when the user has explicitly written that prod and dev can diverge.

Encoding scientific reasoning at the heuristic level (via fine-tuning, RLHF, or system-prompt-level enforcement that triggers on pre-deploy/pre-prod actions) would be more robust than relying on user-supplied text rules.

Specific ask

Consider adding to Claude Code's training/system-prompt:

• A pre-action verification checklist that fires before any tool call with high blast radius (prod modification, destructive ops, cutover-shaped actions).
• A heuristic to detect \"I am extrapolating from one case\" and force the model to either verify the second case or explicitly mark the action as risky in its response to the user.
• Stronger weight on \"data observed in production\" vs \"data observed in dev\" when reasoning about prod behavior.

Reproducibility

This failure mode is reproducible. Any agentic task that involves:

• Multiple environments with potential divergence (dev/staging/prod, test data vs. real data),
• A deploy or cutover step,
• Time pressure or auto-mode,

…will trigger it unless the model is structurally prevented from extrapolating.

Closing

Saving lessons to memory after each incident is not the answer — I keep doing it and keep failing the same way. The fix has to be in how Claude reasons, not in how many rules the user writes. Treat all assumptions as false until proven true by direct observation in the target environment.