[Bug] Fable 5 skipped EBA verification, answered pricing from priors instead of source

Bug Description
Feedback Report: Fable 5 answered a codebase-fact question from priors instead of verifying (EBA skipped)

Submitted by: dhananjaya@idasara.org
Date: 2026-07-21
Model under report: claude-fable-5
Comparison model: claude-opus-4-8[1m] (same question, same repo, correct answer)
Severity (reporter's assessment): High — burns tokens/time and produces confidently wrong answers on business-critical facts (pricing)

Environment

• Product: Idasara Academy monorepo (Claude Code)
• Project instruction file CLAUDE.md present, mandating EBA (Evidence Before Answer) as a cardinal rule: "Logs, debug output, actual

error messages / file evidence BEFORE proposing a solution."

• Question domain: current pricing/access model for a feature ("lesson notes"), which was changed recently in-repo (commit ref #15026,

dated 2026-07-20) and supersedes the older tier-gate model.

Observed behavior

• Prompt: [PASTE THE EXACT PROMPT YOU GAVE FABLE]
• Fable 5 answer: [PASTE FABLE'S VERBATIM ANSWER]
• Tool calls Fable made (if any): [e.g. "none — answered directly" / "ran grep on X"]
• Ground truth (verified by Opus via file:line reads):
• short_notes gate = -1 (free, all tiers) — feature_gates.py:418
• lesson_notes = per-subject Rs 3,000 one-time purchase, tier-independent (#15026) — student_lesson_notes.py:106-168; free-tier

students can purchase (no tier gate on NotesPackCheckoutView, notes_pack.py:41)

• Discrepancy: [State exactly how Fable's answer differed — e.g. "reported the old tier-gate model / claimed free students cannot

access, contradicting current code."]

Impact

• Confidently-wrong answers on pricing (a cashflow-critical domain).
• Wasted tokens/turns: a second model had to re-derive the correct answer from source.
• Erodes trust in the project-instruction contract: if EBA isn't honored, CLAUDE.md guarantees can't be relied on.

Hypothesized root cause (needs confirmation)

Fable 5 appears to have answered from training priors / general knowledge rather than reading the current repo, despite CLAUDE.md
mandating EBA. This is a hypothesis, not confirmed — it could alternatively be (a) CLAUDE.md not injected into that session, or (b)
correct retrieval with faulty reasoning. The transcript will disambiguate.

What would confirm it

• Fable's tool-call log: did it grep/read files at all before answering?
• Whether CLAUDE.md was in that session's context.

Requested outcome

1. Confirm whether Fable 5 received and parsed the project CLAUDE.md EBA directive in this session.
2. If received: guidance on why the directive was not applied, and whether smaller models need EBA reinforced differently (e.g.

tool-forcing) than larger ones.

1. General: is there a known gap in smaller models honoring project-instruction "verify-before-answer" contracts?

Reproduction

• Repo state: branch feat/14962-pricing-exec-summary
• Same prompt run on claude-opus-4-8 produced the correct, source-cited answer; on claude-fable-5 it did not. [Confirm this is

reproducible across ≥2 Fable runs before asserting "systematic" — currently 1 run.]

Environment Info

• Platform: linux
• Terminal: gnome-terminal
• Version: 2.1.216
• Feedback ID: daef61fd-2ff6-41cf-b93f-bff5978d42c8

Errors

[]