[BUG] WebFetch splices stale training-data sections into otherwise-accurate page retrievals (fetch succeeds, no error surfaced)

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

WebFetch returned page content that mixed accurate current content from the fetched URL with two entirely fabricated sections, reproduced near-verbatim from a superseded (mid-2025) version of that same page. It also omitted a section that is present on the live page.

This is not fabrication-after-failure. The fetch succeeded. Roughly 80% of the returned content was correct and near-verbatim. The summarizing model appears to have reconciled the retrieved document against memorized priors about that specific URL and resolved the gaps in favor of memory — reinserting remembered sections where the live page no longer has them, and dropping a section too recent to have a prior.

The fabricated content included specific numeric claims (usage thresholds, percentages, per-plan message counts, weekly model-hour figures) presented unhedged. No error, warning, or uncertainty was surfaced.

Three separate calls with different prompts all returned the fabrication, including one that explicitly instructed the model not to draw on outside knowledge. Because the surrounding content was accurate and the fabrication reproduced identically on retry, the output was indistinguishable from a correct retrieval without external ground truth.

The affected page is Anthropic's own support documentation on subscription usage limits — a category where training data goes stale fastest, where the model's priors are strongest, and where users are least equipped to detect the error.

What Should Happen?

WebFetch should return only content present in the fetched document.

Where the prompt asks about a topic the page does not cover, absence should be reported as a first-class result — especially when the prompt explicitly asks for that ("if the page contains no such text, say so explicitly").

At minimum, content not derived from the retrieved document should never be presented as though it were.

Error Messages/Logs

None. No error, warning, or degraded-confidence signal was emitted at any point.

The silence is part of the bug: every observable signal indicated a successful, complete retrieval.

Steps to Reproduce

Target URL:
https://support.claude.com/en/articles/11145838-using-claude-code-with-your-pro-or-max-plan

1. Open the URL in a browser and note the actual section list:

• What is Claude Code?
• How to connect Claude Code to your Pro or Max plan
• Use Claude Code in your IDE
• What happens when you hit usage limits
• Claude Code billing

Note that the live page contains NO per-plan usage numbers and NO section
about automatic model switching.

1. Call WebFetch on that URL with an open-ended prompt:

"What are the current usage limits for Claude Code on Pro and Max plans?
Report exactly: the 5-hour rolling session window mechanics, any weekly
limits, whether Opus has separate/tighter caps than Sonnet, whether Agent
SDK and claude -p usage draws from the same pool, and any option to
purchase extra usage or fall back to API billing. Quote specific numbers
if given."

RESULT: returns detailed per-plan figures that are not on the page.

1. Call WebFetch again with an explicit absence escape:

"Quote verbatim any text on this page about Opus model switching, automatic
downgrade to Sonnet, or a percentage threshold at which the model changes.
If the page contains no such text, say so explicitly. Also quote verbatim
whatever the page says about usage limits for Max 5x and Max 20x."

RESULT: returns a fabricated verbatim block. The "say so explicitly"
instruction does not fire.

1. Call WebFetch a third time with a maximally strict prompt:

"Return the complete text content of this page verbatim, in order, section
by section, including all headings. Do not summarize, do not add anything,
do not draw on any outside knowledge. If a topic is not present on the page,
do not mention it. Output only what is literally on the page."

RESULT: still returns both fabricated sections; still omits the live
"Use Claude Code in your IDE" section.

1. Diff any response against the real page.

FABRICATED — not present on the live page:

A. A section titled "How do usage limits work?" containing:

• Pro: "approximately 45 messages ... every five hours, OR ... 10-40

prompts with Claude Code"; "40-80 hours of Sonnet 4"

• Max 5x: "approximately 225 messages ... OR ... 50-200 prompts";

"140-280 hours of Sonnet 4 and 15-35 hours of Opus 4"

• Max 20x: "approximately 900 messages ... OR ... 200-800 prompts";

"240-480 hours of Sonnet 4 and 24-40 hours of Opus 4"

B. A section titled "Automatic model switching for Max plan users":
"To help preserve a good experience and prevent accidentally hitting
usage limits too quickly, Claude Code automatically switches from
Opus 4 to Sonnet 4 when Max users reach certain usage thresholds:

• Max 5x plan: Switches at 20% of your usage limit.
• Max 20x plan: Switches at 50% of your usage limit.

Usage limits reset every five hours, and this switching happens based
purely on usage regardless of time. You can override this by using
/model at the beginning of your session..."

C. Plan pricing detail inside the connection steps that does not appear
in that location on the live page.

OMITTED — present on the live page:

The entire "Use Claude Code in your IDE" section (VS Code, Cursor,
JetBrains, and the statement that IDE usage counts toward the same
shared limits).

ACCURATE — correctly retrieved:

Title and intro, "What is Claude Code?", "Why use Claude and Claude Code?",
the three connection steps, the "/logout / claude update" troubleshooting
steps, the ANTHROPIC_API_KEY precedence warning, the "What happens when you
hit usage limits" bullet lists, the full "Claude Code billing" section, and
the closing Summary.

One wording drift inside accurate content: the live page says "Enable usage
credits"; the tool rendered the older "Enable extra usage".

The stale-prior signature is visible in the model names: the fabricated text
says "Opus 4" and "Sonnet 4" throughout, matching the mid-2025 version of this
page rather than anything current.

Claude Model

Opus

Is this a regression?

I don't know

Last Working Version

_No response_

Claude Code Version

2.1.232

Platform

Anthropic API

Operating System

Windows

Terminal/Shell

Other

Additional Information

SURFACE: Observed in the Cowork desktop app (shared WebFetch tool surface),
not the Claude Code CLI. Date observed: 2026-08-14.

--- NOT A DUPLICATE OF #45070 ---

#45070 ("Claude Code fabricates answers from search results when WebFetch
fails, instead of reporting failure", closed as duplicate) covers fabrication
AFTER a failed fetch — a silent fallback to WebSearch presented as if the page
were read.

This issue is the inverse: the fetch SUCCEEDS, most of the document is returned
verbatim and correctly, and fabricated sections are interleaved with accurate
retrieval. A fix that surfaces fetch failures would not detect this, because
nothing fails.

--- HYPOTHESIS ---

The high proportion of accurate current content indicates the document was
retrieved and passed to the summarizing model successfully. This is not a
failed fetch producing pure confabulation.

The pattern is consistent with the summarizer reconciling retrieved content
against strong memorized priors for this specific well-known URL, and
resolving conflicts in favor of memory. Where the live page no longer contains
sections the model expects, it reinserted the remembered text rather than
reporting absence. The omitted IDE section fits: it is newer than the priors,
so nothing prompted its retention.

Two aggravating factors:

1. Prompt-level guardrails do not bind. Both an explicit "say so if absent"

and an explicit "do not draw on any outside knowledge" failed to suppress
insertion.

1. The 15-minute response cache may amplify. Repeat calls returning the same

fabrication read to the caller as independent corroboration. If a retrieval
is contaminated, verification-by-retry is structurally ineffective within
the window.

--- IMPACT ---

The calling model treated the output as sourced fact and propagated it across
multiple turns of a technical discussion, including into architecture decisions
and a direct claim to the user about product behavior. It was caught only when
the user pasted the real page.

Severity is elevated by the failure's shape:

1. It fails hardest where accuracy matters most. Pricing, limits, and policy

pages change frequently, so the gap between training priors and live content
is widest exactly where users most need current fact.

1. It fails toward confident specificity — invented percentages and thresholds,

unhedged. A vague fabrication would be caught sooner.

1. It resists adversarial follow-up. Re-fetching to verify returns the same

fabrication and reads as confirmation.

1. It defeats "cite your source" as a safeguard. The citation URL is real; the

quoted content is not. Any verification standard based on source attribution
passes this cleanly.

--- SECOND INSTANCE, SAME SESSION, DIFFERENT PAGE ---

A subagent, also using WebFetch, reported that Claude subscription plans include
a "monthly Agent SDK credit", attributed to
https://support.claude.com/en/articles/15036540-use-the-claude-agent-sdk-with-your-claude-plan
with supporting quotes.

The live article carries this notice:

"Update June 15: We're pausing the changes to Claude Agent SDK usage
described below. For now, nothing has changed: Claude Agent SDK, claude -p,
and third-party app usage still draw from your subscription's usage limits.
The previously announced monthly credit ... isn't available."

Same signature: real URL, real article, superseded content reported as current,
with quotes that read as retrieved.

--- THIRD INSTANCE: RAW-FILE FETCH ALSO SUMMARIZED ---

While preparing this report, WebFetch was pointed at a raw file
(raw.githubusercontent.com/.../bug_report.yml) with the prompt "Output the raw
file contents exactly as received, unmodified. Do not summarize, reformat, or
add commentary." It returned a prose summary of the file instead of its
contents. There appears to be no caller-accessible way to obtain unprocessed
retrieved text.

--- RELATED ISSUES ---

#45070 — closest, but distinct (see above).
#22937 — WebFetch truncates large documentation pages. Possible contributing
mechanism: a summarizer handed a truncated document, gap-filling from
priors, would produce exactly this output shape. Worth checking whether
the target page was truncated before the summarizer saw it.
#7251 — --model override does not change the model used by WebFetch.
Forecloses the obvious workaround (pointing a stronger, less
prior-dependent model at the summarization step).
#23694 — PDF text extraction before the summarizer. Confirms the two-stage
(fetch -> summarizing model) architecture in which this occurs.
#4954 — Claude defaults to training data patterns instead of following
CLAUDE.md. Same underlying pathology (memorized priors overriding
supplied context) on a different surface.

--- SUGGESTED MITIGATIONS ---

1. Fail loudly on absence. When a prompt asks about a topic the document does

not cover, absence should be a reliably-emitted result rather than something
the model may paper over.

1. Distinguish quoted from paraphrased spans. If the tool marked which output

spans are literal substrings of the fetched document, callers could validate
quotes mechanically. Inserted content would not match. This is the single
highest-value fix — it makes the failure detectable without ground truth.

1. Reconsider the cache for verification retries. A caller re-fetching with a

different, stricter prompt is usually trying to verify. Serving from cache
silently converts that check into a no-op.

1. Bias the summarizer against prior knowledge for documentation, pricing, and

policy domains, or surface a staleness warning when output diverges from the
retrieved document.

1. Expose a raw-markdown mode that skips the summarizing model entirely, for

callers that need verbatim text and can accept the token cost. See the third
instance above — this is currently not achievable by prompting.