[BUG] WebSearch summaries present inferences as source content and assert sources that can't be found

Status Open
Maintainer reply None cached
Activity 0 comments · opened Oct 4, 2026

Preflight Checklist

  • [x] I have searched existing issues and this hasn't been reported yet
  • [x] This is a single bug report
  • [x] I am using the latest version of Claude Code

What's Wrong?

WebSearch result summaries sometimes state things the search results don't support. They attribute a generic statement to a specific entity, or assert a source exists when it can't be found. The text reads as if it came from the cited page, so an agent that can't open the page (blocked or paywalled) has no way to tell the summarizer's inference from the source.

These came up during literature checks in Claude Code on the web sessions in September 2026. The agents recorded them carefully, so each one is documented in my repo:

  1. Generic policy credited to a specific journal. A summary said Springer's line "Posting of preprints is not considered to be a prior publication" applied to the journal Regular and Chaotic Dynamics. But the underlying hit was Springer's generic policy page, and no RCD page said that. The agent's note: "One search summary credited Springer's generic line [...] to RCD, but the underlying hit was Springer's generic policy page, so it is NOT confirmed for RCD." (source, rcd-policy-5)
  2. A preprint that couldn't be found. For Chen & Liu 2024 (Physica D 470, 134392), "no arXiv copy found by search, although one search summary claimed a May 2024 preprint." (source, Alpha-model prior article review)
  3. Contradictory summaries of the same publisher policy. One summary said IOP "will generally consider preprints but not if they have been published under a Creative Commons Licence". A later summary of IOP's own pages said the CC restriction applies to accepted manuscripts, not preprints. (same file as 1, other-journals-9)

The first one is the clearest: the summary didn't just paraphrase, it extended a source's claim to a subject the source never mentioned.

What Should Happen?

  • Summaries should only state what the returned pages say, and should name which result each claim came from.
  • When the summarizer connects or generalizes across results (e.g. "this applies to journal X because X is published by Y"), it should mark that as an inference, not present it in the source's voice.
  • "X exists" claims (a preprint, a version, a page) should come with the URL that shows it, or be left out.

This matters most when the page itself can't be fetched, which is common for publisher sites. Then the summary is the only evidence the agent has, and it ends up in the agent's notes and conclusions.

Steps to Reproduce

This is non-deterministic. The pattern shows up with queries about a specific journal's policy where only the parent publisher's generic page is indexed. For example, search for Regular and Chaotic Dynamics' policy on arXiv preprints, then compare the summary's claims with the URLs of the returned results.

Claude Model

Not recorded in the session notes.

Is this a regression?

I don't know.

Platform

Claude Code on the web (Anthropic-hosted cloud environment)

View original on GitHub ↗