[Model behavior] Claude presented its own inference as a sourced fact, and the source said the opposite

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Other unexpected behavior

What You Asked Claude to Do

I asked Claude (Opus 5.5, Claude desktop app, Code tab) for a company overview of Datadog (Nasdaq: DDOG) as research before reaching out to people who work there. Claude used WebSearch and WebFetch.

What Claude Actually Did

Claude wrote:

One risk to know about: Datadog's largest customer reduced its usage going into Q3. The customer renewed its contract, but it's the main reason growth guidance slowed to 28-29% for Q3. It's the downside of charging by usage...

I asked: "Validate this. Who's the customer, where did you pull this data from?"

Claude then fetched the Q2 2026 earnings-call transcript (Aug 6, 2026) and found:

• Correct: CFO David Obstler: "Regarding our largest customer, we have seen a usage reduction, which is incorporated in our Q3 and full year 2026 guidance." It followed a nine-figure renewal.
• Wrong, stated as fact: "the main reason growth slowed." CEO Olivier Pomel said the opposite: "If you back out our largest customer from our growth, you get pretty much the same growth rate as the rest of the business."
• Left out: Datadog does not name the customer. Press reports speculate it's OpenAI. The first answer didn't say the customer was unnamed or that any identification is speculation.

Root cause (Claude's own analysis in the session):

1. It wrote from summaries, not sources. WebSearch and WebFetch return a small model's paraphrase of a page, and Claude treated that paraphrase as the source.
2. It filled a gap with a plausible inference. The summary gave two facts side by side (the usage cut and the Q3 guide), and Claude connected them causally without saying so.
3. No provenance labels. Sourced facts, Claude's inferences, and press speculation all appeared in the same confident voice.

Expected Behavior

The model may guess, but the user must know when it's a guess. Otherwise, how can we trust it?

1. Label provenance: source says / I infer / press speculates. An inference is fine; an unlabeled inference isn't.
2. Don't upgrade correlation to causation ("one factor" to "the main reason") without a source saying so.
3. For material facts (financials, causal claims, who-is-the-customer), verify against the primary source (SEC filing, press release, transcript), or say the claim comes from a secondary summary.
4. When the source says something is undisclosed ("the customer is not named"), say so.

Files Affected

None. This is about text output in chat, not file changes.

Permission Mode

I don't know / Not sure

Can You Reproduce This?

Sometimes (intermittent)

Steps to Reproduce

1. Ask Claude for a research-style company overview (financials, risks, causes) using WebSearch/WebFetch.
2. Check causal or attribution claims ("main reason", "driven by", "the customer is X") against the primary source.

The same pipeline (search summaries, then confident prose) produced facts for four other company briefs in the same session. After re-verifying all of them against primary sources, several more unsupported claims turned up: a customer count and a total-funding figure with no source, an undated third-party revenue figure presented as dated, and a claim sourced only from a vendor blog.

Claude Model

Opus

Relevant Conversation

Me: "Validate this. Who's the customer, where you pull this data from."

Claude: "What I got wrong: I wrote that it was 'the main reason' growth slowed to 28-29%. That was my own guess, and the CEO said the opposite... Root cause: I wrote from search summaries (a short AI paraphrase of the page) instead of the actual filing and transcript, and I stated my own inference as if it were fact."

Impact

Medium - Extra work to undo changes

Claude Code Version

2.1.198 (claude CLI on this Mac). The session ran in the Claude desktop app, Code tab, macOS. Model: claude-opus-5-5. Tools: WebSearch, WebFetch.

Platform

Anthropic API

Additional Context

Why it matters: I was preparing to talk with people at this company. Repeating "the main reason" to a Datadog employee would contradict what their own CEO said on the record. A user can't tell verified from inferred unless the model marks the difference.

Suggested fixes:

• In research-style answers, default to marking claims by where they came from, especially causal ("because", "main reason", "driven by") and attribution claims.
• Make WebFetch or WebSearch results clearly marked as summaries, so the model treats them as leads rather than sources.

Related: #98234 (numbers without units misread as dates), from the same session.