deep-research: claim verification budget starves angles without primary sources, and the truncation is never surfaced

Status Open
Reported on v2.1.241
Maintainer reply None cached
Activity 0 comments · opened Aug 24, 2026

What happened

I ran the built-in /deep-research workflow on a question that the Scope phase
decomposed into 5 angles. Two of those angles were about operational practice
(how teams organize internal documentation, how they handle content going stale)
rather than product features.

The final report's caveats said, in effect, "no verified external evidence was
found for these two angles."

That reads as a research finding. It wasn't. Those angles' claims were
extracted — they just never reached a verifier.

Why

The workflow ranks claims globally before verification (verbatim from the 2.1.241
bundle):

const MAX_VERIFY_CLAIMS = 25
const rankedClaims = [...allClaims]
  .sort((a, b) => (impRank[a.importance] - impRank[b.importance])
                || (qualRank[a.sourceQuality] - qualRank[b.sourceQuality]))
  .slice(0, MAX_VERIFY_CLAIMS)

sourceQuality is the tiebreaker, ranked primary > secondary > blog > forum.

Angles about product features cite official documentation, so every one of their
claims is primary. Angles about operational practice can only ever reach
blog/forum — not because the claims are weak, but because that's the best
source class the subject matter has.

So the angles that can produce primary sources take the entire budget, and the
others are cut before verification. The run I observed extracted 104 claims and
verified 25.

Simulating the old selection against a claim distribution of that shape
(5 angles, 104 claims, two angles primary-heavy and two blog/forum-heavy) gives:

| angle | selected |
|---|---|
| 1 (docs-backed) | 12 / 30 |
| 2 (docs-backed) | 11 / 28 |
| 3 (mixed) | 2 / 20 |
| 4 (community) | 0 / 14 |
| 5 (community) | 0 / 12 |

Related: this is #69883's principle, applied to a path it doesn't cover

#69883 ("verify phase repeatedly rate-limits itself, then misreports the infra
failure as 'all claims refuted'") established that a coverage gap must never be
reported as a research result. That fix landed — the script now separates
survives / isRefuted / unverified, with a comment citing the issue:

// Three outcomes (go/ccissue/69883 — infra failure must not read as "refuted"):

But that separation only covers claims that reached a verifier and whose votes
errored. Claims cut by MAX_VERIFY_CLAIMS never enter voted at all, so they
appear in no bucket — not confirmed, not refuted, not unverified. The same class
of misreporting survives through a different door.

I searched open and closed issues for this specific failure (deep-research,
MAX_VERIFY_CLAIMS, claim/budget/angle terms) and didn't find it filed. Nearby
but distinct: #69883 (rate-limited votes), #88106 (no retry for those votes),
#70433 (no upfront fan-out estimate), #83325 (verifier conflates "qualifies"
with "contradicts").

The part that makes it a bug rather than a budget limit

The truncation is invisible in the output:

  • The run log prints Fetched 21 sources → 104 claims → verifying top 25 with no

per-angle breakdown.

  • The budgetDropped counter only tracks the source stage. Claim-stage

truncation is counted nowhere and appears in neither the logs nor stats.

  • The synthesis prompt receives only the confirmed/refuted/unverified claim

lists. It has no way to know an angle was cut, so it writes the absence up as
a finding.

The result is that "the sources don't support this" and "we never looked" are
indistinguishable to the reader — and the second one is silently presented as
the first.

What I'd expect

Either outcome would be fine; what I didn't expect was for the gap to be
invisible.

  1. Count budget-cut claims as unverified rather than dropping them, so they

land in the bucket #69883 already created for exactly this purpose.

  1. Give each angle a floor before filling the remainder by global merit.
  2. Log and report per-angle extracted vs verified.
  3. Pass the truncation into the synthesis prompt so caveats can say "N claims

for this angle went unverified due to the budget cap" instead of implying no
evidence exists.

(1) alone would fix the misreporting. (2) additionally improves what gets
verified in the first place.

Verifying the fix

Every Workflow invocation persists its script under the session directory, so I
edited that copy and re-ran the same question. With a per-angle floor of 4 and
the cap raised 25 → 30:

Fetched 21 sources → 105 claims → verifying 30
  angle 1: 14/30 verified (16 dropped unverified — budget)
  angle 2: 4/10 verified (6 dropped unverified — budget)
  angle 3: 4/30 verified (26 dropped unverified — budget)
  angle 4: 4/15 verified (11 dropped unverified — budget)
  angle 5: 4/20 verified (16 dropped unverified — budget)

No angle at zero, and the coverage gap is now stated rather than inferred. The
second run surfaced material from the previously-starved angles that the first
run had reported as nonexistent.

Worth noting that the floor alone doesn't fully solve it — the surplus above the
floor still went entirely to angle 1, leaving angle 3 at 4 of 30 extracted. But
the reader can now see that.

Environment

  • Claude Code 2.1.241 (also observed on 2.1.229)
  • macOS 15 (Darwin 24.6.0), Apple Silicon
  • Built-in /deep-research dynamic workflow, invoked via the Workflow tool

View original on GitHub ↗