deep-research: claim verification budget starves angles without primary sources, and the truncation is never surfaced
What happened
I ran the built-in /deep-research workflow on a question that the Scope phase
decomposed into 5 angles. Two of those angles were about operational practice
(how teams organize internal documentation, how they handle content going stale)
rather than product features.
The final report's caveats said, in effect, "no verified external evidence was
found for these two angles."
That reads as a research finding. It wasn't. Those angles' claims were
extracted — they just never reached a verifier.
Why
The workflow ranks claims globally before verification (verbatim from the 2.1.241
bundle):
const MAX_VERIFY_CLAIMS = 25
const rankedClaims = [...allClaims]
.sort((a, b) => (impRank[a.importance] - impRank[b.importance])
|| (qualRank[a.sourceQuality] - qualRank[b.sourceQuality]))
.slice(0, MAX_VERIFY_CLAIMS)
sourceQuality is the tiebreaker, ranked primary > secondary > blog > forum.
Angles about product features cite official documentation, so every one of their
claims is primary. Angles about operational practice can only ever reachblog/forum — not because the claims are weak, but because that's the best
source class the subject matter has.
So the angles that can produce primary sources take the entire budget, and the
others are cut before verification. The run I observed extracted 104 claims and
verified 25.
Simulating the old selection against a claim distribution of that shape
(5 angles, 104 claims, two angles primary-heavy and two blog/forum-heavy) gives:
| angle | selected |
|---|---|
| 1 (docs-backed) | 12 / 30 |
| 2 (docs-backed) | 11 / 28 |
| 3 (mixed) | 2 / 20 |
| 4 (community) | 0 / 14 |
| 5 (community) | 0 / 12 |
Related: this is #69883's principle, applied to a path it doesn't cover
#69883 ("verify phase repeatedly rate-limits itself, then misreports the infra
failure as 'all claims refuted'") established that a coverage gap must never be
reported as a research result. That fix landed — the script now separatessurvives / isRefuted / unverified, with a comment citing the issue:
// Three outcomes (go/ccissue/69883 — infra failure must not read as "refuted"):
But that separation only covers claims that reached a verifier and whose votes
errored. Claims cut by MAX_VERIFY_CLAIMS never enter voted at all, so they
appear in no bucket — not confirmed, not refuted, not unverified. The same class
of misreporting survives through a different door.
I searched open and closed issues for this specific failure (deep-research,MAX_VERIFY_CLAIMS, claim/budget/angle terms) and didn't find it filed. Nearby
but distinct: #69883 (rate-limited votes), #88106 (no retry for those votes),
#70433 (no upfront fan-out estimate), #83325 (verifier conflates "qualifies"
with "contradicts").
The part that makes it a bug rather than a budget limit
The truncation is invisible in the output:
- The run log prints
Fetched 21 sources → 104 claims → verifying top 25with no
per-angle breakdown.
- The
budgetDroppedcounter only tracks the source stage. Claim-stage
truncation is counted nowhere and appears in neither the logs nor stats.
- The synthesis prompt receives only the confirmed/refuted/unverified claim
lists. It has no way to know an angle was cut, so it writes the absence up as
a finding.
The result is that "the sources don't support this" and "we never looked" are
indistinguishable to the reader — and the second one is silently presented as
the first.
What I'd expect
Either outcome would be fine; what I didn't expect was for the gap to be
invisible.
- Count budget-cut claims as
unverifiedrather than dropping them, so they
land in the bucket #69883 already created for exactly this purpose.
- Give each angle a floor before filling the remainder by global merit.
- Log and report per-angle
extractedvsverified. - Pass the truncation into the synthesis prompt so caveats can say "N claims
for this angle went unverified due to the budget cap" instead of implying no
evidence exists.
(1) alone would fix the misreporting. (2) additionally improves what gets
verified in the first place.
Verifying the fix
Every Workflow invocation persists its script under the session directory, so I
edited that copy and re-ran the same question. With a per-angle floor of 4 and
the cap raised 25 → 30:
Fetched 21 sources → 105 claims → verifying 30
angle 1: 14/30 verified (16 dropped unverified — budget)
angle 2: 4/10 verified (6 dropped unverified — budget)
angle 3: 4/30 verified (26 dropped unverified — budget)
angle 4: 4/15 verified (11 dropped unverified — budget)
angle 5: 4/20 verified (16 dropped unverified — budget)
No angle at zero, and the coverage gap is now stated rather than inferred. The
second run surfaced material from the previously-starved angles that the first
run had reported as nonexistent.
Worth noting that the floor alone doesn't fully solve it — the surplus above the
floor still went entirely to angle 1, leaving angle 3 at 4 of 30 extracted. But
the reader can now see that.
Environment
- Claude Code 2.1.241 (also observed on 2.1.229)
- macOS 15 (Darwin 24.6.0), Apple Silicon
- Built-in
/deep-researchdynamic workflow, invoked via theWorkflowtool