deep-research built-in workflow: verify phase repeatedly rate-limits itself, then misreports the infra failure as "all claims refuted"
Environment: Claude Code, built-in deep-research workflow (the Workflow({name:"deep-research"}) harness).
What happened
This is recurring — it has happened on several separate deep-research runs, not a one-off. In the latest run, a single invocation spawned 105 agents (~1.68M tokens). The Verify phase fans out up to MAX_VERIFY_CLAIMS (25) × VOTES_PER_CLAIM (3) = 75 agents, each doing its own WebSearch. That burst tripped a server-side request throttle — every verify vote failed with:
API Error: Server is temporarily limiting requests (not your usage limit) · Rate limited
(75/75 failed.) That it recurs across runs — and that all 75 failed without recovering — indicates there's no effective backoff/retry saving it.
Bug 1 — no rate-limit handling in the verify fan-out
The min(16, cores−2) concurrency cap bounds parallelism, not request rate, so ~5 back-to-back waves of search-heavy agents burst past the throttle. There's no backoff/jitter, no request-rate throttle on the verify fan-out, and no cheaper-model fallback for what is a high-volume phase. It reproduces repeatedly.
Bug 2 (more important) — infra failure is misreported as a research result
When verify votes error out, they're counted as "abstain," and the final report reads:
All 25 claims refuted by adversarial verification. Research inconclusive — sources may be low-quality or claims overstated.
That's actively misleading — the claims weren't refuted on their merits; the verifier never ran. A user concludes their research found nothing when the verification infrastructure simply failed. The summary should distinguish "refuted on merit" from "could not verify (infrastructure error)."
Repro
Run deep-research on any broad question that yields ≥~20 extracted claims (enough to trigger the full 75-agent verify fan-out). Observed to fail this way across multiple runs.
Suggested fixes
- Throttle/jitter or downgrade the verify-vote model; retry 429s with backoff.
- (Most important) When a claim's votes fail on infra errors, mark it
unverified— notrefuted— and report that distinctly, so the run's summary never masquerades an outage as a finding.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗