/ultrareview consumes free run on Find-phase timeout failure for oversized diffs
Resolved 💬 1 comment Opened Apr 29, 2026 by leandrogg Closed May 31, 2026
Summary
/ultrareview consumes a free run from the 3-of-3 quota even when the cloud review fails before producing any findings. Specifically observed: a Find-phase timeout on an oversized diff failed the run, but the quota was still decremented from 3 to 2.
Reproduction
- Confirm 3 free runs remaining (
/extra-usage). - Check out
mainof a project and create a branch pointing at the project's earliest commit (or any base producing a very large diff). - Run
/ultrareview <large-base-branch>against a diff that's far above the typical PR size — in my repro, 433 files / 241,017 insertions / ~417 commits. - Confirm the launch succeeded (CLI shows \"Ultrareview launched...\", session URL provided).
- Wait for the task to complete.
Observed
- Session UI (example layout) shows:
- Setup: ✅ pass
- Find: ❌ fail (timeout/heartbeat)
- Verify: ⏳ never ran
- Dedupe: ⏳ never ran
- CLI returns `
<remote-review>[]</remote-review>` (zero findings) — easily mistaken for \"no bugs found\" rather than \"review failed before finding anything\". /extra-usageshows 2 of 3 remaining — the failed run was charged.
Expected
A Find-phase failure (especially a timeout for being too large to process) should either:
a. Auto-refund the free run, since the user got nothing actionable out of it.
b. Pre-flight reject the diff with a clear \"diff too large, narrow the scope\" message before consuming a run.
Why this matters
- The CLI output
[]looks identical to \"clean review, no bugs found\" — a user could legitimately mistake this for a successful clean pass and merge / ship without a real review having happened. The session UI does show the failure, but only if you click through. - Free runs are scarce (3 total per Pro/Max account, one-time, expiring May 5, 2026). Burning one on a no-output failure is high-friction.
Suggested fix
- Add a pre-flight diff-size check at launch time. If files-changed × insertions-added is above the cloud fleet's effective processing window, refuse with a clear \"scope is too large; pass a more recent base branch or split into surface-specific reviews\" message.
- Or: don't consume the free run unless Find phase produces output (verified or not).
- Make the CLI output for a failed Find phase distinct from a successful zero-findings result.
[]should not be the only signal for both states.
Environment
- Claude Code CLI on Linux
- Anthropic Pro/Max subscription, extra-usage enabled
- Private GitHub repo (org-owned)
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗