[ultrareview] silently returns empty findings on large branch diffs (no scope warning, burns free quota)

Summary

/ultrareview marks the job as completed and returns an empty findings list ([]) on branches with large diffs, in under 1 minute (proper review should take 5-10 min). No error, no warning, no indication the reviewer truncated or aborted. The same code analyzed via local Sonnet reviewers (Task tool, scope-controlled to ~50 files each) returns 344 concrete findings.

Net effect: I burned 2 of 3 free monthly executions on this silent abort behavior, with zero useful signal.

Expected behavior

Either:

• Findings list (even if short), with confirmation N files / Y bytes were actually analyzed, OR
• Explicit error like diff too large: 1312 files / 615k insertions exceeds limit. Use \/ultrareview <PR#>\ to scope, or split the branch.

Observed behavior

• Job status: completed in < 1 minute
• Findings: []
• No scope/limit/truncation message
• Indistinguishable from \"code is actually clean\"

Reproduction

1. Branch ahead of main with ~1000 files / ~500k+ insertions of diff (code + planning docs + SQL migrations mix)
2. /ultrareview in Claude Code
3. ~1 min later: notification returns []

My two executions

Execution 1 — 2026-05-23

• Session: session_01RqZcksSVmEvCZNWqK1ThsA
• Scope CLI reported: 2812 files, 831376 insertions, 9785 deletions
• Result: []
• Wall time: < 1 min launched → completed

Execution 2 — 2026-05-24 (after cleaning obvious noise: generated artifacts, test dumps, spike directories)

• Session: session_01SuwirRz2E3JkhwuA1Jm8AP
• Scope CLI reported: 993 files, 563671 insertions, 9777 deletions
• Result: []
• Wall time: < 1 min launched → completed

Evidence the code DOES have bugs (rules out \"clean code\" hypothesis)

Between the two /ultrareview runs, I ran 12 Sonnet reviewers via Task tool with controlled scope (40-50 files each). They found 344 concrete findings in the SAME code: 86 BLOCKERS, 186 WARNINGS, 72 INFO.

Reproducible blockers include:

• RLS on obras table open to any authenticated user (verifiable via pg_policy)
• VITE_INTERNAL_TOKEN baked into the public JS bundle (CWE-798) — visible in View Source
• ~9 SECURITY DEFINER functions without SET search_path (CVE-2018-1058)
• get_user_empresa_id() single-tenant in ~30 RLS policies on a group-aware multi-tenant system
• OAuth state base64 without nonce (CSRF) in 4 sites
• CSV formula injection in finance export (operator-side RCE)
• parseFloat(\"1.234,56\") losing 1000x on BR-locale forms

Conclusion: ultrareview returned [] on code with 86 confirmed blockers. The job didn't work — silent failure.

Impact

1. Lost monthly free quota with zero useful signal.
2. False confidence: a user trusting [] as \"clean\" merges without review. Silent failure is worse than explicit error.
3. Forced workaround: I had to manually recreate the ultrareview behavior by dispatching 12 Task-tool reviewers. Worked, but cost time + requires users to know this fallback exists.

Suggested fixes (cheap → ideal)

1. Minimum: return explicit error when diff exceeds limit, never silent []. Example: { \"status\": \"aborted\", \"reason\": \"diff_too_large\", \"files\": 993, \"limit\": 500 }
2. Good: accept large diff but report truncation in the result.
3. Ideal: internal chunked review (split diff into ~50-file batches and aggregate findings — same approach my manual local fallback used).
4. Compensation: refund credits that returned [] (the job aborted on scope check, no compute used).

Environment

• Claude Code CLI (current as of 2026-05-24)
• Windows 11 Pro 10.0.26100
• Plan: Claude Code Max
• Branch dev ahead of master

Full bug report

Detailed version including session URLs and full evidence: docs/anthropic-ultrareview-bug-report.md in my repo, available on request.