[Bug] Workflow tool accepts arguments without validation, causing silent failures with wasted API calls

Open 💬 0 comments Opened Jun 9, 2026 by jonobri

Bug Description
✻ Cogitated for 24s · 1 shell still running

❯ /usage
⎿ Settings dialog dismissed

❯ Those 23 papers used 10% of the week's Sonnet budget, 6% of total budget. That's a lot!

⏺ Worth clarifying: those 23 quant papers haven't actually run yet — the 6.5M was the accidental re-audit of the 36 already-audited papers. _current_batch.json now
has the correct 23 quant IDs, ready to go.

The real per-paper cost was 6.5M ÷ 36 = ~180k tokens/paper. That's a bit higher than the 150k estimate, probably because those papers are content-heavy late-series
reports.

The maths at 180k/paper with ~90% Sonnet remaining:

┌───────────────────────┬────────┬─────────┬─────────────────┐
│ Run │ Papers │ ~Tokens │ % weekly Sonnet │
├───────────────────────┼────────┼─────────┼─────────────────┤
│ RQ2 quant tail │ 23 │ ~4.1M │ ~6% │
├───────────────────────┼────────┼─────────┼─────────────────┤
│ Top 20 priority queue │ 20 │ ~3.6M │ ~5.5% │
├───────────────────────┼────────┼─────────┼─────────────────┤
│ Full priority 60 │ 60 │ ~10.8M │ ~17% │
├───────────────────────┼────────┼─────────┼─────────────────┤
│ All ~309 remaining │ 309 │ ~55M │ ~85% │
└───────────────────────┴────────┴─────────┴─────────────────┘

So the entire remaining corpus is roughly one full week of Sonnet budget. That's the honest scale of the work.

My recommendation given budget caution: run the RQ2 quant tail (23 papers) now — it's the highest-value work and uses 6%, completing the entire quantitative
universe. Then pause and decide whether to run the priority queue in batches over the coming week rather than all at once today. That way a limit hit mid-run
doesn't waste anything on the highest-value papers.

Want me to launch the RQ2 tail now?

✻ Cooked for 38s · 1 shell still running

❯ Wait, I thought you just said: The RQ2 tail currently running is the last 23 quant papers???????

⏺ You're right to call that out. I was wrong when I said that.

When I launched the workflow I believed it would use the args I passed (the 23 quant IDs). It didn't — it read _current_batch.json, which still had the grading
IDs. The workflow ran on the wrong 36 papers. I only discovered this when the result came back with IDs 338–450 instead of 3–291.

So the sequence was:

  1. I said "RQ2 tail running" — incorrect, it was running on the wrong pape…

Note: Content was truncated.

View original on GitHub ↗