[Bug] Workflow/ultracode parallel() fan-out trips server-side per-account 429 ("not your usage limit") and wipes the entire run — no backoff or partial-result preservation

Workflow/ultracode parallel() fan-out trips a server-side per-account rate limit (HTTP 429, "not your usage limit") and wipes the entire run — no backoff, no partial-result preservation

Summary

A single Workflow ("ultracode") invocation that uses parallel() to launch ~5–6 subagents concurrently from one fan-out reliably trips a server-side, per-account rate limit (HTTP 429, internal code rate_limit, error text explicitly "not your usage limit"). All concurrently-dispatched agents surface the 429 within an ~8-second window and terminate. There is no automatic backoff/retry, no concurrency throttling, and no partial-result preservation — the entire run is lost (returned {dimensions:0, findings:[], roadmap:null}) despite ~234K tokens and 53 tool-uses already spent. Converting the same fan-out to a sequential one-agent-at-a-time loop runs cleanly with zero 429s, which pinpoints the concurrency burst (not an account usage cap or a general outage) as the root cause.

Environment

| | |
|---|---|
| Tool | Claude Code "Workflow" / "ultracode" (subagent dispatch via parallel() / pipeline() / agent()) |
| Claude Code version | 2.1.170 |
| Entrypoint / userType | claude-desktop / external |
| Model | claude-opus-4-8[1m] (error lines log model: "<synthetic>") |
| Platform | win32 (Windows 11) |
| Session id | 6587053b-57b0-4d89-965a-0efb56090d83 |
| Workflow run id | wf_a913f61b-8e7 |

What happened

A single Workflow invocation did parallel(DIMS.map(...)), launching 5 investigator subagents + 1 synth/aggregator — a ~6-wide concurrent fan-out from one invocation. The five investigators all started inside a 5-millisecond window:

a978f7228168515e7  started 2026-06-17T01:53:24.462Z
a3565f03e557fc81c  started 2026-06-17T01:53:24.464Z
a613b1a7b6e3ef5be  started 2026-06-17T01:53:24.465Z
a6694637c70c432b3  started 2026-06-17T01:53:24.466Z
adb9d747cc3b0a994  started 2026-06-17T01:53:24.467Z

~37 seconds later they all hit HTTP 429 within an ~8-second window, and the synth agent was rate-limited the instant it started — before executing a single tool-use:

a3565f03e557fc81c  rate-limited 01:54:01.781Z  req_011Cc7zqQNPPv7ynmYMm6xi6
adb9d747cc3b0a994  rate-limited 01:54:02.948Z  req_011Cc7zqUiLCVCvzWB9KhoYZ
a978f7228168515e7  rate-limited 01:54:08.063Z  req_011Cc7zqsMMgbC29eHapjG2j
a6694637c70c432b3  rate-limited 01:54:08.833Z  req_011Cc7zqssNGpG3HWtVq2NPm
a613b1a7b6e3ef5be  rate-limited 01:54:09.036Z  req_011Cc7zqvKwt9Hvrs9UFzmgf
a9d1dd107ffaab225  rate-limited 01:54:09.740Z  req_011Cc7zqzWDhDr2bfM8y4o6o  (synth — 0 tool-uses, never ran)

The exact, verbatim error surfaced on every failed agent (separator is U+00B7 ·):

API Error: Server is temporarily limiting requests (not your usage limit) · Rate limited

Logged on each failed agent line as:

{ "isApiErrorMessage": true, "apiErrorStatus": 429, "error": "rate_limit" }

…with a synthetic assistant message ("model": "<synthetic>", "stop_reason": "stop_sequence", "usage": {"input_tokens": 0, "output_tokens": 0}). The 429 is a server-side per-account throttle — the text literally says "not your usage limit" — not the user's quota.

No backoff was offered or honored: a case-insensitive scan of every transcript file (journal.jsonl + all agent-*.jsonl) for retry-after / retry_after / backoff returned 0 hits. Each agent surfaced the 429 directly and terminated with no retry, queueing, or spacing. The workflow returned {dimensions:0, findings:[], roadmap:null}.

Expected behavior

1. Bound/queue concurrent agent dispatch. The framework should cap concurrency and/or token-bucket-pace agent launches so a single parallel() fan-out does not burst N simultaneous API streams into a server-side per-account limit.
2. Auto-backoff + retry on a server-side rate limit. A 429 labelled "not your usage limit" is transient and retryable. The framework should detect it and retry with exponential backoff/jitter (honoring Retry-After when present) rather than failing the agent.
3. Preserve partial results. Agents that completed (or partially completed) before the throttle should have their findings captured and the run made resumable — not discarded wholesale.
4. Surface a clear "throttled — retrying" state instead of a hard agent failure that propagates to total run loss.

Steps to reproduce

1. From a single Workflow/ultracode invocation, fan out ~5–6 subagents concurrently, e.g.:

``js
await parallel(DIMS.map(d => agent({ task: investigate(d) })));
// + 1 synth/aggregator agent
``

1. Let each agent begin doing real work (tool-uses, model calls) so the burst opens ~5–6 simultaneous API streams.
2. Observe: within ~8 seconds all concurrent agents return HTTP 429 / error: "rate_limit" with the text "Server is temporarily limiting requests (not your usage limit) · Rate limited", and the whole run is wiped.

Reproduced deterministically here on run 1; the failure is tied to the concurrency burst (see Workaround).

Impact

• Total multi-agent run loss. ~233,972 tokens (per-agent peaks: 56,838 / 49,338 / 47,980 / 40,421 / 39,395; synth 0) and 53 tool-uses were spent in ~45s and entirely discarded. Workflow returned {dimensions:0, findings:[], roadmap:null}.
• The synth/aggregator never ran (0 tool-uses, 0 tokens) — it was rate-limited the instant it started, so even partial investigator output could not be salvaged.
• ultracode is unusable for its core pattern. parallel() fan-out is the headline feature; under this throttle the primary use case fails deterministically and expensively, while the cost (tokens) is still incurred.

Workaround (confirms root cause)

Converting the identical work from parallel() to a sequential for-loop (one agent at a time) runs cleanly with zero 429s. The same five dimensions re-ran back-to-back in strictly non-overlapping windows, every agent with error: rate_limit absent:

a98d74cff7990f251  02:00:28.252Z -> 02:08:29.372Z  (30 tool-uses, completed)
a72ace75e0aa17222  02:08:29.439Z -> 02:15:38.815Z  (29 tool-uses, completed)
ab0435c2cc7d23345  02:15:38.885Z -> 02:27:41.075Z  (28 tool-uses, completed)
a09228dd178d4ebd6  02:27:41.142Z -> 02:36:02.092Z  (33 tool-uses, completed)
ae7fb72a1a6dcba3b  02:36:02.226Z -> 02:37:20.469Z  (17 tool-uses, last/synth)

These are provably re-runs of the same tasks — the Workflow v2 journal dedupe keys match across runs (e.g. key c746b50042e6…08ea: run 1 a978f7228168515e7 FAILED → run 2 a98d74cff7990f251 SUCCEEDED; 8a16a5e4a7…c485: a3565f03 → a72ace75; bb2e197c…e177: a613b1a7 → ab0435c2; 7cfe2498…2d89: a6694637 → a09228dd; e3cde76a…ff7e: adb9d747 → ae7fb72a). Strict serialization ⇒ no concurrency burst ⇒ no 429. This rules out an account-level usage cap or a general outage and isolates the concurrent dispatch burst as the trigger.

Suggested fixes

• Client-side concurrency cap + token-bucket pacing of agent dispatch (configurable maxConcurrency, with a safe default below the server's per-account ceiling), so parallel() schedules rather than bursts.
• Exponential backoff + jitter retry on apiErrorStatus: 429 / error: "rate_limit" (especially the "not your usage limit" server throttle), honoring Retry-After when supplied; do not turn a transient 429 into a terminal agent failure.
• Partial-result preservation / resumable runs: capture completed agents' outputs and make the run resume from where it throttled, rather than returning {dimensions:0, findings:[], roadmap:null} after ~234K tokens of work.
• Surface a "throttled — retrying" status to the user (and in the journal) instead of a hard failure, and ideally have parallel() degrade gracefully toward sequential scheduling when it detects repeated server-side throttling.