[Bug] Fan-out launches agents without stagger, causing correlated 529 failures

Status Open
Reported on v2.1.247
Maintainer reply None cached
Activity 1 comment · opened Aug 27, 2026

Environment

  • CLI 2.1.247, macOS darwin-arm64, Max 20x, main loop Opus 5
  • Related: #82605 (a transient 529 terminates a background subagent rather than retrying it)

What happens

When a workflow or an orchestrator fans out N agents, they are launched together and issue their first requests together. There is no jitter, no ramp, no stagger. The result is that a wide dispatch presents a burst to the API and the failures come back correlated: not one agent unlucky, but most of the batch failing inside the same minute.

Because of #82605 a 529 terminates the agent rather than retrying it, so a correlated burst does not degrade gracefully. It takes out a slice of the fan-out at once, and the units those agents owned are simply not done.

Evidence

Scanning 60 days of transcripts on one machine for 529 / Overloaded events:

total 529 events:                     72
distinct minutes containing one:      42
minutes containing MORE THAN ONE:     12

A third of the affected minutes contain multiple failures, which is not what an independent per-request failure rate looks like.

The clearest case is a 12-wide fan-out on 2026-08-24:

529s in the 06:00-07:30 window:       43
distinct transcripts affected:        17   (13 of them subagent transcripts)
span:                                 06:21 -> 07:19, 58 minutes

minute   distinct agents failing
06:21          4
06:25          3
06:28          4
06:29          1
06:30          1
...

Four distinct agents failing in the same minute, twice, at the front of the dispatch. The tail after 06:48 is single failures, which is what an uncorrelated background rate looks like. The difference between the head and the tail is the burst.

Why it is the client's problem to solve

The user's only lever is CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS, which is undocumented, defaults to 20, and had to be found and hand-tuned down to 6 to make wide dispatches survivable. That is the wrong shape of fix: it lowers the ceiling permanently instead of smoothing the launch, so it costs throughput on every dispatch to avoid a failure mode that only appears at the leading edge of one.

A concurrency cap and a launch stagger are different mechanisms. The cap limits how many run at once. The stagger limits how many start at once, which is where the correlation actually comes from.

Ask

  1. Stagger fan-out launches with a small jittered delay between agent starts. Tens to low hundreds of milliseconds is likely enough to decorrelate the first requests.
  2. Retry a 529 with jittered exponential backoff at the agent level, rather than terminating the unit. That half is #82605 and closing it would make this one much less damaging.
  3. Document CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS and surface the resolved value, so the existing lever is at least discoverable.

Workaround

Set the concurrency cap well below the default:

{ "env": { "CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS": "6" } }

This reduces the frequency of the burst but does not remove it, since 6 simultaneous starts are still simultaneous.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗