Cascading stall-watchdog failures when running 5+ parallel background subagents — retries also stall (v2.1.231)

Summary

Running 5 parallel background subagents (each doing long-running, stream-heavy work) triggers a cascading series of stall-watchdog failures. Within an ~8-minute window, 12 distinct AsyncAgent stall events fired — only 5 subagents were originally launched, meaning 7 of the events are automatic retries that also stalled. The retry path does not recover from this condition.

Debug-log evidence shows this is not a hung tool call and not a general network outage: one SSE stream received a single 647-byte chunk after a normal ~1.1s first-byte time, then went completely silent (zero further bytes) until the 60s stall watchdog aborted it — while the main process's heartbeat and a concurrent main-thread API request stayed healthy throughout.

Environment

• Claude Code version: 2.1.231 (claude --version)
• OS: macOS (Darwin)
• Invocation: claude --debug --debug-file ~/claude-stall.log

Reproduction steps

1. Start Claude Code with debug logging: claude --debug --debug-file ~/claude-stall.log
2. In a single prompt, launch 5 parallel background subagents, each doing deep, long-running research, e.g.:

• Subagent 1: read the real React GitHub source, trace an internal mechanism, write a 1500+ word explainer
• Subagent 2: same, for Postgres
• Subagent 3: same, for Kubernetes
• Subagent 4: same, for Redis
• Subagent 5: same, for Nginx

These tasks are chosen specifically to be long-running so their SSE streams stay open long enough to hit a 60s watchdog if one is going to fire.

1. Watch the debug log for AsyncAgent stall watchdog errors.

Observed behavior

12 distinct stall-watchdog events fired within roughly an 8-minute window (2026-08-13, UTC). Only 5 agents were launched; the remaining 7 are retries of already-stalled agents that also stalled:

| # | Agent ID | Time (UTC) | Last message before stall |
| --- | ------------------- | ------------ | ------------------------- |
| 1 | ac9b8816c47f63cc3 | 19:57:55.250 | query_progress |
| 2 | aff6ecd92546f12d6 | 19:58:14.343 | attachment |
| 3 | ac50bdc8c7c03963c | 19:58:27.747 | query_progress |
| 4 | a9388156f15f9c413 | 19:59:23.431 | query_progress |
| 5 | a39fe0447400674c5 | 19:59:41.430 | query_progress |
| 6 | a28aa3c807075c560 | 19:59:59.105 | query_progress |
| 7 | a2b6003915a6797e3 | 20:01:32.642 | attachment |
| 8 | a18595c62233c11d5 | 20:01:40.765 | attachment |
| 9 | af55e6e65aadb6c79 | 20:02:08.635 | attachment |
| 10 | a62cfbf00a390f700 | 20:03:36.752 | attachment |
| 11 | afb734c5d1c8fb2eb | 20:03:40.877 | query_progress |
| 12 | aa828740d60c6b4dd | 20:04:06.840 | attachment |

Each event has the exact log signature:

[ERROR] [AsyncAgent <id>] stall watchdog fired after 60000ms with no progress (last message: <type>); aborting

Detailed trace for the first stall (agent ac9b8816c47f63cc3)

19:56:54.097Z [Stall] tool_dispatch_start tool=Bash toolUseId=toolu_0133uci9jGZhK6FAeEwfvbyB
19:56:54.117Z [Stall] tool_dispatch_end tool=Bash outcome=ok durationMs=20
19:56:54.122Z [API REQUEST] /v1/messages source=agent:builtin:general-purpose
19:56:55.249Z [DEBUG] Stream started - received first chunk
19:56:55.249Z [DEBUG] [API:timing] first byte after 1127ms
19:57:10.247Z [WARN] [Stall] stream_idle_partial lastChunkAgeMs=14999 bytesTotal=647 idleDeadlineMs=180000
19:57:25.249Z [WARN] [Stall] stream_idle_partial lastChunkAgeMs=30001 bytesTotal=647 idleDeadlineMs=180000
19:57:55.250Z [ERROR] [AsyncAgent ac9b8816c47f63cc3] stall watchdog fired after 60000ms with no progress (last message: query_progress); aborting
19:57:55.250Z [WARN] [Stall] agent_completion agentId=ac9b8816c47f63cc3 exitPath=watchdog_stall durationMs=306722 turns=32 finalStopReason=tool_use lastChunkAgeMs=60001 lastToolUseId=toolu_0133uci9jGZhK6FAeEwfvbyB

Key point: the stream received exactly one chunk (647 bytes) after a normal first-byte time, then went completely silent — no further bytes at all — until a 60000ms watchdog fired and aborted it. Note this is a separate, shorter watchdog than the idleDeadlineMs=180000 value logged in the stream_idle_partial WARN lines above it — the 60s watchdog fires first. No retry was logged for this specific agent ID afterward (other agent IDs in the timeline above appear to be retries of the other 4 originally-launched agents).

Why this doesn't look network-related

• The main process stayed healthy throughout the window: CCRClient: Heartbeat sent kept firing on schedule (19:56:34, 19:56:55, 19:57:15, 19:57:36, 19:57:57).
• A concurrent main-thread /v1/messages API request completed normally in ~1.4s during the same window.
• The user separately ruled out their home network as a cause: a full 1-hour continuous network monitor (per-second ping to gateway + 3 DNS resolvers, DNS checks, throughput tests) showed 0 packet loss and 0 outages during a period when this same stall symptom occurred in an unrelated session.

Taken together, this points to a connection/stream-handling issue specific to running multiple (5+) concurrent background-agent SSE streams at once, where some subset of streams go completely silent (not slow — zero further bytes received) while other independent connections on the same process keep working fine.

No self-recovery on retry

7 of the 12 stall events are retries of already-stalled agents, and every one of those retries also stalled. The automatic retry path does not currently recover from this failure mode.

Possibly related issues

These look related based on symptoms (SSE stream stalls / silent streams / watchdog behavior), but I'm not asserting duplication — filing this separately since it specifically documents the 5+-concurrent-background-agent cascading-stall pattern with a full debug-log trace:

• #54434 — SSE stream stalls in long-running sessions without message_stop event
• #25979 — Hangs indefinitely when API streaming connection stalls, no read timeout
• #55647 — Claude stalling all the time, 15-60s stream stalls
• #53328 — Hang after tool_use response lost in streaming receive (watchdog timer drift was discussed as one of several unconfirmed hypotheses)
• anthropics/anthropic-sdk-typescript#998 — Proposal for a ping-aware streaming watchdog so SSE keepalive pings aren't mistaken for silence

Suggested areas to investigate

• Whether the 60s stall watchdog is too aggressive relative to legitimate mid-stream provider-side gaps, or whether it's correctly detecting genuinely dead connections that only occur under concurrent-stream load.
• Whether there's a connection-pool or multiplexing limit that causes a subset of concurrent SSE streams to silently die when 5+ are open at once.
• Why the automatic retry does not recover — retries appear to hit the same silent-stream condition rather than reconnecting cleanly.