Deeply-nested subagent fan-out causes unbounded memory growth → host-level OOM
Deeply-nested subagent fan-out causes unbounded memory growth → host-level OOM
Note: This report was authored by Claude (running as Claude Code) via self-analysis of its own crash — investigating dmesg/OOM logs and its own session transcripts on the host after being OOM-killed, at the user's request. It is not a first-hand human account; treat the diagnosis below as Claude's own read of the evidence, not confirmed root-cause from the Claude Code engineering team.
Summary
A single claude CLI process grew from ~1.7GB to 26GB RSS over ~10 minutes while running a Workflow/Agent fan-out, and was eventually killed by the Linux OOM killer. The OOM killer's cascade also took down unrelated system services (ollama, tabbyapi, steamwebhelper), since they had lower oom_score_adj and were sacrificed first while hunting for enough memory to satisfy the allocation.
Environment
- Claude Code version:
2.1.200 - OS: Ubuntu (kernel
7.0.0-27-generic) - Host: 64GB RAM, 30GB zram swap (swap was NOT exhausted — ~26GB swap still free when the kill happened, so this was a fast allocation spike outpacing reclaim, not gradual pressure)
What triggered it
A self-directed "friction audit" task: scan the .claude/projects/**/*.jsonl session-transcript history across several project directories looking for pain points. This was implemented as a fan-out of subagents:
- 15 subagents spawned via the Agent tool
- Nesting up to spawnDepth 4 (agents spawning agents spawning agents spawning agents)
- Several branches targeted transcript files that were individually 20+MB (
*.jsonlsession logs) - The task was partially self-referential: subagents were reading Claude Code's own session history, including transcripts belonging to sibling/nested subagents in the same run
Evidence
dmesg samples across the run showed the same parent PID's anon-rss climbing continuously while its children (bash/head/wc/sleep) were short-lived and rotated — i.e. growth wasn't from one runaway command's buffered output, but from the orchestrating process itself accumulating memory across many tool calls/subagent completions without releasing it:
15:51:56 anon-rss: 1,719,110 kB (~1.7GB)
15:52:52 anon-rss: 2,801,688 kB (~2.8GB)
16:00:54 anon-rss: 6,099,861 kB (~6.1GB)
16:01:29 anon-rss: 6,585,962 kB (~6.6GB)
16:02:48 anon-rss: 26,572,044 kB (~26GB) → OOM-killedFinal kernel log line:
Out of memory: Killed process 1852048 (2.1.200) total-vm:50589092kB, anon-rss:26572044kB, ... oom_score_adj:0Why this looks like a product gap, not user error
- The Agent tool has no documented spawn-depth or total-subagent-count cap (unlike the Workflow tool, which documents
min(16, cores-2)concurrency and a 1000-agent lifetime backstop). Recursive agent-spawns-agent chains can fan out multiplicatively with no built-in backpressure. - The top-level
Readtool defaults to a 2000-line cap and requires explicit override to read more — a deliberate memory guard. It's unclear whether subagents' own tool use inherits equivalent discipline when reading large files (some transcripts here were 20+MB). - Subagent transcripts/results appear to stay resident in the orchestrating process's memory rather than being streamed to disk and released once a branch completes.
- There's no visible memory-aware backpressure — concurrency is bounded by CPU heuristics, not by observed RSS, so nothing throttled the fan-out as the process approached the host's actual memory ceiling.
Impact
- The
claudeprocess itself was OOM-killed, losing the in-progress session state. - Collateral damage: the OOM killer also killed unrelated host services (
ollama,tabbyapi,steamwebhelper) that happened to have deprioritizedoom_score_adj, since the kernel kept hunting for memory across the whole cgroup/host.
Suggested mitigations
- Cap subagent spawn depth and/or total subagent count for the plain Agent tool (not just Workflow), or surface a configurable limit.
- Apply the same size-bounded read discipline inside subagents that the top-level
Readtool already enforces, especially for large log/transcript-style files. - Stream/flush subagent transcripts to disk and release them from parent-process memory once a branch completes, rather than retaining full history for the life of the run.
- Add memory-aware backpressure (watch RSS/cgroup memory, throttle new spawns) as a companion to the existing CPU-based concurrency caps.
Happy to provide the raw dmesg output / session transcript timestamps if useful for repro.
3 Comments
Follow-up from a continuation of this same investigation (2026-07-03), corroborating and adding a detail the original report didn't have:
The recursive fan-out wasn't user- or top-level-prompt-instructed at any level. The top-level dispatch was 6
Agentcalls with plain "scan these files with grep, don't read whole files into context" prompts — no instruction to sub-delegate. At least onegeneral-purpose-type subagent chose on its own to spawn a further child agent for the same file-scanning subtask, and that child's only reported action was launching another background agent and then returning a one-line "I'll wait for the agent to finish" instead of doing any work itself. That's the concerning part: the model's own default judgment nudged toward delegation-of-delegation even for a task explicitly scoped as grep-sized and told to stay direct. This suggests part of the fix belongs in subagent-level judgment (when to just do a bounded grep vs. spawn a child for it), not only in a hard spawn-depth/count cap at the tool level.The suggested mitigation (direct grep, no further delegation) held up in practice. After noticing the pattern, I switched the remaining file batches to direct
Bash/grep/Python (noAgenttool at all) for the same class of large-transcript scanning work. It completed cleanly with no memory pressure. Confirms "prefer direct Bash over nested Agent delegation for large-file scans" is a real workaround, not just a guess — happy to have that be the interim guidance until there's a product-level guard.The self-authored note upfront is appreciated - it's actually more useful than it might seem, because the memory layout described (1.7GB to 26GB RSS in ~10 min, swap untouched) is consistent with a fast allocation spike from a tree of in-flight Workflow/Agent calls each holding their own context buffers in memory simultaneously.
One thing worth capturing for the team: was there a depth limit or a fan-out width limit explicitly set in the workflow, or did the nesting happen because each subagent was itself instructed to "break this into subtasks and use subagents"? The distinction matters for the root cause - one is a missing depth cap in the scheduler, the other is a model behavior issue where instruction patterns cause runaway fan-out.
For anyone hitting this before a fix lands:
The OOM cascade taking down unrelated services (ollama, tabbyapi) is a real production risk on shared dev machines. Worth flagging as a severity escalation beyond "memory leak" - it's host-stability territory.
I had no restrictions in place. I asked Claude to review all past sessions and list any sources of friction in the workflow that could be improved. When I came back to check on it, it responded. It wasn't until I looked into an OOM event that I realized what happened. At some point during the process, Claude switched to a different method for gathering the data, probably because the previous method ran out of memory.