Resource governance for agent-spawned process trees (memory caps, pressure feedback, teardown ordering)
Problem
Claude Code executes arbitrary process trees (Bash, background tasks, subagents) on a machine it shares with the user's interactive session, with no resource governance: no memory ceiling, no system-pressure feedback, no guarantee spawned trees die with the session. An agent will also do things a human wouldn't: re-run a heavy command repeatedly, run it concurrently across sessions, or deliberately reproduce a pathological workload while investigating it.
Incident that motivated this (macOS, 48GB)
A session ran a standard test target (pytest -n 4). A third-party SDK's background thread hit a pathological serialization loop (a MagicMock reached its JSON encoder; each pass spawned new mock children), allocating GB/s invisibly to the test harness. One worker reached ~60GB in ~4 minutes → kernel jetsam → machine froze and rebooted, killing every session.
Hours later it happened again — during the agent's own capped reproduction. The agent's watchdog polled ps RSS, and macOS memory compression had moved the ballooning pages out of the resident set: ps reported the tree at 8GB while the kernel's footprint accounting showed 87GB. The instrument the agent could build from inside a session was structurally blind; the machine paid again.
Diagnosis eventually required jetsam-report forensics plus a footprint-based watchdog that SIGABRTs the fattest process (with PYTHONFAULTHANDLER=1) to capture the allocating stack. All of that is generic and automatable — none of it is available from the harness.
Requested capabilities, most valuable first
- Opt-in resource caps on spawned process trees. Like
timeout, but for memory: kill the process group when its combined physical footprint crosses a limit, and put which process and (where cheap) which stack in the tool result. cgroups v2 on Linux; on macOS a polling watchdog over/usr/bin/footprint— RSS is the wrong metric there (compression hides ballooning pages fromps). - System-pressure feedback to the model. Surface "this tree is at NGB and climbing; system free X%" in tool results or as a mid-turn warning. Agents can be defensive only if the harness gives them the signal.
- Teardown ordering. When a session dies, spawned trees should die with it, leaf-work first. In the incident, session teardown killed the agent's watchdog while the multi-GB workload it guarded kept running — guard died before the guarded.
- Cumulative budget across a session's fan-out. Subagents, background tasks, and concurrent sessions multiply footprint with no shared ceiling; a per-session (or per-machine) budget would bound the blast radius.
Happy to share the jetsam forensics recipe and the footprint-watchdog script the diagnosis produced.
Showing cached comments. Read the full discussion on GitHub ↗
3 Comments
The pytest + MagicMock serialization spiral you documented is a great concrete example of a broader pattern: an agent running a test suite has no way to distinguish "this is taking a while because of legitimate test complexity" from "this process has entered a pathological allocation loop." By the time RSS hits 60GB, the damage is done.
A few things that would help at the resource governance layer:
Memory ceiling with a graceful-degradation path: rather than letting the OS OOM killer decide, a configurable memory cap that sends SIGTERM (then SIGKILL after a grace period) to the spawned process group. The agent could then observe the non-zero exit and report it explicitly rather than the session silently running out of headroom.
Process group ownership: the issue with orphaned trees isn't just about memory - it's also that there's no authoritative answer to "what processes did this agent spawn?" Tying spawned process groups to a session ID would let teardown be complete rather than best-effort.
Pressure feedback to the agent: even a simple "you're using more memory than expected, consider narrowing the scope of the next tool call" signal in the context would let the model make a recovery decision rather than just continuing blind.
Running parallel Claude Code sessions as workers in a GTM pipeline and the resource governance problem is real - one runaway subprocess in one worker can affect the whole machine without any session-level containment.
Thanks for the reply. It seems that
CLAUDE_CODE_TOOL_MEMORY_LIMITis built on cgroups of Linux, and macOS has no equivalent kernel facility for it to use. So it seems that there is no claude native way to do memory ceiling on MacOS?Session teardown must kill leaf process trees first — watchdog dying while the workload lives is how machines melt overnight.