Background tasks mass-killed by memory-pressure reaper on Linux hosts with low MemFree but high MemAvailable (PSI ~0)

Corroborating from a second Linux host — v2.1.216, same memory signature, plus one new wrinkle: the reap kill does not take down the child process tree (filed separately as #79727).

Environment: Ubuntu, kernel 6.8.0-111-generic, x86_64, 16 CPU, 62 GB RAM; interactive CLI in tmux. At kill times: MemFree ≈ 1.9 GB, MemAvailable ≈ 41 GB, buff/cache ≈ 40 GB, /proc/pressure/memory avg10/60/300 all 0.00 — the same "chronically low MemFree, zero actual pressure" steady state you describe.

Kill events: one interactive session spanning ~20 h lost 4 background tasks (all run_in_background, no timeout set): 2026-07-20 13:43:15 UTC (a long CLI subprocess dispatch), then a burst of three within 10 minutes on 2026-07-21 — 09:27:49, 09:33:45, 09:37:49 UTC — which took out every long-running tracked task of that period, one per dispatch, as fast as they were re-issued. All reported <status>killed</status> with the generic Background command "…" was stopped. No TaskStop in the transcript, nobody at the keyboard during the burst, journald/dmesg clean at the kill seconds, host cron audited (no kill logic).

Discriminator consistent with your per-process listener mechanism: during the burst, an identical job started with nohup … & outside the task system survived to completion on the same box while its tracked twins died.

One divergence from the "kills the tree" reading of the disassembly: in all three of our suite kills only the tracked shell died — the make child tree survived as orphans and kept running to completion; overlapping orphans then corrupted shared test state and produced a phantom "regression". A manual TaskStop on the same version does kill the whole tree (verified with a parent→child heartbeat test), so the reap path appears to use a weaker kill than the stop path. Details in #79727.

+1 to keying the reaper on MemAvailable and/or PSI rather than MemFree, and to attributing reaper kills in the task notification — the unattributed was stopped cost us two days of misdirected forensics (user action, cron, OOM, lease contention were all suspected first).

Adding evidence on what emits the memoryPressure event on Linux — it is not a free-memory read, which explains why MemFree, MemAvailable and the PSI averages all look uncorrelated with the kills.

Environment: Claude Code 2.1.251 (native installer), WSL2 kernel 6.6.114, 44 GB RAM, /proc/pressure/memory present. Same signature as above: background tasks across 3 idle sessions killed in a 28-minute window, every live task of a session in the same second, two sessions in different processes 4 ms apart, <status>killed</status> / Background command "…" was stopped, MemAvailable 37% at every instant, no OOM, no TaskStop.

1. The listener arms a PSI trigger on /proc/pressure/memory. Watching the claude process's fd table while a background task is live:

# no background task running: no such fd (3 claude processes scanned)
# one background task running, sampled once per second for its lifetime:
fd 21 -> /proc/pressure/memory
fd 21 -> /proc/pressure/memory
... (8/8 samples)

The fd is present for the task's whole lifetime and absent whenever no background task is live. That is the shape of a PSI trigger (a spec written to the file, then polled) — not a periodic /proc/meminfo or os.freemem() read. The runtime side confirms the vocabulary: the bundle's runtime symbol table carries memoryPressureWatcherHasOsBackend and memoryPressurePsiTrigger, and the only trigger-spec-shaped string in the binary is:

some 150000 2000000

i.e. fire when tasks are stalled on memory for 150 ms within any 2 s window. (The spec actually written to the fd can't be read back from /proc, so this is the strongest inference available from outside the runtime, not a read of the source.)

2. Why the averages read 0.00 at the kill. A trigger fires on the raw 2 s window, before avg10/avg60 reflect it. In our incident some avg60 read 0.00 through the kill instant (5 s sampling) and bumped to 0.34 shortly after it — which is what one ~200 ms stall at the kill looks like once the 60 s average catches up. sar at 10-minute resolution cannot see it at all. So "PSI ~0" in the averages is consistent with the reaper firing, not evidence against it; the discriminating observation is the trigger window, which nothing in /proc exposes after the fact.

**3. The idle gate is 30 minutes against user interaction.** In 2.1.251 the early-return is Date.now() - lastInteractionTime() < 1800000, where the timestamp is the terminal user-presence one. Harness-injected turns (background task notifications, hook output) do not count, so a headless or long-running background session is permanently past the gate after its first half hour. The remaining guards are: task still running and not yet notified, main loop idle, and no live non-shell task (agent/teammate) in the registry. Only main-thread tasks register the listener — a subagent's background tasks are not pressure-reaped.

4. The [killed] sink has at least five authors, and three of them are benign. This is the part that most affects anyone trying to reproduce or count these, and it corrects the framing in my own report. That exact 10-byte body is written by a shared helper, so it is not a signature of an external reap. On our host it is produced by at least: the pressure reaper, the agent-run cleanup path, an explicit TaskStop, a Monitor that reaches its timeout_ms, and session wind-down. The monitor case is the nastiest — the [Monitor timed out — re-arm if needed.] string is delivered to the session as a notification and is never written into the task's output file, so an expired monitor leaves a bare 10-byte sink byte-identical to a mystery kill.

The discriminator is the session transcript, not the sink. TaskStop appears as a tool use; monitor expiry appears as a notification carrying the task id. Both are operator-caused and both land on the same body.

5. A counting trap worth flagging for this thread. grep -l '\[killed\]' over the task-output directory over-counts, because task outputs that discuss this bug contain the literal string as prose — in our case, upstream issue drafts about this very problem counted themselves as victims. The correct test is a file whose content ends with the marker, since it is appended after whatever the task had already written. Re-counting our own window on that basis moved it in both directions: 4 false positives removed, 3 real sinks recovered that a substring count had missed.

Corrected figures for the 06:00–08:30Z window: 11 true sinks across 63 background-task outputs (17.5%), of which 3 are self-inflicted (1 TaskStop, 2 expired monitors), leaving 8 unexplained (12.7%).

6. What survives and what does not, on the "one actor swept the session" reading. "Every in-flight task in a session died together" is not discriminating on its own — a parked-owner cleanup path (killAsyncAgent, which reaps the shell tasks of a parked owner) produces exactly that shape within a single session, and we had not considered it. What still needs a host-wide actor is the cross-session pair: two tasks in two different sessions in two different processes, killed 4 ms apart. That observation is unchanged.

7. Consequences for the report above. Low MemFree is not the input either — the trigger is sensitive to transient reclaim stalls, which a loaded host with a large page cache produces routinely even with tens of GB reclaimable (ours was at load ~250 on 32 CPUs when it fired). That matches the "high MemAvailable, PSI averages ~0, kills anyway" pattern in every report in this cluster. Two things would make this diagnosable and survivable:

• Key the reap on sustained pressure (e.g. full rather than some, or a longer window with a higher stall fraction) rather than a single 150 ms some stall — the current spec fires on ordinary reclaim activity on a busy box.
• Name the actor in the notification. All five paths above write the identical 10-byte sink and the identical was stopped line; the only distinguishing record is a debug-channel line the user never sees. Attributing a kill to a code path took a multi-hour cgroup/PSI/journal investigation that a single word in the notification would have replaced. This is the single highest-value fix in the report, and the sink-ambiguity finding above raises its value rather than lowering it.

Workaround, with an A/B. CLAUDE_CODE_DISABLE_BG_SHELL_PRESSURE_REAP is still the guard on the process.on("memoryPressure", …) registration in 2.1.251. With it set: 3 true sinks across 117 background-task outputs, all 3 explained (one deliberate TaskStop, one expired monitor, one wind-down test), zero unexplained — against 8 unexplained in 63 outputs before it. P(0 unexplained | n=117, p=0.127) ≈ 1.2e-7. Note the flag is read at listener-registration time, so it takes effect for newly started sessions rather than running ones.

Follow-up, and a correction to my own point 7 above: the pressure reaper writes no debug line at all. I built the instrument my last comment implied existed, ran it for 21 hours, and read 2.1.251's kill paths directly. The headline is that this kill is currently unattributable from the client side — which makes the "name the actor" ask stronger, not weaker.

1. Six paths reach the common killer. Three name themselves. The reaper is not one of them.

The shared kill helper logs exactly one line:

LocalShellTask <id> kill requested

Three callers log a preceding line naming themselves — agent-run cleanup, the parked-owner cascade, and session wind-down:

killShellTasksForAgent: killing orphaned shell task <id> (agent <id> exiting)
killAsyncAgent: killing orphaned shell task <id> (parked owner <id> killed)
print wind-down: killing background shell <id> ("<description>") after <n>ms grace

The pressure reaper does not. Its registration in 2.1.251, with the minified identifiers kept:

if (d === void 0 && !$e() && !a.CLAUDE_CODE_DISABLE_BG_SHELL_PRESSURE_REAP) {
  let v = () => {
    let A = r.get(e);
    if (A?.status !== "running" || A.notified || Date.now() - Eh() < ylr || nme() || tL(r.all())) return;
    _("task_local_shell_pressure_reap"), lNe(e, t, "killed", void 0, ...), jU(e, r)
  };
  process.on("memoryPressure", v)
}

jU is the common killer (it emits the kill requested line above). But the reaper's own identifier goes to _, and _ is the telemetry emitter, not the debug logger. The bundle carries ~500 distinct snake_case event names through that call — task_stop_user, mcp_oauth_flow, git_worktree_create, native_cleanup_versions and so on — whereas every self-naming path above goes through the debug logger with an interpolated human sentence.

Consequence: a pressure reap leaves only the generic kill requested line, which is byte-identical in the debug channel to a TaskStop, an expired Monitor, or a stop from the task UI. So running with DEBUG=1 does not let a user confirm this bug. My point 7 above said the attributing record was "a debug-channel line the user never sees"; for the reaper there is no line to hide. Verified empirically as well as by reading: across the whole instrumented corpus, every occurrence of task_local_shell_pressure_reap in the debug logs turned out to be command-text echo from my own sessions investigating this, and the only genuine self-naming line ever captured was from a deliberate wind-down test.

There is currently no client-side artifact anywhere — sink body, user notification, or debug log — that distinguishes a pressure reap from an operator stop. Moving that existing _(...) event to (or duplicating it on) the debug logger would be a one-line change and would make the bug confirmable by the people reporting it.

2. A 21-hour instrumented window — reported with its limits, because they matter

Same host, reaper enabled, debug capture live: 173 background-task outputs, 2 true sinks, both attributed, zero attributable to the reaper. One was an explicit TaskStop issued 5 ms before the debug line. The other had no named line, no TaskStop and no monitor armed — its pressure interval is what cleared it (§3).

This is not a retraction and not a failed reproduction. The 12.7%-unexplained window in my comment above was measured on a host at load ~250. This one was quiet: some avg10 and avg60 read 0.00 in all 80 sampled intervals and available memory never fell below 15.6 GiB. The reaper's gate was barely ajar, so the window is unmeasured, not clean — a quiet box can run hundreds of background tasks and offer the reaper zero opportunities. I am reporting it for the method, not as evidence against the report.

3. A cheap one-sided exclusion test, for anyone else in this thread

Sample /proc/pressure/memory's some total= counter on a fixed interval and store the per-interval delta. The trigger spec is some 150000 2000000 — 150 ms of stall inside any 2 s window. If the cumulative stall across an entire interval is below 150000 µs, then no 2 s sub-window inside it can have reached the threshold, so the reaper is excluded for any kill in that interval.

That is how the second sink above was cleared: 63 ms cumulative against a 150 ms threshold.

It is strictly one-sided, and worth saying so because the inverse reading is tempting and wrong: a delta above the threshold proves nothing. Mine reached 941 ms spread across 571 s with avg10 still 0.00 — cumulative micro-stalls, not a 150 ms burst. Only the below-threshold direction is evidence.

4. One correction in the other direction, in favour of the report

Reading the registration settles a piece I had previously only inferred from statistics. process.on("memoryPressure", ...) is a host-wide signal driving per-session kills — which is exactly the mechanism the cross-session observation needs (two tasks, two different sessions, two different processes, killed 4 ms apart). That shape now follows from the registration rather than resting on the timing coincidence, and no per-session cleanup path can produce it.