Background Bash tasks (run_in_background) killed mid-run silently — no OOM, not user-initiated; setsid-detached processes immune

Status Open
Maintainer reply None cached
Activity 4 comments · opened Aug 6, 2026

Summary
Long-running background Bash tasks launched with run_in_background: true are killed mid-run with no OOM, no user action, and no error surfaced to the session. Observed ~10 times across 2026-07-28 → 2026-08-06 on the same host. Tasks die silently; the session only notices when it polls output and finds the process gone.

Environment

  • NVIDIA DGX Spark, aarch64, Ubuntu 24.04, 121 GB RAM, Linux 6.17.0-1014-nvidia
  • Claude Code background sessions (claude.ai/code), long-lived (multi-day, with context compaction cycles)
  • Killed workloads: CPU/RAM-bound Python pipelines (uv run python …), 20–120 min runtimes, 10–40 GB RSS (well under host RAM)

Evidence

  • dmesg / journalctl: no oom-kill events at the kill timestamps
  • User explicitly confirmed the tasks were not killed manually
  • Identical commands re-run in the foreground completed fine
  • 100% reliable workaround: setsid nohup <runner>.sh & (fully detached from the harness process tree) plus a Monitor tailing a log file — detached runners have never been killed. This points at the harness's process-group lifecycle management (a cleanup pass reaping the background task's process group on some internal event — compaction, tool-result timeout, or session reconnect) rather than the OS.

Repro sketch

  1. In a long-lived session, run a 30+ minute CPU-heavy command with run_in_background: true.
  2. Continue interactive work in the session (including context compaction).
  3. Task dies mid-run (observed 5–40 minutes in), no notification, exit status lost.

Ask

  • Identify what harness event reaps background-task process groups mid-session.
  • Surface any kill of a background task to the session as an explicit event with a reason.

View original on GitHub ↗

4 Comments

aleks-azen · 23 days ago

Corroborating this on macOS (Darwin 25.5.0, Claude Code 2.1.221, CLI), with signal-level detail that may narrow it.

Same phenomenon, and I can independently confirm your key finding: detached processes are immune. A nohup-detached poller of mine survived an event that killed both tracked background tasks in the same session.

Signal sequence

Two probes launched as run_in_background tasks:

  • Probe A — a long-running child trapping HUP/INT/TERM/QUIT/PIPE/USR1/USR2, logging whichever arrives.
  • Probe B — the tracked wrapper itself made signal-proof:

trap "" TERM HUP INT QUIT; while true; do <child>; rc=$?; [ $rc -lt 128 ] && exit $rc; sleep 1; done

At the kill:

Probe B: died with zero log output    -> wrapper took an uncatchable signal (SIGKILL)
Probe A: SIGTERM ... ppid_now=1       -> child signalled only after its parent was already gone

So it is SIGKILL to the shell the harness tracks, then SIGTERM to the orphaned descendants — not SIGTERM-first as described in #76942. Nothing in userspace survives it: traps, supervisor loops and retry wrappers are all defeated, which is consistent with fully detached processes being the only survivors.

Scoping: the owning instance, not a machine-wide reaper

A 1-second process sampler captured the full set of harness-tracked shells across the host before and after one event:

  • 2 killed — both belonging to a single claude instance
  • 9 survived — across 6 other claude instances, same host, same user, same second

The claude processes themselves were untouched (uptime in days). Whatever issues the kill is working from the instance's own task list; this is not an OS-level or terminal-level sweep.

Age independence

Across 293 kill events in local transcripts, victim age at kill ranged from 112 s to 27 h with no clustering. That argues against the "only after a session runs long" framing in #76942 and against any fixed runtime cap.

Local correlates ruled out

On this host: OOM/jetsam (pressure level normal, no kill events), memory pressure, task age, sleep/wake, per-uid process limits (752 of 8000), Claude Code updates, plugin hooks, terminal-multiplexer process-reaping features, and turn boundaries. Idle time does not explain it either — of 166 cluster-leading kills, 48 fired at under 5 minutes of session idle and 28 at over 4 hours.

No mobile, web, or desktop client was attached to this host, so the bridge-related paths in #82586 / #78860 / #63023 do not appear to apply here.

One addition to the ask

On the CLI these kills are surfaced — as a task notification with <status>killed</status>. That is better than silence, but the surfacing is itself the dominant cost: each notification wakes the agent for a full turn against a full context. In my transcripts that is 116 wakeups and roughly 701k output tokens spent purely on noticing and relaunching. A background task the harness terminated for its own internal reasons could reasonably be auto-restarted, or at minimum carry a reason so an agent can distinguish it from a real completion without spending a turn to investigate.

sergeiwallace · 11 days ago

Related report, cross-linked in case it's useful: #88071 — same broad symptom (background run_in_background task killed, no OOM, not user-initiated), though our evidence points to a specifically turn-end-plus-idle trigger rather than the compaction/reconnect-cleanup candidate this issue's evidence suggests, so possibly a different specific trigger under the same general harness process-group lifecycle behavior.

One finding that might be useful here even if the exact trigger differs: making a blocking wait call on the background task in the same turn it's launched (rather than letting the turn end and relying on the async completion notification) prevented the kill in our controlled test — a task well past the point where it would normally have been killed completed cleanly. Might be worth trying alongside your setsid nohup workaround to see if it also helps in your environment.

msshives-gif · 9 days ago

We caught these kills with signal logging and can confirm the harness hypothesis directly: the SIGTERM comes from the session's own claude process.

Setup: Claude Code v2.1.231, Ubuntu 24.04 (x86_64), interactive session in tmux. We run background commands under a small wrapper (installed via a PreToolUse hook) that logs any signal it receives before forwarding it to the real command. The kernel siginfo supplies the sender pid, uid, and signal source; the wrapper resolves that pid's command line from /proc.

The clearest capture (timestamps from the wrapper log and the session transcript):

  1. 22:32:34 — the model starts a long simulation as a background task (run_in_background: true). The command ended in a pipeline: godot --headless ... 2>&1 | tee run.log | tail -12.
  2. 22:32:45.346 — the transcript records the task notification being enqueued: status: killed, "Background command ... was stopped".
  3. 22:32:45.628 — the wrapper records receiving SIGTERM, ~282ms later. Sender: pid of the session's own claude process (verified against ~/.claude/sessions/<pid>.json), si_code 0 (a plain kill()), uid = the logged-in user. Together with the ordering, this shows Claude Code had already entered an internal stop path before the signal landed.
  4. The task's output file contains only [killed].

What it wasn't: No stop-tool call or user input occurred; the model had ended its turn and reacted to the notification on the next turn. The SIGTERM came from Claude Code — not OOM (kernel log clean), not another process.

Other observations that may help narrow it down:

  • The log holds three attributed kills total, all SIGTERM from the same session's claude pid: two on 2026-08-19 (plain cd ... && ... commands, no pipeline) and this one on 2026-08-21.
  • This kill landed ~11 seconds after launch and ~5 seconds after the launching turn's final message — earlier than #87496's 30–90 second window, but the same post-turn pattern.
  • An immediate relaunch of the same command with plain > file redirection instead of the | tee | tail pipeline ran to completion. That's one paired observation, and the two 08-19 kills lacked the pipeline, so the pipeline can't be a necessary trigger.
  • One possible confound on our side: because of the logging wrapper, the process tree under the harness's bash is wrapper → real command rather than the command directly. Most wrapped tasks survive fine (long sims, sleep 400, multi-minute jobs the same evening), but noting it in case the harness treats the extra process layer differently.

Happy to share the raw log records (JSONL) or add more instrumentation if that would help.

benlavalley · 3 days ago

Been fighting these issues for a while now, hope this one gets some attention. I've ramped up development pace significantly over the last few months after optimizing my workflows and have been contending with OOM growing pains (~3.8m LOC Enterprise IT platform) and needing to implement my own instructions and hooks for Claude to detect and kill stale node processes. I finally got those issues seemingly buttoned down, only to find half the time I was fighting these phantom background process kills. Especially painful when multi-hour E2E runs get killed.