[BUG] Attach-stall watchdog kills background sessions waiting on in-flight subagents, then misreports it as "keeps stalling at startup"

Version: 2.1.220 (GIT_SHA 4073f59596e272f39393db4f96abc5f4b10eff21)
Platform: Linux 5.15 (Ubuntu), x86-64
Component: daemon / background sessions (claude agents)

Summary

A background session that has finished its turn while a background subagent is still running produces no PTY output — correctly, there is nothing to redraw. The attach-stall watchdog in the daemon's attach handler treats "zero bytes of output since attach" as a hung worker and kills + respawns it. Each kill orphans the in-flight subagent. After two respawns the watchdog gives up, SIGKILLs the session with outcome failed, and prints:

Session <short> keeps stalling at startup — check ~/.claude/jobs/<short> for logs.

The session never had a startup problem. It booted and responded within ~3s on every attempt. The reported diagnosis sends the user looking in the wrong place.

What makes this a bug rather than intended behavior

The codebase already has a precise predicate for "this session has in-flight work": the inFlight: {queued, tasks, kinds} record written into ~/.claude/jobs/<short>/state.json, consumed by an in-flight predicate that returns true when queued > 0, tasks > 0 (excluding an allowlist of local_bash / in_process_teammate / dream / auto_mode_scan), or kinds includes session_cron.

The version-staleness respawn path (respawnIfIdleStale) consults it and explicitly refuses to respawn, returning reason: "inflight" or reason: "busy":

if (!dm(n) && n.tempo !== "idle") return { respawned: false, reason: "busy" };
if (h7s(n))                       return { respawned: false, reason: "inflight" };

The attach-stall respawn path does not. Its only guards are:

if (q > 0 && D >= q && !b.isKilling && !b.isRetiring && !b.isBooting
    && b.dispatch.launch.mode !== "exec") { ... }

— no tempo check, no in-flight check. It then calls e.kill("SIGTERM") unconditionally. Two code paths that both terminate-and-respawn a worker, one of which has the guard and one of which doesn't, reads as an oversight.

Corroborating: the product already ships explicit orphan-recovery for exactly the state this watchdog creates —

Background agent "<label>" had no completion record after the previous Claude Code process exited, and was automatically restarted from its saved transcript.

so the system knows an in-flight subagent losing its host is a bad state. The watchdog is manufacturing that state on a healthy session.

Timing

The stall window is short. With ycf = 500 (initial delay), uIa = 1000 (tick), tengu_bg_attach_stall_ms defaulting to 5000, and a floor of 2000 or 12000:

W = max(floor, tengu_bg_attach_stall_ms)
q = max(1, ceil((W - 500) / 1000))     // 5 ticks at the 5000 default

So roughly 5–12 seconds of silence after attach is enough to trigger a kill, and the detector only requires that the attached client received zero bytes (L === 0). An idle session waiting on a long-running background agent trivially meets that bar.

What I couldn't determine

I diagnosed this from the shipped binary plus local logs, not from source, so one thing
stays open: I can't distinguish "the worker was healthy and correctly silent" from "the
worker was genuinely wedged and never answered the daemon's repaint request." The
artifacts don't settle it — the worker was alive and had booted, but I have no record of
whether it serviced the rv repaint message.

That ambiguity doesn't change the reportable defect, which holds under either reading:

• If the worker was healthy, this is a watchdog false-positive that kills good sessions.
• If the worker was wedged, the watchdog fired correctly — but the remediation still

destroyed in-flight background work without checking for it, and still reported a
startup failure for a session that started fine three times.

Observed sequence (~/.claude/daemon.log)

12:34:58.604  bg spawned d7195a42 (spare)
12:35:42      session launches a background Explore subagent
12:35:50      session ends its turn; transcript records pendingBackgroundAgentCount: 1
12:37:11.336  bg settled d7195a42 (killed)          <- attach-stall respawn, attempt 0
12:37:11.433  bg claimed-spare d7195a42 (respawn)
12:39:49.803  bg settled d7195a42 (killed)          <- attach-stall respawn, attempt 1
12:39:49.888  bg claimed-spare d7195a42 (respawn)
12:41:01.781  bg settled d7195a42 (failed)          <- attempt 2, gave up, SIGKILL

Three strikes matches attachStallRespawns >= 2 in the give-up branch. failed is the only occurrence in the entire local daemon.log history (against 94 done, 66 killed, 17 crashed), i.e. this give-up path had never fired before on this machine.

The subagent transcript shows the cost. On each restart it replays its ~17k-token transcript, emits a thinking block and one lead-in sentence, and is killed before issuing a single tool call — every turn ends with stop_reason: null and no tool_use block:

12:35:55  "Found ... directories. Let me explore in parallel."
12:37:14  "Found key leads immediately. Let me dig in."               (after restart 1)
12:39:54  "Found strong leads. Let me dig into the ... directories."  (after restart 2)

Zero forward progress across all three cycles. The work was never completed and the session died.

Also worth noting: the first kill wrote state: "stopped" / firstTerminalAt into state.json, so the job was already marked terminal at 12:37:11 while the daemon went on to respawn it twice more — the job-list state and the daemon's view disagreed for the remaining ~4 minutes.

Ruled out

• Not OOM — 121 GB available, no OOM records in dmesg or journalctl.
• Not disk — 420 GB free on /.
• Not the subagent's workload — the process dies before any tool call is emitted; the two tool calls that did run returned in under 2s.
• Not auth or startup — the worker booted and the model responded within ~3s on every attempt, which is precisely why "stalling at startup" is the wrong diagnosis.
• Not version-staleness respawn — worker and daemon were both 2.1.220, so isVersionStale is false and that path returns not-stale.

Suggested fix

1. Consult the existing in-flight predicate before an attach-stall kill, exactly as respawnIfIdleStale already does. A session with inFlight.tasks > 0 or inFlight.queued > 0 should never be killed for not repainting.
2. More generally, "no PTY output since attach" is orthogonal to worker liveness. The worker is already sending rv heartbeats (lastRvHeartbeat, with a separate 120s stall threshold used only for the tengu_bg_worker_stalled telemetry event). Gating the attach-stall remediation on heartbeat loss — rather than on absence of a repaint — would avoid this class of false positive entirely.
3. Fix the give-up message. "keeps stalling at startup" is emitted regardless of whether the session ever failed to start; here it started fine three times. Something like "session is not redrawing" would point at the real symptom.
4. On give-up, surface that in-flight background agents were discarded, so the loss isn't silent.

Repro sketch

1. Start a background session (claude agents, or any bg job).
2. Have it launch a background subagent that runs for several minutes, then end its turn with a final reply while the subagent is still in flight.
3. Attach to that session in the fleet view and leave it attached.
4. The session gets killed and respawned twice, then SIGKILLed with "Session ... keeps stalling at startup", and the subagent's work is lost.