[BUG] CLI hangs indefinitely when SSE stream dies silently — no inactivity watchdog

Resolved 💬 3 comments Opened Mar 21, 2026 by WYSIATI Closed Mar 21, 2026

Summary

When a streaming SSE connection dies silently (no TCP RST packet), the CLI hangs indefinitely. There is no watchdog timer monitoring the SSE stream for event gaps. Common triggers: NAT timeout during extended thinking, silent Wi-Fi disconnect, load balancer idle timeout. Users must kill the process manually.

Root Cause Analysis

The streaming API call iterates over SSE events using an async generator:

for await (const event of stream) {
  // process event
  yield event;
}

When the TCP connection dies silently (e.g., NAT drops the mapping), the read() call on the response body stream blocks forever — there's no FIN or RST to signal the connection is dead.

Why silent deaths happen

  1. NAT gateway timeout: Corporate/home NATs expire idle TCP mappings after 5-30 min. If the API is "thinking" (extended thinking for Opus can take 60s+), no events flow. The NAT drops the mapping. Server sends events into the void. Client waits forever.
  2. Network transition: Wi-Fi drops briefly without sending RST. Old TCP session is gone.
  3. Load balancer timeout: CDN/API gateway closes idle connections. No RST reaches the client through the proxy chain.

Current timeout constants (from minified cli.js)

uF9 = 1800000  // 30 min — max operation timeout
BF9 = 20000    // 20 sec — retry delay cap
mF9 = 600000   // 10 min — fallback timeout

None of these is a stream inactivity watchdog. They appear to be operation-level timeouts, not event-gap detectors. The Anthropic API sends periodic SSE ping events during extended thinking, but without a watchdog to detect when pings stop arriving, they provide no protection.

Proposed Fix

class StreamWatchdog {
  private timer: ReturnType<typeof setTimeout> | null = null;

  constructor(
    private readonly timeoutMs: number,
    private readonly onTimeout: () => void,
  ) {}

  /** Call on every SSE event (including ping/heartbeat) */
  kick() {
    if (this.timer) clearTimeout(this.timer);
    this.timer = setTimeout(() => this.onTimeout(), this.timeoutMs);
  }

  stop() {
    if (this.timer) {
      clearTimeout(this.timer);
      this.timer = null;
    }
  }
}

// Integration into streaming API call:
async function* streamApiResponse(request: ApiRequest, options: StreamOptions) {
  const timeoutMs = parseInt(
    process.env.CLAUDE_CODE_STREAM_TIMEOUT || '45000', 10
  );

  const abortController = new AbortController();
  const watchdog = new StreamWatchdog(timeoutMs, () => {
    debug(`Stream watchdog: no events for ${timeoutMs}ms, aborting`);
    abortController.abort(new Error('Stream inactivity timeout'));
  });

  try {
    const stream = await client.messages.create({
      ...request,
      stream: true,
      signal: AbortSignal.any([options.signal, abortController.signal]),
    });

    watchdog.kick(); // Start watchdog

    for await (const event of stream) {
      watchdog.kick(); // Reset on every event
      yield event;
    }
  } catch (error) {
    if ((error as Error).message === 'Stream inactivity timeout') {
      // Convert to retryable connection error
      const err = new Error('Connection lost during streaming');
      (err as any).code = 'ESTREAM_TIMEOUT';
      throw err;
    }
    throw error;
  } finally {
    watchdog.stop();
  }
}

Design Decisions

  1. 45-second default: Anthropic API sends SSE pings. Extended thinking can have long pauses, but pings should still arrive. 45s is generous. Configurable via CLAUDE_CODE_STREAM_TIMEOUT
  2. Reset on ALL events: ping, content_block_start, content_block_delta, etc. Any event proves liveness
  3. Custom error code ESTREAM_TIMEOUT: Integrates with the proposed connection retry logic — stream timeouts should be retried
  4. AbortSignal.any(): Composes user abort (Escape) with watchdog abort — whichever fires first wins
  5. No partial response recovery: When watchdog fires, the entire response is aborted and retried from scratch. Partial recovery is a separate feature (#26729 Layer 3)

Testing Plan

Unit Tests

| Test | Input | Expected |
|------|-------|----------|
| Fires after timeout with no kicks | Watchdog(1000ms), no kick | onTimeout at ~1000ms |
| Resets on kick | Watchdog(1000ms), kick at 800ms | onTimeout at ~1800ms |
| No fire with frequent kicks | Kick every 500ms for 5s | Never fires |
| Stop cancels timer | Stop at 500ms | Never fires |
| Multiple rapid kicks | 100 kicks in 10ms | Only 1 active timer |
| Custom timeout from env var | CLAUDE_CODE_STREAM_TIMEOUT=10000 | 10s timeout |
| Default is 45s | No env var | 45000ms |

Integration Tests

| Test | Setup | Expected |
|------|-------|----------|
| Normal stream completes | Events every 100ms for 5s | Completes, watchdog never fires |
| Extended thinking with pings | No content 30s, pings every 10s | Watchdog reset by pings |
| Dead connection detected | 3 events then silence 46s | ESTREAM_TIMEOUT at 45s |
| Watchdog + user Escape | Dead connection + Escape | First signal wins, no crash |
| Timeout triggers retry | Dead connection | ESTREAM_TIMEOUT retried (with connection retry fix) |
| Cleanup on normal completion | Stream completes | Timer cleared, no leaks |
| Cleanup on error | API returns 500 | Timer cleared |

E2E Tests

| Test | Steps | Expected |
|------|-------|----------|
| Firewall drop mid-stream | Start prompt → iptables DROP port 443 → wait 50s | "Connection lost during streaming. Retrying..." at ~45s |
| Extended thinking works | Complex Opus prompt | No false timeout; pings keep watchdog alive |
| Escape still works | Dead connection → Escape before 45s | Immediate abort by user |
| CLI doesn't hang | Kill network during streaming | Error message within 45s, prompt returns |

Impact

Prevents the worst possible UX: infinite hang requiring kill -9. The fix is ~40 lines with clear boundaries. Pairs naturally with the connection retry fix (issue #37077) for automatic recovery.

Related Issues

  • #26729 — Streaming Resilience: Detect network loss, save state, auto-resume
  • #5674 — Persistent ECONNRESET (often manifests as hangs during streaming)

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗