Remote Control Bridge: Critical race conditions, silent failures, and reliability issues

Summary

Deep analysis of the remote control bridge (src/bridge/, src/cli/transports/) reveals 15 reliability issues — 6 critical, 3 high, 6 medium — that compound to produce the flaky "just doesn't work" behavior users experience. The root cause across most issues is fire-and-forget async operations (void promise) combined with insufficient serialization of the auth/rebuild lifecycle.

This writeup documents each issue with file locations, reproduction scenarios, and proposed fixes.

---

Critical Issues

1. Race condition: concurrent auth refresh + 401 recovery

Location: src/bridge/remoteBridgeCore.ts:334-372

The authRecoveryInFlight boolean guard is synchronous, but both the proactive JWT refresh timer and the SSE 401 handler are async. After events like laptop wake or network restoration, both paths fire near-simultaneously:

1. Proactive refresh timer fires → sets authRecoveryInFlight = true → awaits /bridge
2. SSE receives 401 → recoverFromAuthFailure() checks authRecoveryInFlight
3. If proactive fetch resolves first → flag resets to false → 401 handler also runs
4. Two /bridge calls → two epoch bumps → first rebuild uses stale epoch → 409 Conflict

The code at lines 330-332 acknowledges this risk in a comment but the boolean flag is fundamentally insufficient for async coordination.

Proposed fix: Replace the boolean flag with a mutex or a shared Promise that both paths await. Only one /bridge call should be in-flight at a time:

let recoveryPromise: Promise<Credentials | null> | null = null

async function refreshCredentials(cause: string): Promise<Credentials | null> {
  if (recoveryPromise) return recoveryPromise
  recoveryPromise = doRefresh(cause)
  try { return await recoveryPromise }
  finally { recoveryPromise = null }
}

---

2. Fire-and-forget writes lose session archival

Location: src/bridge/remoteBridgeCore.ts:678, 811

void transport.write(makeResultMessage(sessionId))  // enqueued, not awaited
// ... shortly after:
transport.close()  // uploader killed, pending writes dropped

The result message that archives the session is enqueued but close() kills the SerialBatchEventUploader before it flushes. The server never archives the session, leaving it stuck in "in progress" — users cannot re-enable remote control until it times out.

The same pattern at line 811 (void transport.writeBatch(events)) means normal message writes can also be silently lost.

Proposed fix: await the write before closing, or add a drain() method to CCRClient that flushes pending writes before shutdown:

await transport.write(makeResultMessage(sessionId))
transport.close()

---

3. FlushGate deadlock on 401 during initial history flush

Location: src/bridge/remoteBridgeCore.ts:408-413

if (transport !== flushTransport || tornDown || authRecoveryInFlight) {
  return  // ← does NOT drain or drop the gate
}
drainFlushGate()

If a 401 arrives while the initial history flush is still running, this guard returns early without calling drainFlushGate() or flushGate.drop(). Messages are stuck in the queue permanently — the bridge appears completely hung. flushGate.deactivate() exists (in flushGate.ts:68-70) but is never called in this path.

Proposed fix: Add flushGate.drop() or flushGate.deactivate() to the early-return path so queued messages are released (or discarded cleanly) instead of orphaned.

---

4. Delivery acknowledgments silently swallowed

Location: src/cli/transports/ccrClient.ts:968

reportDelivery(eventId, status): void {
  void this.deliveryUploader.enqueue({ eventId, status })  // errors discarded
}

During transport rebuilds, close() is called on the uploader, causing enqueue() to return without delivering. The server never learns the client received events → re-queues them → phantom duplicate prompts and repeated tool calls.

Proposed fix: Either buffer delivery acks across rebuilds and replay them on the new transport, or make reportDelivery return a Promise so callers can handle failures.

---

5. Inbound events dropped during transport rebuild window

Location: src/bridge/remoteBridgeCore.ts:486-526

Between flushGate.start() and wireTransportCallbacks(), the new transport can connect and begin receiving SSE events before the setOnData callback is wired. Those events (including user messages) are silently dropped.

flushGate.start()
  ↓ (live writes now queue)
transport = await createV2ReplTransport(...)  // new transport created
  ↓ (gap: transport connected, callbacks NOT wired)
wireTransportCallbacks()  // callbacks wired here
transport.connect()
drainFlushGate()

Proposed fix: Wire callbacks on the transport before calling connect(), or create the transport in a paused state and only start the SSE stream after callbacks are attached.

---

6. Stream accumulator state abandoned on rebuild

Location: src/cli/transports/ccrClient.ts:283-284

Each rebuildTransport() creates a new CCRClient with a fresh streamTextAccumulator. If text deltas arrive on the old SSE stream after rebuild starts but before the old transport fully closes, the old accumulator flushes a partial snapshot (e.g., just the tail of a message instead of the full text). Streaming responses appear corrupted to the user.

Proposed fix: Either transfer accumulator state to the new CCRClient, or ensure the old transport's SSE stream is fully closed and drained before the new one is created.

---

High Severity Issues

7. Permission responses dropped during 401 recovery

Location: src/bridge/remoteBridgeCore.ts:825-845

sendControlResponse(response) {
  if (authRecoveryInFlight) return  // silently dropped
  void transport.write(event)
}

If a user answers a permission prompt while 401 recovery is in-flight, the response is discarded. The server blocks waiting, times out (~10-14s), and the tool call fails. On high-latency networks this window is wider.

Proposed fix: Queue control responses during recovery and flush them when the new transport is ready, similar to how the FlushGate handles regular messages.

---

8. Heartbeat timer leak on epoch mismatch

Location: src/cli/transports/ccrClient.ts:677-703

handleEpochMismatch() has return type never (throws), but the heartbeat timer calls void this.sendHeartbeat() which swallows the exception. The timer keeps firing, hitting 409 every 20s in a loop — wasting bandwidth and spamming logs.

Proposed fix: stopHeartbeat() should be called inside handleEpochMismatch() before throwing, or the heartbeat tick should catch and check for epoch errors specifically.

---

9. Null token edge case in 401 recovery

Location: src/bridge/remoteBridgeCore.ts:543-545

const stale = getAccessToken()
if (onAuth401) await onAuth401(stale ?? '')
const oauthToken = getAccessToken() ?? stale  // both could be null

If getAccessToken() returns null on both calls (token cleared during onAuth401), oauthToken is null. The subsequent null-check exists but fetchRemoteCredentials could receive null if the guard is bypassed in future refactors.

Proposed fix: Explicit null check with early return and state transition to 'failed' before proceeding.

---

Medium Severity Issues

10. BoundedUUIDSet eviction enables duplicate tool execution

Location: src/bridge/bridgeMessaging.ts:435, src/bridge/remoteBridgeCore.ts:265

The FIFO ring buffer unconditionally evicts old UUIDs when full. Under high message throughput, a server retry of an older message finds its UUID already evicted → treated as new → duplicate tool call. The default uuid_dedup_buffer_size may be too small for sustained conversations.

Proposed fix: Either increase the default buffer size significantly, or switch to a time-based expiry (TTL) instead of count-based eviction.

11. SSE partial batch delivery loses events permanently

Location: src/cli/transports/SSETransport.ts:264-266

Last-Event-ID is set to the highest sequence number seen. If a batch is partially delivered (e.g., seq 95-105, only 100-105 processed), reconnection resumes from 106 — events 95-99 are permanently lost.

Proposed fix: Track the highest contiguous sequence number rather than the absolute highest.

12. Silent transport failures for up to 120s

Location: src/cli/remoteIO.ts:189-193

Keep-alive write failures are logged but not propagated. A dead transport goes unnoticed until the next liveness timeout fires.

13. Close reason codes not passed to onClose handler

Location: src/cli/remoteIO.ts:105-109

All close reasons (auth failure, epoch mismatch, intentional close) appear identical to RemoteIO. No way to differentiate failure modes.

14. Initial prompt async iterator errors swallowed

Location: src/cli/remoteIO.ts:202-214

Fire-and-forget async iterator — if it throws, the error is silently discarded.

15. Sequence number pruning assumes dense numbering

Location: src/cli/transports/SSETransport.ts:371-378

The pruning threshold lastSequenceNum - 200 assumes monotonic dense sequences. Server-side batching could create gaps that confuse dedup.

---

Compounding Failure Scenarios

These issues don't just fail independently — they chain together:

| Scenario | Issues | User experience |
|----------|--------|-----------------|
| Laptop wake / network blip | #1 + #3 + #6 | Double refresh → 409 → rebuild → flush deadlock → bridge hung |
| "Can't re-enable remote control" | #2 | Result message dropped → session stuck "in progress" indefinitely |
| Duplicate tool calls | #4 + #10 | Events re-delivered + dedup eviction → tools run twice |
| Permission prompt timeout | #7 | User answers during 401 window → response silently dropped → timeout |
| Garbled streaming output | #5 + #6 | Accumulator abandoned + events dropped → corrupted responses |

Suggested Test Coverage

• [ ] Simulate concurrent proactive refresh + 401 recovery, confirm only one /bridge call is made
• [ ] Verify session archival message is sent before transport close
• [ ] Verify FlushGate is properly drained/dropped on all early-return paths
• [ ] Verify delivery acks survive transport rebuilds
• [ ] Verify transport callbacks are wired before SSE stream connects
• [ ] Verify permission responses sent during 401 recovery are queued and replayed
• [ ] Verify heartbeat timer is stopped on epoch mismatch
• [ ] Stress test BoundedUUIDSet under high throughput with server retries