[Bug] Claude hangs indefinitely on simple tasks, requires cancellation to resume

Root Cause Analysis (source-level)

_Traced through the published source and verified independently via Codex review._

The "hang" is a 5-layer failure chain where context silently grows past the API limit, reactive compaction takes minutes, and ripgrep crashes prevent recovery.

---

Layer 1: Proactive compaction silently disabled

File: src/services/compact/autoCompact.ts (lines 195–222)

When either REACTIVE_COMPACT (gated by tengu_cobalt_raccoon) or CONTEXT_COLLAPSE is enabled, shouldAutoCompact() returns false — proactive autocompact is completely suppressed. The system relies entirely on reactive compact (after the API rejects the request) to catch overflows:

// autoCompact.ts:195-198
if (feature('REACTIVE_COMPACT')) {
  if (getFeatureValue_CACHED_MAY_BE_STALE('tengu_cobalt_raccoon', false)) {
    return false  // Proactive autocompact disabled
  }
}

This means context grows unchecked until the API returns a 400 error.

Layer 2: Token estimation undercount

File: src/services/tokenEstimation.ts (line 205)

The rough estimator defaults to 4 characters per token:

export function roughTokenCountEstimation(
  content: string,
  bytesPerToken: number = 4,  // Optimistic for JSON/tool-result content
): number {
  return Math.round(content.length / bytesPerToken)
}

A file-type-aware variant (roughTokenCountEstimationForFileType) exists and is used by FileReadTool, but the main tokenCountWithEstimation() (src/utils/tokens.ts:226) uses the generic estimator for tail content. The undercount is concentrated in recent unsent content — especially tool results with dense JSON.

Layer 3: Reactive compact model call = the visible "hang"

File: src/query.ts (lines 1119–1166)

When the API returns "prompt is too long: 200725 tokens > 200000 maximum":

1. The error is withheld from the user (line 811 — not yielded)
2. Context collapse drain is attempted first (lines 1085–1117)
3. reactiveCompact.tryReactiveCompact() is called (line 1120)

Reactive compact makes its own model API call to summarize the conversation. For a 200K-token context, this call can take 2–3+ minutes. During this time the user sees Claude "thinking" with zero progress. This is the hang.

Evidence from the error timestamps in this issue:

• 02:13:08 — first "prompt is too long" error
• 02:15:29 — second error (2 min 21 sec gap = reactive compact duration)
• 02:15:31–32 — rapid retries after compact failed to free enough

Layer 4: Ripgrep code === null not handled gracefully

File: src/utils/ripgrep.ts (lines 187–202, 269–273, 323–334)

When ripgrep is killed by a signal (timeout → SIGTERM → SIGKILL), the close event fires with code === null:

child.on('close', (code, signal) => {
  if (code === 0 || code === 1) {
    callback(null, stdout, stderr)
  } else {
    // BUG: code is null when killed by signal
    const error = new Error(`ripgrep exited with code ${code}`)  // "code null"
    callback(error, stdout, stderr)
  }
})

This generates the misleading "ripgrep exited with code null" errors visible in the issue's log. The higher-level ripGrep() function (line ~411) does convert some of these to RipgrepTimeoutError, but the raw error propagation from the lower-level helpers can still cause cascade failures.

Layer 5: Single-shot recovery guard

File: src/query.ts (line 1157)

hasAttemptedReactiveCompact: true,  // Set after first attempt — prevents retry

After one reactive compact attempt, the guard prevents further attempts within the same turn. If compact succeeded but didn't free enough tokens, the next API call still fails — but no further recovery is possible. The error surfaces via return { reason: 'prompt_too_long' } (line 1175).

The guard resets only on a new turn (lines 275, 1721), which is why "cancel and continue" works — the user's cancellation aborts the hanging compact call, and the fresh turn allows another attempt.

Circuit breaker amplification

File: src/services/compact/autoCompact.ts (line 260–264)

If ripgrep crashes cause 3 consecutive compact failures, MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES permanently disables autocompact for the session — creating an unrecoverable state.

---

Suggested Fixes (refined after code review)

Fix 1: Better ripgrep signal-kill handling

Don't blanket-return success on signal kills (would hide real timeouts in ripGrepFileCount()). Instead, fix the misleading error message and preserve signal information:

// ripgrep.ts, lines 192-202
if (code === 0 || code === 1) {
  callback(null, stdout, stderr)
} else {
  const msg = code === null && signal
    ? `ripgrep killed by signal ${signal}`
    : `ripgrep exited with code ${code ?? 'unknown'}`
  const error: ExecFileException = new Error(msg)
  error.code = code ?? undefined
  error.signal = signal ?? undefined
  callback(error, stdout, stderr)
}

Let the higher-level ripGrep() decide when partial results are acceptable (it already does for timeout cases).

Fix 2: Bounded reactive compact duration

The compact model call should have a hard timeout aligned with the existing 90s idle timeout in claude.ts:1877. If compact doesn't complete within ~90s, abort and surface the error immediately rather than blocking the user for 2+ minutes with no feedback.

Fix 3: Progressive compaction (most impactful)

Replace the boolean hasAttemptedReactiveCompact with a bounded counter. compactConversation() already has bounded PTL retries with token-gap-based truncation (lines 227, 450 in compact.ts). Allow 2–3 attempts with progressively more aggressive message-group dropping, calibrated by getPromptTooLongTokenGap() (errors.ts:104–118). Keep the anti-loop guard around stop hooks.

Fix 4: Use file-type-aware estimation for tool results

Instead of changing the global 4-chars/token default (which would increase all estimates by 33%), use roughTokenCountEstimationForFileType for tool result content in tokenCountWithEstimation(). This targets the undercount where it matters most without affecting other callers.

Fix 5: Emergency proactive compact near the limit

Even in reactive-only mode, add a last-resort proactive check: if estimated tokens > 98% of the effective context window, trigger compaction before the API call. This doesn't conflict with the reactive-only design intent (which suppresses routine compaction) — it's a safety valve for the edge case where reactive compact failed and context keeps growing.

---

Summary

| Layer | Root Cause | Impact | Fix |
|-------|-----------|--------|-----|
| 1 | Proactive compact disabled by feature flags | Context grows unchecked | Fix 5: emergency proactive check |
| 2 | Token estimation undercounts JSON/tool results | Threshold missed | Fix 4: file-type-aware estimation |
| 3 | Reactive compact model call takes 2+ min | User sees hang | Fix 2: hard timeout |
| 4 | ripgrep code===null not handled | Cascade failures | Fix 1: preserve signal info |
| 5 | Single-shot recovery guard | No retry after failure | Fix 3: progressive compaction |

The most impactful fixes are #2 (timeout — immediately resolves the hang UX) and #3 (progressive compaction — improves recovery success rate).

---

_Analysis based on source code, verified by independent Codex review against the same codebase._

Have noticed the same recently, it just gets stuck for 15 minutes "thinking" and wont move forward until I cancel and resume

has got to be caused by /buddy LMAO (just a human guess)

+1 on the response slowdown. It gives a feeling that opus is either not thinking hard enough or somehow the context is mismanaged. No data to back my claim though.

+1 responses seem extremely degraded in throughput. Based on my current session, I'm averaging ~10.4 tokens/sec, with significant slowdowns as the context grows:

• 17m 33s to 20m 23s: 2.6k tokens in 170s (~15.3 t/s)
• 20m 23s to 24m 28s: 1.0k tokens in 245s (~4.1 t/s)
• 24m 28s to 29m 26s: 1.2k tokens in 298s (~4.0 t/s)

Average: ~10.4 t/s across the full 12-minute window, but dropping to ~4 t/s in the latter half.

Also experiencing this a lot this week. Seems to me it's getting worse, today every single prompt that has to read or edit files gets stuck one or more times.
Im on windows, using terminal in powershell. v2.1.91