TaskOutput burst polling causes unrecoverable 'Prompt is too long' when multiple background agents complete simultaneously

Bug Description

When multiple background Task agents complete and the assistant polls their results via TaskOutput in a single turn (parallel tool calls), the combined results push the context past the token limit. This produces an unrecoverable "Prompt is too long" error where even /compact fails.

Environment

• Claude Code version: 2.1.34 (also reproduced on 2.1.33)
• OS: macOS (Darwin 25.2.0)
• Model: claude-opus-4-6
• IDE: VSCode extension

Reproduction Steps

1. Start a long conversation that accumulates ~100K+ tokens of context (after multiple compactions)
2. Launch 4+ background Task agents via run_in_background: true
3. Wait for all agents to complete
4. The assistant calls TaskOutput for all agents in a single turn (parallel tool calls)
5. Each TaskOutput returns ~30,156 chars of truncated transcript
6. Combined results push context past the ~200K token limit
7. API returns "Prompt is too long" with input_tokens: 0, output_tokens: 0
8. /compact also fails with: Error: Conversation too long. Press esc twice to go up a few messages and try again.
9. Session is permanently dead — no further messages can be processed

Observed Behavior (3 incidents in 2 days)

Incident 1 — Session dce4fc55 (Feb 8)

• Context was at ~100K tokens
• 11 background agents launched to process 11 companies through a pipeline
• All 11 completed successfully
• User asked "What is the status of all the agents?"
• Assistant issued 11 sequential TaskOutput calls → 11 × ~30,156 chars = ~331K chars (~83K tokens) added in one turn
• Context exceeded limit → "Prompt is too long"
• /compact failed with "Conversation too long"
• Session died (12-day session lost)

Incident 2 — Session cf66f225 (Feb 9)

• Context was at ~152K tokens
• 8 background agents launched to write solution files
• Assistant polled 4 TaskOutput results, then 4 more
• 8 × ~30,156 chars = ~241K chars (~60K tokens) added
• Context exceeded limit → "Prompt is too long"
• Session died (22-hour session lost)

Token Usage Data from Logs

| Metric | Session 1 (dce4fc55) | Session 2 (cf66f225) |
|--------|----------------------|----------------------|
| File size | 35 MB | 31 MB |
| Total lines | 3,782 | 3,071 |
| Task (subagent) calls | 34 | 75 |
| TaskOutput polls | 31 | 45 |
| Compaction events | 17 | 16 |
| Pre-compaction avg tokens | 122,504 | 125,878 |
| Pre-compaction max tokens | 160,090 | 173,234 |
| Context at crash | ~183K (estimated) | ~182K (estimated) |
| TaskOutput results at 30K cap | 28/31 (90%) | 40/45 (89%) |

Root Cause Analysis

Primary: No context budget check before TaskOutput calls

The assistant issues multiple parallel TaskOutput calls without checking whether the combined results will fit in the remaining context window. Each result is ~30,156 chars (~7,500 tokens). With 4-11 agents, this adds 30-83K tokens in a single turn with no opportunity for compaction to intervene.

Secondary: Compaction fails when needed most

When context exceeds the API limit, /compact itself makes an API call that also exceeds the limit. This creates an unrecoverable death spiral:

1. Context too long → can't send messages
2. Try to compact → compaction API call also too long → fails
3. No way to recover the session

Contributing: TaskOutput truncation is still too large

The ~30,156 char truncation limit per TaskOutput result is generous. For status checks, a brief summary would suffice. The full truncated transcript is rarely needed in the main conversation context.

Expected Behavior

1. Context-aware TaskOutput polling: Before issuing TaskOutput calls, check remaining context budget. If budget is tight, poll agents one at a time with compaction between polls.
2. Graceful compaction: /compact should work even when context exceeds the API limit — e.g., by truncating from the middle before sending the compaction request, or by performing local summarization.
3. Shorter TaskOutput summaries: Return a brief status + summary by default rather than the full truncated transcript. Offer a "verbose" option for full output.
4. Proactive compaction before burst operations: Auto-compaction should trigger before issuing bulk TaskOutput calls when context is already high (e.g., >130K tokens).

Workaround

• Poll background agents one at a time instead of in parallel
• Start new sessions before context gets too large
• Avoid launching more than 3-4 background agents in sessions with significant existing context

Impact

• Lost a 12-day session at a critical moment (all work was done, just couldn't report status)
• Lost a 22-hour session right before final validation step
• Both times, the background agents had completed their work successfully — only the main conversation thread died
• Users with long-running complex workflows are most affected