Background Task agent output retrieval unreliable (~40% failure in multi-agent session)

Summary

During a multi-agent orchestration session (5 background Task agents across two phases), 2 of 5 agents had output retrieval failures despite completing successfully. The agent internal state was always intact — resuming recovered full context — but the output delivery mechanism was unreliable.

Environment

• Claude Code version: 2.1.39
• Model: claude-opus-4-6
• Platform: macOS Darwin 25.2.0 (Apple Silicon)
• Feature flag: CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1 (though TeamCreate was not used — only parallel background Task agents)

What happened

Session structure

• Phase 1: 3 background Task agents launched in parallel (specialists for a design review)
• Phase 2: 2 background Task agents launched in parallel (adversarial reviewers)

Agent results

| Agent | Subagent Type | Output Retrieved? | Recovery |
|-------|--------------|-------------------|----------|
| a91ca33 | apple-silicon-ml | Yes (via task notification) | N/A |
| a0755e2 | general-purpose | Yes (via TaskOutput) | N/A |
| a9e166f | fastapi-reviewer | No — TaskOutput returned status: completed but content was empty (only system reminder tags) | Resumed agent, got full output |
| ac486ee | mlx-reviewer | No — TaskOutput returned raw transcript JSON (internal message objects with UUIDs, timestamps, tool traces) instead of agent summary | Resumed agent, got full output |
| aabcd40 | fastapi-reviewer | Yes (via task notification) | N/A |

Failure mode 1: Missing output file (a9e166f)

1. Agent launched with run_in_background: true
2. TaskOutput returned status: completed with effectively empty content
3. Attempted to read output file at the path provided during launch — file did not exist
4. Glob search for the file found nothing under /private/tmp/
5. Resuming the agent recovered full context and analysis

Failure mode 2: Raw transcript instead of summary (ac486ee)

1. Agent launched with run_in_background: true
2. TaskOutput returned status: completed
3. Output contained raw JSON transcript (internal message objects with parentUuid, isSidechain, agentId, slug, tool use traces, etc.) instead of the agent's actual response
4. Resuming the agent recovered full context and clean summary

Impact

• ~40% failure rate on output retrieval (2/5 agents) in a single session
• Each failure required a resume call to recover, roughly doubling the cost for those agents
• The orchestrating agent cannot distinguish "agent produced no output" from "output retrieval failed" — status: completed is returned in both cases
• Significant orchestration overhead diagnosing and recovering from failures

Expected behavior

• TaskOutput with status: completed should reliably return the agent's final response content
• Output files should persist until explicitly read or the session ends
• If output is unavailable, the status should indicate this (e.g., output_status: complete | truncated | missing)

Suggested improvements

1. Output status field: Add output_status to TaskOutput response to distinguish successful retrieval from failures
2. Output persistence: Ensure output files are not cleaned up before the orchestrating agent reads them
3. Fallback: If the output file is missing, attempt to reconstruct from agent internal state before returning empty

Reproduction notes

This occurred during a complex orchestration pipeline (design chair → implementation plan → adversarial review → revised plan) for a GitHub issue. The session involved multiple rounds of parallel agent dispatch with synthesis between rounds. The failures were not deterministic — 3 of 5 identical-pattern launches succeeded.

Workaround

Resuming the agent via resume: "<agent_id>" reliably recovers the full context. The orchestrating agent can prompt the resumed agent for its summary. This works but doubles the cost for failed retrievals.