Feature Request: Task Tool Execution Metadata for Subagent Monitoring

Summary

The Task tool returns only the subagent's output text with no execution metadata. This prevents orchestrators from monitoring subagent health, tracking costs, or detecting freezes.

Request: Return structured metadata alongside subagent output.

---

Problem: Frozen Subagents Go Undetected

User observation:

"Subagents sometimes freeze, but the orchestrator thinks they're still running. I can only detect this by monitoring my Anthropic API dashboard - if usage isn't increasing, the subagent is frozen."

Current limitations:

result = Task(
    subagent_type="Explore",
    model="opus",
    prompt="Research codebase"
)
# result = string output only
# No metadata: tokens used, time elapsed, success status

Impact:

• Can't track per-subagent token costs
• Can't detect frozen/stuck subagents
• Can't warn if approaching handover threshold (160K)
• No visibility into execution for debugging
• Wasted tokens when subagents freeze silently

---

Proposed Solution

Option 1: Return Object with Metadata

result = Task(...)

result.output            # The subagent's response text
result.metadata = {
    "tokens_used": 15234,               # Input + output tokens
    "context_window": "15234/200000",   # Current context state
    "time_elapsed_ms": 12400,           # Execution time
    "model": "claude-3-opus-20240229",  # Model used
    "success": True,                    # Completed successfully
    "error": None,                      # Error if failed
    "handover_recommended": False       # True if >160K (80% of 200K)
}

Minimal Implementation (Most Important)

These 4 fields would be transformational:

{
    "tokens_used": int,        # CRITICAL for cost tracking
    "time_elapsed_ms": int,    # For freeze detection
    "success": bool,           # Did it complete normally?
    "error": Optional[str]     # Error message if failed
}

---

Use Cases

1. Frozen Subagent Detection

if result.metadata['time_elapsed_ms'] > 1800000:  # 30 min
    if result.metadata['tokens_used'] == 0:
        # Frozen - no tokens consumed but still "running"
        print("⚠️ Subagent appears frozen")

2. Token Budget Tracking

# Track costs across multiple subagents
stage_1 = Task(subagent_type="Explore", model="opus", ...)
stage_3 = Task(subagent_type="General", model="sonnet", ...)

total = (stage_1.metadata['tokens_used'] +
         stage_3.metadata['tokens_used'])
print(f"This issue consumed {total:,} tokens")

3. Handover Prevention

if result.metadata['context_window'].split('/')[0] > 160000:
    print("⚠️ Subagent near handover threshold")
    # Create handover before launching next subagent

4. Performance Optimization

After analyzing 20 issues:

• Stage 1 research: Avg 15K tokens (too thorough? scope too broad?)
• Stage 3 implementation: Avg 22K tokens (close to estimate)
• Identify expensive patterns → optimize prompts

---

Current Workaround (Brittle)

We're instructing subagents to self-report in their output:

## Research Report
[findings...]

---
## Execution Report
- Token usage: 15,234 / 200,000 (7.6%)
- Time elapsed: 12 minutes

Problems:

• Relies on subagent compliance (can forget)
• Must parse text output (error-prone)
• Subagent can only estimate (no actual metadata)
• Adds tokens to every response

Native metadata would be cleaner and more reliable.

---

Why Both Are Valuable

Metadata from Anthropic (precise, always available):

result.metadata['tokens_used']  # Exact count

Self-reporting by subagent (contextual, informative):

Checkpoint: Analyzed 28/42 files, tokens: 7,891
→ Progress context that metadata alone can't provide

Ideal: Both together

• Metadata for precise metrics
• Self-reporting for progress context during execution

---

Impact

High Priority for Orchestration Use Cases:

• Building autonomous agent systems
• Multi-stage workflows with subagents
• Cost tracking and optimization
• Health monitoring and freeze detection

Example: SST2 (Self-Sustaining Testing & Tooling)

• 7-stage workflow per issue
• 3-7 subagents per issue
• Need cost visibility and health monitoring
• Currently experiencing frozen subagents with no detection

---

Backward Compatibility

Option A: New parameter

# Old way (backward compatible):
result = Task(...)  # Returns str

# New way (opt-in):
result = Task(..., return_metadata=True)  # Returns object

Option B: Str subclass

class TaskResult(str):
    """Backward compatible - works as string"""
    def __new__(cls, output, metadata):
        instance = super().__new__(cls, output)
        instance.metadata = metadata
        return instance

---

Related Enhancements

If metadata is added, consider:

• Task progress streaming: Updates while subagent runs
• Task cancellation: Kill frozen/stuck subagents
• Task quotas: Limit subagent token budget per launch

But metadata alone would be highly valuable!

---

Reference: https://github.com/hoiung/dotfiles/issues/120 (detailed spec)
Project: SST2 - Autonomous orchestrator managing 3-7 subagents per issue
Contact: hoiung (GitHub)