[FEATURE] File-system-based agent task queue for persistent multi-agent coordination

Resolved 💬 3 comments Opened May 14, 2026 by yilixiang888-dot Closed Jun 28, 2026

Problem

Claude Code's sub-agents communicate only through the Agent tool's prompt/response cycle. When the parent session ends, sub-agents die. There's no way to:

  1. Run persistent agents that survive parent session restarts
  2. Queue tasks for agents to pick up asynchronously
  3. Coordinate multiple agents working in parallel on different machines/terminals
  4. Debug agent communication (prompt/response is ephemeral)

Proposed Solution: File-system task queue

A simple JSON-based protocol using shared directories. Agents watch for task files and write result files. No MCP, no server, no dependencies.

Directory structure

project/
├── .agent_tasks/          # Task inbox
│   └── {agent_id}_{uuid}.json
├── .agent_results/        # Result outbox
│   └── {agent_id}_{task_id}.json
└── .agent_state/          # Agent heartbeat/status
    └── {agent_id}.json

Task file format (.agent_tasks/shangyang_abc123.json)

{
  "task_id": "abc123",
  "agent_id": "shangyang",
  "task": "Write a Python script to scrape competitor prices",
  "status": "pending",
  "priority": "normal",
  "created_at": "2026-05-14T21:00:00",
  "created_by": "guiguzi",
  "depends_on": ["task_xyz789"],
  "context": {
    "relevant_files": ["prices.csv"],
    "constraints": ["use requests library", "output JSON"]
  }
}

Result file format (.agent_results/shangyang_abc123.json)

{
  "task_id": "abc123",
  "agent_id": "shangyang",
  "agent_name": "商鞅",
  "result": "<agent output>",
  "status": "completed",
  "started_at": "2026-05-14T21:00:05",
  "completed_at": "2026-05-14T21:02:30",
  "artifacts": ["/path/to/output.py"]
}

Agent loop (pseudocode)

while True:
    for task_file in glob(".agent_tasks/{my_id}_*.json"):
        task = read_json(task_file)
        if task["status"] != "pending":
            continue
        task["status"] = "processing"
        write_json(task_file, task)
        result = execute(task)
        write_json(f".agent_results/{my_id}_{task['task_id']}.json", result)
        delete(task_file)
    sleep(3)

Benefits over current approach

| Aspect | Current (Agent tool) | Proposed (FS queue) |
|--------|---------------------|---------------------|
| Persistence | Dies with session | Survives restarts |
| Parallelism | Parent must orchestrate | Agents self-serve from queue |
| Debugging | Ephemeral prompts | Readable JSON files |
| Scheduling | Manual trigger | Cron can write task files |
| Cross-language | Claude Code only | Any language can read/write JSON |
| Dependencies | Requires MCP/Auth | Just a filesystem |

Prior art

We've been running 6 persistent agents (鬼谷子六分身) with this pattern for weeks. Tasks are routed by agent_id, results are collected by a coordinator, and the whole system survives terminal restarts because state is on disk.

Cron writes task files to trigger scheduled work (e.g., cron-observe.sh writes a weather check task every hour). Agents pick up tasks whenever they're idle.

Suggested implementation

Add a --task-queue flag to Claude Code that:

  1. Watches .agent_tasks/ for files matching the agent's ID
  2. Processes pending tasks
  3. Writes results to .agent_results/
  4. Supports depends_on for task ordering

This could also integrate with the existing Agent tool — when a sub-agent is spawned, its task definition could optionally be persisted to the queue for async execution.

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗