Workflow resumeFromRunId re-runs every completed agent after the first parallel fan-out (40 redundant opus agents, 231M tokens)

Claude Code 2.1.274, Workflow tool, session model Claude Fable 5.1, subagents on opus / sonnet / haiku. WSL2 Ubuntu on Windows 11. 2026-09-17, 05:00–07:25 UTC (2026-09-16, 22:00–00:25 PDT).

Summary

Workflow({scriptPath, resumeFromRunId}) is documented as replaying completed agent() calls from cache: "the longest unchanged prefix of agent() calls returns cached results instantly; the first edited/new call and everything after it runs live." In a script that uses parallel() or pipeline(), that prefix is broken on every resume, because concurrent agent() calls start in a different order each run. The cache therefore misses from the first fan-out onward and every downstream agent runs again at full cost, even though the script and the arguments feeding those prompts did not change.

The same tool description also says "Completed agent() calls with unchanged (prompt, opts) return their cached results instantly" and "Same script + same args → 100% cache hit." Those two sentences describe content-keyed caching. The behaviour observed is position-keyed. For any real workflow (which is to say, any workflow that fans out), the first two sentences are false.

Result on one build run: 40 redundant agent runs, 230.8M tokens (217.8M cache-read, 12.3M cache-write, 0.66M output), 28 % of the run's raw tokens and about 29 % of its weighted cost, spent re-reading and re-verifying files that were already written and committed. The redundant agents were all on opus. The user noticed only because the agents' narration said "already written, just verifying."

Timeline

| UTC (PDT) | Event |
|---|---|
| 2026-09-17 01:00 (16 Sep 18:00) | Run launched with args.stopAt = 'G1'. Phase 1: 5 agents in one parallel(). All complete. Script returns at gate G1. |
| 01:15 (18:15) | resumeFromRunId with stopAt = 'G2'. The 5 phase-1 agents replay from cache (correct). Phases 2–4 launch: a scaffold agent, then two parallel() fan-outs of 15 + 11 doc agents. |
| 01:48 (18:48) | Session usage guard trips; run stopped with TaskStop. 25 of 33 agents done, 8 in flight (expected loss). |
| 05:02 (22:02) | resumeFromRunId again, same stopAt = 'G2', script unchanged. Expected: 25 cached, 8 live. Observed: the 15 houdini-dev doc agents replay, but the 4 completed houdini-pdg-dev doc agents run again (12 redundant starts in the 05:00 hour, including the scripts/close-out agents that followed). The fan-out order had changed. |
| 06:50 (23:50) | Script returns at G2: 53 agents "done". (Only 33 had been launched before, 8 lost: the tally already contained 12 extra runs. Not noticed.) |
| 07:00 (00:00) | One prompt edited in a late phase (the audit-fix agent, phase 6, not yet run). resumeFromRunId with stopAt = 'G3'. Expected: phases 1–4 cached, phase 5 onward live. Observed: every agent from the phase-3 PDG fan-out onward starts again — 24 redundant starts in the 07:00 hour: 11 PDG docs (third run), 14 GUI docs (second run), both scripts agents, both close-outs. Each re-read the committed docs, "verified" them, and returned. |
| 07:25 (00:25) | User asks why agents say "already written, I'm just verifying." Run stopped. |

Journal evidence (journal.jsonl, same run id throughout): 53 distinct labels, 93 started events, 0 duplicate keys — every re-run got a new cache key for an identical prompt and identical opts. Redundant started events by hour: 05 UTC 12, 06 UTC 4, 07 UTC 24.

Token cost (summed from the per-agent transcripts' usage blocks)

| | Agents | Cache read | Cache write | Output | Raw total |
|---|---|---|---|---|---|
| Needed (first run of each label) | 53 | 564.5M | 23.6M | 2.48M | 590.5M |
| Redundant (second/third run) | 40 | 217.8M | 12.3M | 0.66M | 230.8M |

Weighted at list prices (cache read 0.1×, cache write 1.25×, output 5×): about 40M input-equivalent tokens wasted against about 98M needed. The account's 5-hour window read 250M weighted at the stop; roughly 70–100M of that was the redundant work.

Why the cache missed

The cache key is derived from call position in the launch sequence, not from (prompt, opts) content. parallel(thunks) and pipeline(items, …) start their agents concurrently, so the order in which agent() calls are recorded depends on scheduling. Two runs of the same script with the same args produce different launch orders inside every fan-out; the "longest unchanged prefix" ends at the first call whose position holds a different prompt than last time. From there on nothing is cached.

An edit anywhere in the script (even to a prompt that has never executed) has the same effect on everything after the first fan-out — but note that the 05:02 resume had no script edit and still re-ran completed agents.

Why this is severe

• It defeats the feature's stated purpose. Resume exists for long runs that hit session limits or need gates. Those are exactly the runs with fan-outs. The documented pattern (return at a gate, resume with resumeFromRunId) costs more than relaunching would, because the "verify" pass on an already-written artifact is a full read of it.
• It is silent. The tool result on resume says only "completed agents return cached results". /workflows shows agents running with the same labels as before; nothing marks them as re-runs. The only signals are the journal (duplicate labels with fresh keys) and the agents' own narration.
• The session model did not catch it either. Fable 5.1 read the resume contract, took the content-keyed sentences at face value, designed the gate/resume pattern on them, and after two resumes did not compare the completion tally (53) against launched-so-far (33) plus known losses (8). The doc string invited that reading; the model should still have verified it. Both halves of the product failed the user here.

Reproduction

export const meta = { name: 'resume-cache-repro', description: 'show parallel fan-out breaking resume cache', phases: [{ title: 'A' }, { title: 'B' }] }
phase('A')
const a = await parallel([1, 2, 3, 4, 5, 6].map((i) => () =>
  agent(`Return the single word ok-${i}.`, { label: `a${i}`, model: 'haiku' })))
if (args.stopAt === 'A') return { a }
phase('B')
const b = await agent('Return the single word done.', { label: 'b', model: 'haiku' })
return { a, b }

1. Workflow({script, args: {stopAt: 'A'}}) — 6 agents run.
2. Workflow({scriptPath, resumeFromRunId, args: {stopAt: 'B'}}).
3. Expected: a1–a6 cached, only b runs. Observed (in our run, with the same shape at scale): some or all of a1–a6 run again; journal.jsonl shows their labels started twice with different key values. Repeat step 2 a few times to see the count vary with scheduling.

What would fix it

1. Key the cache by content, (prompt, opts, phase), and match by set membership rather than sequence prefix. Order-independent replay is what the doc already promises.
2. Failing that, say so in the tool description: "resume caching is prefix-based; any parallel()/pipeline() breaks the prefix; use an explicit phase-skip argument instead." Remove the "unchanged (prompt, opts)" and "100 % cache hit" sentences, which are not true for scripts with fan-outs.
3. Surface re-runs. On resume, report cached: N, live: M, of which M_dup previously completed with identical prompt. A label that completed before and is starting again should be visible in /workflows.
4. Budget stop on resume. A resume that is about to launch more live agents than the original leg had remaining should pause and ask.

Workaround we now use

An args.startAt phase number; each phase is wrapped in if (!skip(n)) and finished phases are skipped by code. We no longer pass resumeFromRunId for anything downstream of a fan-out. After any resume, we count started per label in journal.jsonl before trusting the tally.

Environment

• Claude Code 2.1.274 (CLI, WSL2 Ubuntu 22.04 on Windows 11, repo on /mnt/c)
• Session model: Claude Fable 5.1; workflow agents: opus (docs, close-outs), sonnet, haiku
• 16 CPUs → concurrency cap 14; workflow size guideline raised via /config
• Run: 53 distinct agents across 7 phases, one script file, args unchanged between the 01:15 and 05:02 resumes

Filed 2026-09-17 (PDT). Journal and per-agent transcripts available on request.