Agent isolation: worktree — parallel agents lose work due to git lock contention + auto-cleanup
Problem
When dispatching multiple agents with isolation: "worktree" in a single message, the agents' git worktrees share the parent repository's .git/ directory. Concurrent git operations (git add, git commit) across worktrees create transient lock contention on shared resources (ref updates, object packing). When an agent's git commit fails due to a lock, the agent typically exits without committed changes. The worktree auto-cleanup then removes the worktree — permanently destroying the agent's work.
Reproduction
- Send a single message with 5+
Agenttool calls, each withisolation: "worktree" - Each agent makes code changes and attempts to
git add+git commit - Observe: some agents fail with
Unable to create '.git/index.lock': File exists - Those agents exit without committing
- Worktree auto-cleanup removes the worktree and its uncommitted changes
- Work is permanently lost — requires full re-dispatch
Scale
Tested with 13 parallel agents: 5 committed successfully, 8 failed due to lock contention. The failure rate increases with agent count. At 5 agents it's intermittent; at 10+ it's near-certain that some will fail.
Root Cause
Git worktrees have independent index files, but certain operations still acquire locks on the shared .git/ directory (ref packing, loose refs, object creation). With N agents committing simultaneously on N branches, transient lock collisions are inevitable. The locks are held for milliseconds — a single retry with ~200ms backoff would succeed in nearly all cases.
Suggested Fixes (any of these would help)
Fix 1 — Retry on lock contention (best ROI):
When the agent's git commit fails with an index.lock error, retry 3-5 times with exponential backoff (200ms, 400ms, 800ms). The lock is transient — retries almost always succeed.
Fix 2 — Don't auto-clean worktrees with uncommitted changes:
Before removing a worktree on agent exit, check git -C <worktree> status --porcelain. If there are uncommitted changes, preserve the worktree and report it rather than destroying work. The user can then manually recover via git stash or commit.
Fix 3 — Add jitter to worktree creation:
Stagger worktree creation by 100-500ms random jitter when multiple agents are dispatched in the same message. This desynchronizes the agents' git operations naturally.
Fix 2 is the most important — even if contention can't be fully eliminated, preserving uncommitted work prevents data loss.
Environment
- macOS (Darwin 24.6.0)
- Claude Code CLI
- Git 2.x with standard worktree support
- Parallel agent dispatch via multiple
Agenttool calls in a single message
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗