skill-creator eval/optimizer leaks MCP child processes via headless 'claude -p' — exhausted memory, forced hard reboot

Summary

The skill-creator plugin's eval / description-optimizer harness measures skill triggering by shelling out to one headless claude -p process per test query. In a project with MCP servers configured, every headless instance also boots that project's MCP servers (in my case the Firebase MCP node server), and those child processes are never reaped.

Running the description-optimizer (scripts/run_loop.py: queries × ~3 reps × up to 5 iterations) — together with an ad-hoc probe script the assistant wrote that did the same fan-out — stacked ~59 orphaned firebase mcp node processes (170–193 MB RSS each), plus several lingering claude -p processes. System free memory dropped to ~21% with millions of pageouts; the Terminal itself became unusable even after I quit every other app, and I had to hard-reboot the machine. It also exhausted my token budget for the period.

Environment

• Claude Code 2.1.178, macOS (Darwin 25.5.0)
• skill-creator@claude-plugins-official
• Project with an MCP server configured (Firebase MCP via firebase-tools), so each headless launch starts node … firebase mcp --dir .

Two distinct bugs

1. Leaked MCP child processes from headless eval probes. Each claude -p invocation spawns the project's MCP servers, and those node children outlive the parent and are not cleaned up. The eval harness's cost model assumes a probe is cheap; in an MCP-heavy repo it is not, and the leak accumulates without bound across a multi-iteration loop. The same applies to eval-viewer/generate_review.py and any probe script that fans out claude -p.

2. The auto-optimizer appears structurally non-functional on 2.1.178. It relies on the model auto-invoking a generated slash command to score triggering, but the model invokes the real Skill(<name>) instead and ignores the generated command — so the optimizer produces no usable signal. The loop therefore burns memory and tokens for nothing.

Reproduction

1. Open Claude Code in a project that has an MCP server configured in settings.
2. Use skill-creator to create or optimize a skill and let the eval / description-optimizer loop run (scripts/run_loop.py).
3. Observe accumulating headless claude -p processes and, critically, orphaned MCP node children (ps aux | grep -E 'claude -p|firebase mcp'). Process/memory growth continues across iterations.

Suggested fixes

• Reap MCP child processes when a headless claude -p instance exits (or disable MCP server startup entirely for eval probes — they don't need project MCP servers).
• Bound the eval harness: hard cap on concurrent/total claude -p children, with cleanup on completion and on interrupt.
• Consider disabling MCP autostart in headless eval mode by default.
• Fix or gate the description-optimizer so it doesn't run a scoring loop that can't score on current Claude Code versions.

Workaround I deployed locally

A global PreToolUse/Bash hook that denies claude -p / --print fan-outs and the skill-creator loop/probe scripts, so the harness can't be launched from inside a session. Happy to share it if useful.