Proposal: Token budget regression testing in CI pipeline

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

As Claude Code's feature surface grows (Agent Teams, Skills, Tool Search, etc.), the system prompt and tool schema payload has grown with it. Since January 2026, the system prompt has grown by ~12K tokens across releases — primarily from Agent Teams (v2.1.16) and subsequent additions. There is currently no automated way to detect when a release significantly increases per-session or per-agent token overhead.

This matters because subagents inherit the full system context with no opt-out mechanism, so prompt growth has a multiplier effect on real-world token consumption. A 7K addition to the system prompt becomes 7K × every agent spawned in a session. A community member built https://cchistory.mariozechner.at/ to track system prompt changes externally because there's no internal visibility into this.

This complements #7336 (lazy-loading) by addressing the prevention side rather than the fix. Lazy-loading reduces existing overhead; this ensures new overhead doesn't accumulate untracked.

Proposed Solution

A CI job on each release that runs a standardized token measurement suite:

1. Initialize a session with a reference config (defined set of MCP tools, skills, memory files)
2. Record baseline input tokens before any user message
3. Send a trivial message, record per-turn token overhead
4. Spawn one subagent with a no-op task, record inherited context size
5. Diff all metrics against the previous release
6. Flag for review if any metric crosses a threshold (e.g. >5% increase)

No need to block releases — even a dashboard or report artifact would be a big step forward. The goal is visibility, not gatekeeping. Features that are expensive in token overhead get that cost surfaced early, when it's cheapest to optimize (lazy-load, conditional inclusion, etc.) rather than after users report it.

The measurements are straightforward and the reference configs could live alongside existing test fixtures.

Alternative Solutions

_No response_

Priority

Medium - Would be very helpful

Feature Category

Performance and speed

Use Case Example

_No response_

Additional Context

_No response_