[Perf/Cost] Tool catalog (~24k tokens) sent uncached on every API request — cache_control breakpoint missing

Summary

Claude Code sends the full tool catalog on every API request without a cache_control breakpoint, causing the entire tool catalog to be billed at full input token price on every turn. This is particularly severe for users with large MCP server stacks.

The problem

Claude's API supports prompt caching via cache_control: { \"type\": \"ephemeral\" } on the last message in a static block. The tool catalog — which Claude Code assembles from built-in tools + all loaded MCP server tools — is structurally identical across every turn in a session. It never changes mid-conversation. It is a perfect cache candidate.

However, Claude Code does not add a cache_control breakpoint to the tool definitions array. The result: the full catalog is re-billed as fresh input tokens on every single API call.

Cost impact

For a user with a moderate MCP setup (Synthesis, Trello, Notion, Chrome DevTools, etc.), the tool catalog can easily exceed 24,000–40,000 tokens. On Opus pricing ($15/M input tokens), that is $0.36–$0.60 per turn in pure tool-catalog tax, before any actual conversation content.

Over a 40-turn session:

• Tool catalog cost (uncached): $14–$24
• Tool catalog cost (cached, at $1.875/M cache read): $1.80–$3.00
• Savings per session: ~$12–21 from a one-line fix

Third-party tools like Pino Proxy have emerged specifically to inject this missing cache_control field as a reverse proxy — which is a MITM workaround for what should be a built-in behavior.

Root cause

In Claude Code's API request builder, the tools array is populated with tool definitions but lacks a cache_control breakpoint on the final entry. Adding it would tell Anthropic's API to cache the tool block for the standard ephemeral TTL (5 minutes, extendable by the user).

The fix is approximately:

// In the request construction, on the last tool definition:
tools[tools.length - 1].cache_control = { type: \"ephemeral\" };

Or equivalently, injecting a cache_control at the system prompt level to cover both system prompt and tools together.

Why the 5-minute default TTL also matters

Even when caching is triggered by other means, the default 5-minute TTL means that any session longer than 5 minutes re-pays the cache-write surcharge on the system prompt. For the tool catalog specifically, a 1-hour TTL would be appropriate since tools do not change mid-session. This is the same insight that motivated Pino Proxy's 1-hour TTL extension.

Who is affected most

• Users with large MCP server stacks (each MCP server adds tool definitions to the catalog)
• Users running long sessions (40+ turns — the uncached tool tax compounds linearly)
• Users on Opus pricing ($15/M input, where the delta vs. cached $1.875/M is largest)
• Teams that have built workflows around extended Claude Code sessions (agentic workloads, complex debugging, iterative coding)

Expected behavior

The tool catalog, being structurally static within a session, should be cached after the first turn. Claude Code should add cache_control breakpoints to the tools array so users benefit from Anthropic's existing caching infrastructure without needing third-party proxies.

Environment

• Platform: linux
• Claude Code version: 2.1.x
• MCP servers in use: Synthesis, Trello, Notion, Chrome DevTools, and several custom servers (~300+ tool definitions total)

References

• Anthropic prompt caching docs
• Pino Proxy — third-party workaround that injects the missing cache_control as a reverse proxy, confirming the fix works
• LinkedIn post by Alexander Suvorov documenting the 87% cost reduction achievable with this fix