System scaffold tokens (tools/prompt) should use global cache and not be billed as user input

Problem

Every Claude Code API call includes ~21K tokens of identical, immutable system scaffolding — tool definitions (Read, Write, Edit, Bash, Grep, Glob, Agent, WebFetch, etc.) and the system prompt. These tokens:

• Are the same for every user, every session, every call
• Cannot be modified or opted out of by users
• Are required for Claude Code to function at all
• Are billed as input tokens to the user

At scale, this means millions of API calls per day are transmitting and billing for the exact same ~21K tokens. This is analogous to a phone manufacturer charging customers for the storage consumed by the OS itself.

Current State

Anthropic's API supports prompt caching, which reduces repeated token costs to 0.1x after the first call in a session. Claude Code likely already uses this per-conversation. However:

1. Per-conversation caching still charges users (at 0.1x) for tokens they have no control over
2. The cache is per-session — a new conversation means a fresh cache write
3. There is no global/shared cache across users, even though the content is identical

Proposal

Since the system tool definitions and base system prompt are identical across all Claude Code users globally, Anthropic should:

Option A: Global Shared Cache (Recommended)

Implement a global KV cache for system scaffolding tokens shared across all Claude Code sessions. Benefits:

• Eliminates redundant network transmission of ~21K tokens on every call
• Eliminates redundant compute for processing identical tokens
• Reduces inference cost at massive scale (millions of calls/day × 21K tokens)
• Users only pay for their content (messages, tool results, MCP tools, memory, etc.)

Option B: Exclude System Scaffolding from Billing

If global caching is not architecturally feasible, simply exclude the fixed system scaffold tokens from the billed input token count. Users should only be billed for tokens they control.

Option C: Pre-bake into Model Serving

For the Claude Code product specifically, pre-load the tool definitions at the model serving layer so they never need to be transmitted or processed per-request.

Impact

• User cost savings: ~21K input tokens per API call, across every Claude Code user
• Infrastructure savings: Massive reduction in redundant data transmission and KV cache computation
• User trust: Users shouldn't feel they're paying for the "operating system" overhead of the tool they're already subscribing to

Context

This was identified while optimizing MCP server token usage. After reducing MCP tools from ~30K to ~2.9K tokens, the remaining ~21K of immutable system scaffolding became the dominant and only non-optimizable cost in the context window.

---

Submitted via Claude Code on behalf of a user who rightly pointed out: "If there is a structured, unchanging context of tokens that can't change, then anytime a Claude Code call is made it should be cached and not charged. Think about the millions of times you do that in a day."