Feature Request: Multi-Tiered Prompt Caching for Cost Optimization

Feature Request: Multi-Tiered Prompt Caching

Summary

Claude Code currently uses only 5-minute prompt caching (ephemeral_5m) for all cached content, despite the Claude API supporting 1-hour caching (ephemeral_1h) at the same price. Implementing multi-tiered caching would reduce costs by 8-12% while maintaining quality.

Current Behavior

All cached content uses a single 5-minute TTL:

• System prompt & tool definitions (~18,000 tokens)
• User's CLAUDE.md (~4,500 tokens)
• Auto-loaded skills (~2,500 tokens)
• Session context (~2,000 tokens)

Current cost per session: ~$0.114 for cache write + ~$0.043 per cache read

Problem: Starting 10 sessions in an hour re-caches the same system prompt and CLAUDE.md 10 times, even though this content rarely changes.

Proposed Solution

Implement multi-tiered caching with different TTLs based on content volatility:

┌─────────────────────────────────────────┐
│ Tier 1: Static (1-hour cache)          │
│ - System prompt & tool definitions      │
│ - Invalidate on version update          │
├─────────────────────────────────────────┤
│ Tier 2: User Config (1-hour cache)     │
│ - CLAUDE.md                             │
│ - Invalidate on file mtime change       │
├─────────────────────────────────────────┤
│ Tier 3: Session (5-minute cache)       │
│ - Skills, hooks, environment            │
│ - Invalidate per session                │
└─────────────────────────────────────────┘

API Support

The Claude API already supports this via multiple cache breakpoints:

# Pseudocode
messages = [
    {
        "role": "user",
        "content": [
            {"type": "text", "text": system_prompt, "cache_control": {"type": "ephemeral", "ttl": "1h"}},
            {"type": "text", "text": claude_md, "cache_control": {"type": "ephemeral", "ttl": "1h"}},
            {"type": "text", "text": skills, "cache_control": {"type": "ephemeral", "ttl": "5m"}},
            {"type": "text", "text": user_message}
        ]
    }
]

Configuration Proposal

Add optional configuration in ~/.claude/settings.json:

{
  "cache": {
    "system_prompt_ttl": "1h",
    "user_config_ttl": "1h",
    "skills_ttl": "5m",
    "invalidate_on_file_change": true
  }
}

Defaults: Keep current behavior (all 5m) for backward compatibility. Users opt-in to 1-hour caching.

Benefits

Cost Savings (assuming 10 sessions/hour):

• Current: 10 cache writes × $0.114 = $1.14/hour
• With 1h cache: 1 write + 9 reads = $0.114 + (9 × $0.043) = $0.50/hour
• Savings: $0.64/hour or 56% on cache operations

Per 100 sessions: ~$9-12 in savings

Quality Impact: None - content freshness is maintained via file change detection

Implementation Notes

1. File change detection: Hash or mtime check for CLAUDE.md before each session
2. Version detection: Include Claude Code version in cache key for Tier 1
3. Backward compatibility: Default to current behavior if config not present
4. Cache key structure: claude-code:{version}:{tier}:{content_hash}

Alternative: Simple Flag

Minimal implementation - just add one flag:

{
  "cache": {
    "use_1h_for_static_content": true  // system + CLAUDE.md
  }
}

This alone would capture most of the savings with minimal code change.

Related

• Claude API docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
• Current cache usage visible in session JSONL files: cache_creation.ephemeral_5m_input_tokens vs ephemeral_1h_input_tokens (always 0)

Priority

Medium - Provides measurable cost savings for power users without affecting functionality. Particularly valuable for users running dozens of sessions daily.