[FEATURE] Token cache guard

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

The client API and Plan has currently technical no method to know, if the next request uses the token cache
or the token cache got evicted (expire or rebalancing) on the server.

Proposed Solution

The anthropics api and plan should allow a cache guard header into the request.

This header can then be used by the client to mark a request that a token cache hit is required and otherwise it should simply fail.

On the server side in the cache layer, when the header is present, but the cache entry not, instead of re-caching the request, it should simply be declined/fail.

The client uses the failure then as the trigger that the cache got prematurely evicted
and can then react to it (user prompt or hook or something else).

The user prompt could have possible actions:

• Resend without an re-cache the session, including display session token size
• Resend the session for compaction without cache breaks for the session content (no cache)
• Clear context and resend the prompt
• Cancel request

Alternative Solutions

_No response_

Priority

Medium - Would be very helpful

Feature Category

API and model interactions

Use Case Example

There is a hidden usage/coast multiplier in API and Plan usage for a user.
The user never knows if the next request makes an cache hit or not.

This feature would resolve it, because request aborted by the cache miss,
should not count to the token usage and only be rate limited.

Additional Context

This was most likely the reason why many users experience this high usage burn over the 2x happy hours.

For API usage it also makes sense, because for a session with 200k tokens that should be cached a "." prompt
can produce ~0.82€ or 0.027€ and the user cannot know it upfront and also decline the "purchase"

I had near the same issue with aws cf and billing team, where the charge for the requests blocked in the WAF,
a DDoS attacker produced bad requests on a simple website the WAF blocked it as it should,
BUT it charged for each block with a total sum of ~1000$/hour, the effect was the same and the Site went down
and the attacker "won". Only after a major/bug company got hit 1-2 years later with a WAF 404 request charge aws adjusted the billing.
https://aws.amazon.com/about-aws/whats-new/2024/11/amazon-cloudfront-charges-requests-blocked-aws-waf/

This above story is strongly related also to the token usage/billing, the customer should always know how much the service will coast upfront, at least in a sensible range. Only to look at the usage after the fact is to little and will produce anger.

When anything triggers a cache eviction often like technical or overload or re-balancing or request retry
then coast can explode faster then a customer can react to it and will file a billing support ticket or bankruptcy.

Sadly all issues happened already over the past view weeks.