[DOCS] Logic Flaw in Token Calculation for SDK Compaction with Server-Side Tools

Documentation Type

Unclear/confusing documentation

Documentation Location

https://platform.claude.com/docs/en/build-with-claude/context-editing#server-side-tools

Section/Topic

The "Limitations" subsection under "Client-side compaction (SDK)" regarding Server-side tools.

Current Documentation

"When using server-side tools, the SDK may incorrectly calculate token usage, causing compaction to trigger at the wrong time. For example, after a web search operation, the API response might show: ... The SDK calculates total usage as 63,000 + 270,000 = 333,000 tokens. However, the cache_read_input_tokens value includes accumulated reads from multiple internal API calls made by the server-side tool, not your actual conversation context. Your real context length might only be the 63,000 input_tokens, but the SDK sees 333k and triggers compaction prematurely."

What's Wrong or Missing?

The documentation outlines a significant logic flaw in how the SDK calculates token usage for compaction. By summing internal API call tokens (specifically accumulated cache_read_input_tokens from server-side tools) into the context count, the SDK triggers compaction based on inflated numbers rather than the actual conversation context.

This makes the compaction feature unreliable for workflows involving Web Search or Web Fetch, effectively rendering the feature unusable for these common agentic patterns. The current workaround suggested in the docs ("Avoid compaction when using server-side tools extensively") forces users to disable a critical memory management feature rather than addressing the calculation logic.

Suggested Improvement

Short Term (Documentation Fix):
This limitation should be moved from the bottom "Limitations" section to a prominent warning box at the very beginning of the "Client-side compaction (SDK)" section. Users need to know immediately that this feature is incompatible with Web Search/Fetch before implementing it.

Long Term (Logic Fix):
The SDK logic should be updated to exclude server_tool_use tokens (or internal accumulated cache reads) from the context window calculation used for the compaction trigger. The documentation should then be updated to reflect that the SDK correctly calculates context even when server-side tools are used.

Impact

High - Prevents users from using a feature

Additional Context

• This issue specifically affects users leveraging the new web_search_20250305 and web_fetch_20250910 tools.
• The current logic flaw results in premature context compaction, which can degrade model performance by summarizing context that didn't actually need to be summarized yet.