Long sessions cause dramatically increased response times — is caching/context size the culprit?
Issue
After extended coding sessions, response times appear to degrade dramatically (roughly 10x slower compared to the start of the session). Wondering if others are experiencing this and what the root cause is.
Session details when noticed
- Context: ~98k / 200k tokens (49% of window)
- Cache hit rate: 100% (98k cached, ~185 new tokens per turn)
- Compactions: 0 in the current session (prior session was compacted before handoff)
- Model: claude-sonnet-4-6
Observed behavior
At the start of a session, responses and code generation feel fast. As the session accumulates context (long debugging sessions, many tool calls, etc.), response times slow significantly — to the point where simple reasoning or short code edits take noticeably longer.
Questions
- Is the large cached context (98k tokens) the primary driver of latency, even with a 100% cache hit rate?
- Does conditioning on a large context window add non-trivial generation latency even when the tokens are cached?
- Is anyone else seeing a ~10x slowdown as sessions approach 50%+ of the context window?
- Is the recommended mitigation just
/compactmore aggressively, or is there a smarter strategy?
Workaround tried
Running /compact reduces context size and appears to help. Starting a fresh session also resolves it. But it would be helpful to understand the threshold at which this becomes noticeable and whether there's a way to get ahead of it automatically.
---
Happy to provide more diagnostics if useful.
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗