[BUG] totalTokensReminder causes repeatable prompt-cache floor in tool loops; off restores incremental hits

Status Open
Reported on v2.1.246
Maintainer reply None cached
Activity 8 comments · opened Aug 27, 2026

Preflight Checklist

  • [x] I searched existing issues. #88211 covers the meaning of the padded countdown, but not prompt-cache invalidation.
  • [x] This is a single bug report.
  • [x] I reproduced on the latest Claude Code version available to me.

What's Wrong?

With the default totalTokensReminder behavior (padded-countdown), a long tool-use loop repeatedly fell back to the same prompt-cache floor even though the conversation was growing append-only in the local transcript.

In one continuous Claude Desktop session, ten consecutive requests reported cache_read=8192 while uncached input grew from 60,327 to 77,252 tokens:

| Request | input | cache_read |
|---:|---:|---:|
| 1 | 60,327 | 8,192 |
| 2 | 63,476 | 8,192 |
| 3 | 66,656 | 8,192 |
| 4 | 68,497 | 8,192 |
| 5 | 70,562 | 8,192 |
| 6 | 74,477 | 8,192 |
| 7 | 75,292 | 8,192 |
| 8 | 76,161 | 8,192 |
| 9 | 76,735 | 8,192 |
| 10 | 77,252 | 8,192 |

The local transcript contained a changing total_tokens_reminder after tool results during this window, for example:

<total_tokens>14931235 tokens left</total_tokens>
<total_tokens>14926828 tokens left</total_tokens>
<total_tokens>14923572 tokens left</total_tokens>

I then changed exactly one user setting:

{
  "totalTokensReminder": "off"
}

The setting hot-reloaded into the same real Claude Code session. I did not change the Claude Code version, Desktop version, gateway, selected model, tools, or conversation. After the change:

  • total_tokens_reminder attachments in the observed window: 10 before, 0 after.
  • The first request was expectedly cold because removing the reminder changed the system prompt: input=89,880, cache_read=1,024.
  • The next request immediately recovered to input=3,438, cache_read=90,112.
  • Cache reads then grew incrementally to 120,832 while uncached input stayed mostly in the hundreds or low thousands.

Selected post-change requests:

| Request | input | cache_read |
|---:|---:|---:|
| cold after setting change | 89,880 | 1,024 |
| next | 3,438 | 90,112 |
| next | 5,358 | 93,184 |
| next | 613 | 98,304 |
| next | 1,445 | 100,352 |
| next | 1,434 | 103,424 |
| next | 870 | 107,520 |
| next | 329 | 112,640 |
| next | 3,379 | 115,712 |
| next | 2,477 | 118,784 |
| final observed | 1,547 | 120,832 |

hook_success events continued to occur after the setting was disabled, while the cache prefix kept growing. No task_reminder occurred in the post-change window.

This same-session A/B result suggests that the dynamic totalTokensReminder path may be changing the effective cacheable prefix during tool loops. One possible client-side mechanism is the message normalization / reminder-to-tool_result folding family described in #49585 and the historical message drift in #48734.

I do not have the final raw request bodies because my tracing layer intentionally logs hashes and usage but omits message content, so I cannot yet name the exact differing byte or content block.

Important Provider Scope

This was observed through an Anthropic Messages-compatible LLM gateway. The selected Claude Code model alias was claude-fable-5, but the gateway routed to a non-Anthropic upstream model. The gateway, upstream model, and routing stayed unchanged throughout the same-session A/B test.

I have not reproduced this against api.anthropic.com, and I understand that non-Anthropic upstream model behavior may be outside Anthropic's support scope.

I am reporting this because the only changed variable was the Claude Code client setting, and the result matches previously reported client-side reminder re-serialization/cache-floor signatures. It would be useful for the maintainers to verify the outgoing request-body invariant using a first-party endpoint.

What Should Happen?

Changing reminder values should not mutate or re-serialize an already cached historical message prefix. A reminder can be appended at the current tail, or historical serialized messages should remain byte-stable between adjacent tool-loop requests.

With the default setting, consecutive requests should continue reading the previous request's stable message prefix instead of repeatedly falling back to the system/tools cache floor.

Steps to Reproduce

  1. Use default settings with totalTokensReminder unset (padded-countdown).
  2. Start a long-context session and run a task that produces many sequential tool calls.
  3. Confirm that the transcript receives changing attachment.type = "total_tokens_reminder" entries after tool results.
  4. Record per-request input_tokens and cache_read_input_tokens.
  5. If cache reads repeatedly collapse to a fixed system/tools floor, set "totalTokensReminder": "off".
  6. Continue the same session with the same model and tools.
  7. Confirm that new total_tokens_reminder attachments stop and compare whether cache reads resume incremental growth.

For a definitive first-party reproduction, capture two adjacent outgoing /v1/messages bodies and compare the ordered messages content blocks and cache_control placement before and after each tool result.

Environment

  • Claude Code: 2.1.246
  • Claude Desktop: 1.37937.3
  • Entry point: Claude Desktop third-party inference / gateway configuration
  • Operating system: macOS
  • Shell: zsh
  • Context mode: 1M variant enabled
  • Prompt cache controls observed: 3 on the main streaming requests
  • Last locally observed version without total_tokens_reminder: 2.1.229
  • First locally observed version with it: 2.1.234 (this may also involve rollout/feature-gate timing, so I am not asserting a hard introduction boundary)

Related Issues

  • #88211 — exact totalTokensReminder / 15M padded-countdown behavior, but focused on misleading context headroom.
  • #49585 — dynamic system reminders folded into historical tool_result.content, breaking the prompt cache.
  • #48734 — byte drift in historical system-reminder content invalidating the prompt prefix.
  • #63930 — repeated collapse to a system/tools-only cache floor after tool-heavy turns.

I can provide a redacted per-request timeline and message-structure metadata if useful, but I cannot publish the private conversation or tool-result contents.

View original on GitHub ↗

3 Comments

Firststudy1 · 1 month ago

Same bug from a different angle, with hard version-boundary data that may help triage.

Setup: Claude Code over a local relay (cc-switch) → non-Anthropic upstream (opencode.ai, deepseek-v4-flash). Here there are no cache_control breakpoints in the loop at all — caching is 100% the upstream's automatic prefix cache, so the ONLY thing that matters is a byte-stable request prefix. Same symptom as reported: a fixed cache floor while uncached input grows every turn.

1. Version boundary. From session transcripts matched against the relay's per-request cache accounting:

  • 2.1.220…2.1.232: ~75–85% within-session hit (e.g. one 404-request session at 82%)
  • Session 452c2760 auto-upgraded 2.1.232 → 2.1.233 mid-session: hit rate collapsed at exactly that point (85% → 3.6%)
  • 2.1.241 / 2.1.247 / 2.1.250: 0.4–21%

In my environment the regression walks back to 2.1.233, not 2.1.237.

2. Same session, only the version changed (before/after downgrade):

| time | version | input | cache_read |
|---|---:|---:|---:|
| consecutive turns | 2.1.250 | 82,994 → 107,042 | 0 every time |
| 1st turn | 2.1.232 | 111,064 | 0 (cold, normal) |
| 2nd turn | 2.1.232 | 1,706 | 111,616 |
| 3rd turn | 2.1.232 | 309 | 117,248 |

3. Deterministic A/B through the same relay (proves upstream is healthy):

  • byte-identical repeat: cache_read 2,048/2,127
  • same head + growing tail: 8,704/8,803 (~99%)
  • one line injected at the HEAD of system: cache_read 0/8,884 — the entire ~8.8k prefix is invalidated
  • append block at the TAIL of system: 8,704/8,803

=> The upstream matches from the first tokens; any per-turn insertion at the head of the system prompt kills the whole prefix cache. ≤2.1.232 kept the head byte-stable across turns; ≥2.1.233 does not. Consistent with the padded-countdown totalTokensReminder floor described here, and with #89651's uncacheable reminder blocks.

Workaround that restores ~99% hits for me: pin 2.1.232. Happy to run a byte-level capture of the outgoing request head across two turns (2.1.250 vs 2.1.232) if it would help.

DanLanEX · 1 month ago

@Firststudy1

Workaround that restores ~99% hits for me: pin 2.1.232. Happy to run a byte-level capture of the outgoing request head across two turns (2.1.250 vs 2.1.232) if it would help.

If you want to stay on a current build instead of pinning 2.1.232, the workaround that recovered cache hits for me (same session, nothing else changed) is:

{
  "totalTokensReminder": "off"
}

It hot-reloaded. After one expected cold request, total_tokens_reminder attachments stopped and cache_read started growing again instead of sitting on the floor. Worth trying on 2.1.250 before pinning.

edubraqd · 1 month ago

First-party data point (api.anthropic.com, no gateway), since the report asks for one.

1. Passive: 674 local sessions, Claude Code 2.1.156 → 2.1.272, Windows. I re-read the .jsonl transcripts and took every pair of consecutive calls (same model, gap < 60 min, no compaction) with a total_tokens_reminder attachment between them, flagging a "rewrite" when cache_read collapsed to the shared system+tools prefix (≤ 60k) while cache_creation re-wrote > 20% of the context — the floor signature in your table.

  • 61,850 such boundaries, 54 rewrites = 0.09% — below the ~0.2% average for post-tool boundaries in the same sessions. No floor: cache_read keeps growing across tool loops with the reminder on.

2. Active A/B, 2.1.272, Opus 5, claude -p. 8 tasks × 2 arms, ABBA, fresh session each, only difference CLAUDE_CODE_TOTAL_TOKENS_REMINDER=off (the env var outranks settings.json in this build):

| arm | sessions | <total_tokens> blocks / session | cache_creation / session | cache_read / session | US$ / session |
|---|---:|---:|---:|---:|---:|
| default (padded-countdown) | 8 | 4–18 | 27,477 | 91,726 | 0.346 |
| off | 8 | 0 | 25,244 | 92,600 | 0.323 |

  • In the 6 pairs where both arms took the same number of calls, cache write differs by +23 to +111 tokens — the literal size of the blocks (4 blocks → +23, 8 → +79/+111), ≈ 6–14 tokens per reminder. Cost delta US$ 0.000–0.001 per session.
  • No mid-session rewrite in any of the 16 transcripts: cache_read grows monotonically after every tool result, in both arms.
  • The two large cache_creation values (38.7k, first request, cache_read 0) are a cold shared tools+system prefix under 3 parallel workers, and they happen in both arms — not the reminder. The average cost gap above is those two cold starts plus one extra call in one task.

So against the first-party endpoint the reminder is a tail append that costs its own tokens and nothing else — consistent with @Firststudy1's own A/B ("append block at the TAIL of system" keeps the hits) and with the floor showing up only where the upstream does automatic prefix matching with no cache_control breakpoints. I can't say anything about the head-of-system drift on those upstreams; with explicit breakpoints on api.anthropic.com it doesn't cost anything measurable here.

Method, per-run numbers and the runner (--arms config,nore): https://github.com/edubraqd/claude-context-forensics — report/issues-2026-09-15.md (addendum) and experiments/ab-config-vs-bare/data/results-nore-2026-09-15.jsonl.

Showing cached comments. Read the full discussion on GitHub ↗