Thinking volume ~2x between 2.1.206 and 2.1.211 at identical model/effort/prompts (measured across 4 benchmark runs)

Summary

An automated benchmark workload (byte-identical prompts, same input files, claude-opus-4-8 on every message, effort held constant per run) roughly doubled in session cost between Claude Code 2.1.206 and 2.1.211, at both xhigh and medium effort. The growth is almost entirely thinking tokens: visible output (text + tool calls) grew ~38% while total output tokens grew ~2.9×. Nothing on the user side changed — the prompt diff between the cheap and expensive runs is empty — and the CHANGELOG for 2.1.207–2.1.211 contains no entry about thinking or effort behavior that would explain it.

Environment

• macOS (Darwin 24.6.0); Claude Code versions as stamped in the transcript JSONL version field (uniform within each run — no mid-run upgrade).
• Model: claude-opus-4-8 on every assistant message of every run.
• Effort: set via settings.json effortLevel / /effort, and verified empirically per run from the transcripts (thinking-block frequency signature), because config alone proved unreliable (separate issue, filed alongside this one, about a settings.json read-modify-write race).
• Costs are API-list-equivalent, computed from the transcripts’ message.usage objects deduped by message.id (Max plan, so these are not billed dollars — but they measure real token volume).

Measurements

Same workload every time: a full automated analysis pipeline over the same musical score — one main session + 18 subagents (~19 transcripts per run), prompts byte-identical across all four runs (git diff of every prompt file between the 2026-07-09 and 2026-07-16 runs is empty).

| Date | CC version | Effort | Deduped asst. msgs | Output tokens | Output/msg | Thinking freq¹ | Est. thinking tokens/block² | API-equiv. cost |
|---|---|---|--:|--:|--:|--:|--:|--:|
| 2026-07-04 | 2.1.198 | xhigh | 291 | 337.9k | 1,161 | 77% | ~1,120 | $31.56 |
| 2026-07-09 | 2.1.206 | medium | 335 | 286.0k | 854 | 59% | ~990 | $32.18 |
| 2026-07-16 | 2.1.211 | xhigh | 377 | 832.6k | 2,209 | 91% | ~2,070 | $68.62 |
| 2026-07-16 | 2.1.211 | medium | 493 | 535.1k | 1,085 | 51% | ~1,450 | $61.88 |

¹ Share of deduped assistant messages carrying ≥ 1 thinking block.
² Thinking text is redacted in transcripts ("thinking":""), so volume is estimated: output_tokens − visible_chars ÷ 4, divided by the thinking-block count, where visible chars = text block lengths + JSON-stringified tool_use.input lengths.

Key observations:

• The inflation is thinking, not prose. Comparing the two adjacent-in-time runs (2.1.206-medium → 2.1.211-xhigh): visible text + tool_use content grew only ~38% (354k → 486k chars) while output tokens grew 2.9×; inferred thinking share of output rose from ≈ 69% (197k of 286.0k) to ≈ 86% (712k of 832.6k).
• It is not effort-specific. Medium-vs-medium across the version change: $32.18 → $61.88 (+92%), output 286.0k → 535.1k (+87%), cache reads 28.5M → 64.7M (+127%). Est. tokens per thinking block rose at both effort levels (~990 → ~1,450 at medium; ~1,120 → ~2,070 at xhigh).
• It is uniform, not localized. In the 2.1.211-xhigh run, every one of the 19 transcripts ran 2–4× its baseline output at near-flat message counts; no single stage dominates the delta. This is a per-turn property of the run, not a prompt or stage property.
• The doubling compounds. Longer thinking per turn lengthens every subsequent turn’s context, so cache reads roughly doubled too — output and cache-read growth together account for nearly the whole cost delta.
• Corroborating request-shape changes between the 2.1.206 and 2.1.211 runs: session-total uncached input collapsed 131k → 728 tokens, and 5-minute cache writes per message rose ~69% — consistent with a change in how the harness constructs requests.

What we ruled out

• Prompt/workload changes: the prompt diff between the $32 and $69 runs is empty; same input files, same orchestration.
• Model change on our side: claude-opus-4-8 stamped on every message of every run.
• Effort misconfiguration: effort was verified per run from the transcripts’ thinking signature, not just from config.
• Stalls/retries: every main-transcript gap maps 1:1 to a subagent stage’s duration; wall-clock grew by the same uniform per-turn factor.
• Documented behavior changes: the CHANGELOG entries for 2.1.207–2.1.211 (checked 2026-07-16) contain no mention of thinking, effort, or reasoning-budget changes. The 2.1.198 note “Subagents and context compaction now inherit the session’s extended thinking configuration” predates both sides of the comparison, so it does not explain the shift.

Questions / asks

1. Is this an intentional change to the effort → thinking-budget mapping in the harness (or a server-side change that happens to correlate with the 2.1.211 release)? From transcripts alone the two are indistinguishable — the CC version is the observable correlate.
2. If intentional, please surface changes of this magnitude in release notes — this one is not there. A silent ~2× token-volume change at fixed declared config is very hard to distinguish from a user-side regression; isolating this cost roughly $100 of benchmark runs and several hours of transcript forensics.
3. Related, with thanks: 2.1.212’s “session transcripts record the reasoning effort level on each assistant message” gives future forensics the effort ground truth this investigation lacked. Exposing the effective _thinking budget_ (not just the effort label) the same way would make shifts like this one directly diffable instead of statistically inferred.

Happy to share the per-run, per-transcript statistics tables (message counts, output, cache reads, thinking-block counts per subagent) on request.