2.1.233: ~10x per-turn latency regression against custom ANTHROPIC_BASE_URL (vLLM) — prefix cache no longer hits

Summary

Claude Code 2.1.233 shows a ~10× per-turn latency regression compared to 2.1.232 when running against a custom ANTHROPIC_BASE_URL (self-hosted vLLM behind a thin Anthropic-API proxy). Engine metrics indicate the regression defeats vLLM's automatic prefix caching, forcing a full context re-prefill on every turn.

Environment

• Claude Code 2.1.232 vs 2.1.233 (native installer, Linux ARM64, NVIDIA DGX Spark / GB10, 2-node)
• ANTHROPIC_BASE_URL=http://localhost:8080 → minimal streaming proxy → vLLM 0.21.1rc1 (/v1/messages), serving a DeepSeek-V4-Flash derivative with --enable-prefix-caching
• All model tiers (opus/sonnet/haiku env overrides) aliased to the same served model
• Same box, same serve config, same proxy for all measurements below — the only variable is the Claude Code version

Evidence

Median assistant-turn latency (gap between user message and next assistant message in session JSONL, per session; n = turns):

| Date | CC version | Turns | Median | p90 |
|------|-----------|-------|--------|-----|
| Aug 06 (×3 sessions) | 2.1.223 | 41–144 | 5.7–10.0 s | 60–99 s |
| Aug 08 (×3 sessions) | 2.1.224/226 | 40–108 | 3.5–10.3 s | 71–76 s |
| Aug 10 | 2.1.226 | 57 | 5.4 s | 57 s |
| Aug 14 | 2.1.232 | 135 | 8.4 s | 86 s |
| Aug 16 (×2 sessions) | 2.1.233 | 69–139 | 91.4 / 99.5 s | 109–129 s |
| Aug 17 (pinned back) | 2.1.232 | 19 | 8.4 s | 51 s |

vLLM engine metrics during the 2.1.233 sessions vs after pinning back to 2.1.232:

| Metric | 2.1.233 | 2.1.232 (pinned) |
|--------|---------|------------------|
| Avg time-to-first-token | 59 s | ~11 s |
| Prefix-cache hit rate | 8.7% | 70% |
| Prompt vs generated tokens | 35.7M vs 91K (99.7% prefill) | normal mix |

Downgrading the binary (~/.local/share/claude/versions/2.1.232) with no other change restored the previous latency immediately.

Hypothesis

Something in 2.1.233's request construction varies early in the prompt across consecutive turns of the same session (system block content, ordering, or an injected element), so the vLLM prefix cache no longer matches the previous turn's prefix and the entire context is re-prefilled every turn. Against api.anthropic.com this may be masked by server-side caching semantics, but against OpenAI-compatible/vLLM backends it is a hard 10× regression.

Workaround

Pin 2.1.232 and set DISABLE_AUTOUPDATER=1.

Happy to provide proxy-side request captures diffing 2.1.232 vs 2.1.233 request bodies if useful.