[FEATURE] Restore Opus 4.6 (200k) as a model picker option alongside Opus 4.6 (1M)
Problem
In v2.1.76, the standalone Opus 4.6 (200k context) option was removed from the /model picker. Previously (v2.1.75), users had the choice between Opus 4.6 with 200k and Opus 4.6 with 1M context. Now the only Opus option defaults to 1M.
Users who prioritize accuracy and reliability over context capacity have lost the ability to make that tradeoff through the UI.
Empirical evidence: retrieval accuracy collapses beyond 200k
We conducted a controlled needle-in-a-haystack test using Opus 4.6 with 1M context on Claude Code v2.1.76. The test methodology:
- Generated realistic corporate contract documents at 4 different sizes (200k, 500k, 900k, 1.26M tokens) using
tiktoken cl100k_basefor accurate token counting - Planted 25 identical needle facts (specific dollar amounts, dates, IDs, section numbers) evenly spaced throughout each document, embedded naturally in legal contract language
- Asked Claude to read the entire document using the Read tool (no grep/search), then answer 25 retrieval questions from memory — a blind recall test
- Scored answers against a verified answer key (all needles confirmed present via grep)
Results
| Document Size | Context Window | Score | Accuracy |
|:---:|:---:|:---:|:---:|
| 200k tokens | 1M | 24/25 | 96% |
| 500k tokens | 1M | 1/25 | 4% |
| 900k tokens | 1M | 2/25 | 8% |
| 1.26M tokens | 1M | 1/25 | 4% |
At 200k tokens, the model is nearly perfect. Beyond 200k, accuracy drops to single digits.
The failures are not "I'm not sure" responses — the model confidently states that the entities "do not appear anywhere in the document" when they verifiably do (confirmed via grep at specific line numbers). This is a hallucination, not a retrieval miss.
Why the cliff exists in Claude Code specifically
The issue is compounded by how Claude Code reads files. The Read tool returns ~2000 lines per call. For large documents, Claude reads sequentially in chunks. Each chunk enters the context window, but as more chunks are read, earlier chunks get displaced. By the time reading finishes, only the most recent chunks remain in active context.
Evidence: across all tests, the model consistently recalled only needles near the end of the document (the most recent reads still in context) while claiming earlier content didn't exist.
The compaction problem
This has critical implications for the auto-compaction threshold:
| Context Window | Compaction Threshold (~83%) | Effective Operating Range |
|:---:|:---:|:---:|
| 200k (old default) | ~170k tokens | Reliable zone (96% accuracy) |
| 1M (new default) | ~830k tokens | Degraded zone (4-8% accuracy) |
With the old 200k window, auto-compaction kicked in at ~150-170k tokens — well within the reliable accuracy range. The model never drifted past the point where it starts hallucinating.
With the new 1M window, the model can accumulate 830k tokens before compaction triggers — a zone where our tests show it confidently fabricates answers. For users who run long unattended sessions (leave Claude Code coding and come back later), this means the model spends most of its time operating in the degraded zone without the user knowing.
Why this matters
Anthropic's own documentation acknowledges this problem:
- The context engineering guide states: "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases" — describing this as "context rot."
- The context windows documentation describes this as "a performance gradient rather than a hard cliff" — meaning accuracy silently erodes rather than failing obviously.
- The same documentation recommends finding "the smallest set of high-signal tokens that maximize the likelihood of some desired outcome" — which is exactly what a 200k constraint naturally enforces.
The workaround exists but is not discoverable
The CLAUDE_CODE_DISABLE_1M_CONTEXT=1 environment variable exists, proving Anthropic anticipated this need. However:
- It's not discoverable through the UI
- Most users won't know it exists
- It's an all-or-nothing switch rather than a choice per session
Proposed solution
Option A (minimal): Restore the Opus 4.6 (200k) option in the /model picker alongside the 1M variant, as it was in v2.1.75:
❯ 1. Default (recommended) ✔ Opus 4.6 with 1M context · Most capable for complex work
2. Sonnet Sonnet 4.6 · Best for everyday tasks
3. Sonnet (1M context) Sonnet 4.6 with 1M context
4. Haiku Haiku 4.5 · Fastest for quick answers
5. Opus Opus 4.6 · Most capable, optimized accuracy ← RESTORE THIS
Option B (better): Scale the auto-compaction threshold relative to the context window size, or allow users to configure a token-count-based compaction trigger (e.g., "compact at 180k tokens regardless of window size") rather than only a percentage-based threshold.
Option C (best): Both — give users the 200k option in the picker AND allow configurable compaction thresholds. Different workflows have different accuracy requirements.
Reproduction
The test scripts and documents are available for reproduction:
- Generated using Python with
tiktoken(cl100k_base encoding) for accurate token counting - Needle facts are specific, unguessable values (dollar amounts like $73,412, IDs like TXC-9917-B, dates like March 19, 2031)
- All needles verified present in documents via grep before testing
- Each test run in a fresh Claude Code session with no prior context
References
- Anthropic: Effective context engineering for AI agents
- Context windows documentation
- Research: "When More Becomes Less: Why LLMs Hallucinate in Long Contexts"
- Hallucinating Law: Legal Mistakes with LLMs are Pervasive — Stanford
- Related: #25679 (configurable compaction threshold), #15719, #23711
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗