[BUG] Accumulated PDF/image Read tool results permanently kill conversations — unrecoverable rate limit loop

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

Conversations that accumulate multiple multimodal tool results (specifically PDF reads via the Read tool) can reach a state where a single API request exceeds the per-minute throughput token limit, making the conversation permanently unrecoverable. Every /resume, retry, or new message immediately re-triggers the rate limit because the full context — including all accumulated PDF document blocks — is resent on every API call.

The user receives the generic API Error: Rate limit reached with no indication that the conversation itself is the problem, leading to hours of futile debugging.

---

Root Cause Analysis

How PDF reads accumulate in context

When the Read tool opens a PDF, Claude Code inserts a document content block containing the entire PDF as base64 into the conversation JSONL:

{
  "type": "tool_result",
  "content": [
    { "type": "text", "text": "PDF file read: ...pdf (16.2KB)" },
    { "type": "document", "source": { "type": "base64", "media_type": "application/pdf", "data": "<~21KB base64>" } }
  ]
}

The API renders each PDF page as an image internally. Even though the raw base64 is only ~21KB per read, each page costs ~1,600–2,000+ image tokens in processing.

The compounding problem

Since Claude Code sends the entire conversation history with every API call, each PDF read permanently increases the token cost of every subsequent request. In an iterative edit-render-review workflow (edit markdown → pandoc → read PDF → review → repeat), the document blocks accumulate rapidly:

| PDF Reads in Context | Estimated Image Tokens Added | Cumulative |
|:---:|:---:|:---:|
| 1 | ~2,000 | ~2,000 |
| 2 | ~2,000 | ~4,000 |
| 3 | ~2,000 | ~6,000 |
| 4 | ~2,000 | ~8,000 |

Combined with the text context (system prompts, CLAUDE.md, conversation history, tool results), the total input tokens eventually exceed the per-minute throughput limit in a single request.

The "point of no return"

Once a single request's input token count exceeds the per-minute throughput budget:

User sends message
  → Claude Code constructs full context (all PDF reads + all text)
  → API call exceeds tokens-per-minute limit
  → "API Error: Rate limit reached"
  → User waits, retries
  → Same full context resent → same error
  → /resume → same full context resent → same error
  → Conversation is permanently dead

There is no automatic recovery path. The conversation is stuck in an infinite rate-limit loop.

---

Confirmed Reproduction — Two Independent Conversations

This bug was independently triggered in two separate conversations during the same session, following the identical workflow pattern.

Session 1 — Original Conversation (1fac3c7e)

A letter-writing session (edit markdown → generate PDF with pandoc → read PDF to verify layout → iterate).

| # | Line | PDF File | Base64 Size | Timestamp | Result |
|:---:|:---:|---|:---:|---|---|
| 1 | 157 | Recommendation Letter - Ami Myint Than.pdf | 20,120 chars | 11:08:55Z | ✅ Success |
| 2 | 185 | same | 20,176 chars | 11:11:11Z | ✅ Success |
| 3 | 224 | same (Desktop copy) | 21,540 chars | 11:15:43Z | ✅ Success |
| — | — | User interrupted before 4th read | — | — | — |

Last successful usage block (line 260):

input_tokens: 3, cache_creation: 96, cache_read: 22,578, output_tokens: 220

The user then left and returned hours later. On resume, subsequent messages worked briefly (cache had cleared), but the conversation was unstable.

Session 2 — Debug Conversation (1cbdcd48)

Opened to diagnose Session 1's rate limit. Independently reproduced the same bug by continuing the letter-editing workflow.

| # | Line | PDF File | Base64 Size | Timestamp | Result |
|:---:|:---:|---|:---:|---|---|
| 1 | 17 | Recommendation Letter - Ami Myint Than.pdf | 22,124 chars | 22:12:57Z | ✅ Success |
| 2 | 139 | same | 21,224 chars | 22:33:22Z | ✅ Success |
| 3 | 152 | same | 21,412 chars | 22:38:18Z | ✅ Success |
| 4 | 167 | same | 23,252 chars | 22:45:53.115Z | ❌ RATE LIMITED (367ms later) |

Last successful usage block (line 164, immediately before crash):

input_tokens: 1, cache_creation: 265, cache_read: 66,033, output_tokens: 74

Note the 3× increase in cache_read tokens between sessions: 22,578 → 66,033. Session 2 was carrying additional context from the rate-limit debugging work.

The crash — timeline (Session 2)

22:45:42.000Z  Edit tool — added new paragraph to markdown
22:45:49.000Z  Bash tool — pandoc regenerates PDF
22:45:53.058Z  Last successful API response (usage: cache_read 66,033)
22:45:53.115Z  Read tool returns PDF document block (23,252 chars base64)
22:45:53.480Z  ❌ "API Error: Rate limit reached" (367ms after PDF read)
22:46:15.000Z  User types "test"
22:46:16.369Z  ❌ "API Error: Rate limit reached" — conversation permanently stuck

---

Recovery Attempts (All Failed Until Aggressive Truncation)

| Attempt | Action | Result |
|---|---|---|
| 1 | /compact in the stuck conversation | ❌ Still rate limited |
| 2 | Remove last JSONL line (the error entry) | ❌ Still rate limited (PDF blocks still in context) |
| 3 | Remove all error/duplicate lines after line 295 | ❌ Still rate limited |
| 4 | Truncate JSONL to line 246 (before PDF accumulation zone) | ✅ Conversation recovered, but lost context/history |

The only fix was surgically removing enough JSONL lines to drop the accumulated PDF document blocks below the threshold.

---

The PDF Being Read

• File: Recommendation Letter - Ami Myint Than.pdf
• Size: ~16–17 KB (single page)
• Content: One-page recommendation letter, text-only, no images
• Source markdown: 4.5 KB
• Generated with: pandoc + XeLaTeX

This is a trivially small PDF. The bug does not require large or complex documents — it's purely a function of how many times the Read tool has been used on PDFs within a single conversation.

---

Multiple Compounding Issues

This bug report describes a failure mode that results from several independent issues interacting:

1. No context size warning or guard rail

Claude Code provides no warning when accumulated context is approaching a dangerous size. The user has no visibility into how many tokens the next API call will cost.

2. No automatic compaction on rate limit

When a rate limit error occurs, Claude Code should attempt to compact the context (dropping old multimodal results) and retry. Instead, it just fails and re-sends the same oversized payload on every subsequent attempt.

3. PDF document blocks persist indefinitely

Each Read of a PDF adds a full document block that is never dropped or deduplicated, even when reading the same file multiple times. In an iterative workflow (edit → render → review → repeat), this is the expected usage pattern.

4. Misleading error message

API Error: Rate limit reached gives no indication that:

• The conversation itself is the problem (not a temporary throttle)
• The fix is to reduce context, not wait
• Multimodal content is the primary token consumer

This overlaps with #25805 (rate limit errors don't distinguish usage vs. throughput limits).

5. No recovery path

Once stuck, the user has no in-product way to recover. /compact should help but didn't in this case. The only successful recovery required manually editing the JSONL file — something no normal user would know how to do.

---

Suggested Fixes (by priority)

1. Auto-compact on repeated rate limit: If the same conversation hits rate limit N times in a row, automatically drop old multimodal content blocks and retry
2. Deduplicate PDF reads: If the same file is Read multiple times, only keep the most recent document block in context
3. Context budget warning: Warn when accumulated multimodal content exceeds X% of the estimated token budget
4. Differentiated error messages: Distinguish between "wait and retry" throttling and "conversation too large" structural issues (see #25805)
5. Graceful degradation: Offer to drop old PDF/image reads when approaching limits, rather than hard-failing

---

Related Issues

| Issue | Relationship |
|---|---|
| #25805 | Same generic error message; doesn't distinguish throughput vs. usage limits |
| #4095 | Same underlying mechanism (context/cache token explosion) but caused by infinite loops, not PDF accumulation |
| #28537 | "Hitting much faster usage limits than before" — may include users hitting this silently |
| #16157 | "Instantly hitting usage limits" — some reports may be this bug misidentified |
| #29579 | "Rate limit at 16% usage" — same misleading error, possibly same root cause |

---

Steps to Reproduce

1. Start a new Claude Code conversation
2. Create a small markdown file and generate a PDF with pandoc
3. Ask Claude to Read the PDF (triggers document block insertion)
4. Edit the markdown, regenerate PDF, ask Claude to Read again
5. Repeat steps 3–4 approximately 3–5 times (threshold depends on total conversation size)
6. Observe: the Nth PDF read triggers API Error: Rate limit reached
7. Every subsequent message, including /resume, returns the same error
8. The conversation is permanently stuck

Variables that affect the threshold:

• Size of system prompts and CLAUDE.md
• Amount of text conversation accumulated
• Size of PDF being read (even small PDFs trigger this)
• User's rate limit tier

---

Error Messages/Logs

⏺ Read 1 file (ctrl+o to expand)
  ⎿  API Error: Rate limit reached

No additional error detail is provided. No retry. No suggestion to compact.

JSONL error entry:

{
  "type": "assistant",
  "message": {
    "content": [{ "type": "text", "text": "API Error: Rate limit reached" }],
    "model": "<synthetic>",
    "usage": { "input_tokens": 0, "output_tokens": 0 },
    "isApiError": true,
    "error": "rate_limit"
  }
}

Claude Model

Opus 4.6 (claude-opus-4-6)

Is this a regression?

I don't know

Last Working Version

_N/A — this is an architectural issue, not a version regression_

Claude Code Version

v2.1.63

Platform

Anthropic API (Claude Max plan)

Operating System

macOS 15.7.3 (Apple Silicon / arm64)

Terminal/Shell

Terminal.app / zsh

Additional Information

Both JSONL conversation files are available for developer analysis:

• Original: 1fac3c7e-ed0c-423f-93f1-fa80c16d84b9.jsonl (267 lines)
• Reproduction: 1cbdcd48-3b85-4238-b134-4fe71bcc1d58.jsonl (174 lines)

The PDF being read was a single-page, text-only recommendation letter (~16KB). The bug does not require exotic content — just a normal iterative document workflow where you verify PDF output after each edit.