Three diagnosability issues around context limits and custom base URLs

Status Open

Reported on v2.1.220

Maintainer reply ✓ Yes — bcherny

Activity 4 comments · opened Jul 31, 2026

💡 Likely answer: A maintainer (bcherny, collaborator) responded on this thread — see the highlighted reply below.

Filing these together because they compound: each one individually sends
diagnosis in the wrong direction, and in combination they cost me several months
of intermittent debugging on a setup that was, in the end, misconfigured in a way
the client already knew about and never said.

All three are observed on 2.1.220, Linux x64, Max subscription, with a local
pass-through proxy in front of the API for traffic inspection.

---

1. A 413 from an intermediary is reported as an images/attachments problem

What happens: any HTTP 413 reaching the client is surfaced as

Request too large (max 32MB). Accumulated images and attachments in the
conversation pushed the request over the limit. Run /compact, or double press
esc to go back and remove attachments.

The suggested recovery is to drop images and attachments. If the 413 came from
something other than accumulated media, that recovery does nothing and the
session wedges: /compact hits the same wall, and /clear is the only way out.

How I hit it: a proxy in front of the API had a body-size limit lower than
the request. It returned 413. The client told me I had too many images. The
session had zero images, and the actual request on the wire was about 1 MB,
roughly 30x under the stated 32 MB limit. I spent a long time looking at
attachment accumulation because that is what the error said to look at.

Suggested fix: distinguish "the server told us the body was too large" from
"we believe accumulated media caused this". If the client has not itself
determined that images are responsible, say the request was rejected as too
large and report the size it sent, rather than naming a cause it has not
verified. Including the response body, or even just its origin, would make an
intermediary's 413 immediately identifiable.

---

2. A non-default base URL silently reduces the context window, with no notice

What happens: when ANTHROPIC_BASE_URL points at a host other than
api.anthropic.com, the client stops extending the long-context window and falls
back to the smaller default. Auto-compaction then begins near 167k tokens.

Every surface continues to display the full 1M window while this is in effect.
There is no log line, no warning, no status change, and no error. The only
observable symptom is that sessions compact much earlier than they should.

Evidence, same conversation, only the proxy setting changed:

| configuration | result |
|---|---|
| proxy on | auto-compacted at 166,403 tokens, then again at 167,108 |
| proxy off | ran past 300,000 tokens with no compaction |
| proxy on, plus the first-party flag below | 311,229 tokens, no compaction |

A single-request check is even sharper. With an identical ~168k-token prompt and
the model pinned to a non-[1m] id:

without the flag: Prompt is too long, refused locally, never sent
with the flag: succeeded normally

The remedy is _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL=1. I found it because
the client names it in a user-facing message elsewhere (a Remote Control notice
saying the flag does not apply there). Nothing surfaces it at the point where it
actually matters.

Why this was so hard to pin down: it looks intermittent. Selecting the
explicit 1M model variant sidesteps the check entirely, so the same machine with
the same proxy behaves correctly or incorrectly depending only on which model is
selected. That produced months of contradictory results.

Suggested fix: say something. A single line at session start, along the lines
of "base URL is not api.anthropic.com; long context unavailable, using the
default window; set _CLAUDE_CODE_ASSUME_FIRST_PARTY_BASE_URL=1 if this endpoint
forwards to the API unmodified" would have collapsed months into minutes.

Worth noting that a different code path already warns about custom endpoints
for Remote Control. The capability to detect and report this exists; it just is
not wired to the window decision.

---

3. The local over-length block is indistinguishable from a server rejection

What happens: once the estimate crosses the blocking threshold, the client
produces an assistant turn reporting that the prompt is too long. It never
contacts the API.

In the transcript that turn carries no error details and zero token usage. It
looks exactly like an API rejection. Nothing marks it as locally generated.

This is what made issue 2 so persistent. The failing turn appeared to be the
server rejecting an oversized request, so I kept investigating request size. The
request immediately before it had carried 264,337 input tokens and returned
end_turn successfully, which made the "rejection" look inconsistent rather than
local. Correlating against proxy logs, which showed no request at all at that
moment, is what finally separated the two.

Suggested fix: mark locally generated failures as local. Any distinguishing
field would do. Including the threshold that was applied and the estimate that
tripped it would also make issue 2 self-diagnosing, since a user seeing a 180k
threshold on a session that should have 1M available would immediately know
something had reduced their window.

---

Summary

These are all diagnosability issues rather than correctness bugs. The client
behaves as designed in each case. The cost is entirely in what it does not say:

It names a cause (images) it has not verified.
It silently reduces a limit and keeps displaying the old one.
It reports a local decision in a form that looks like a remote one.

The second is the one I would prioritize. Silent capability reduction with a
stale display is difficult to detect at all, and the remedy already exists as an
environment variable that nothing points you toward.

Happy to supply more detail on any of these.

For transparency: I tracked this down and drafted this report with AI tooling.

View original on GitHub ↗

4 Comments

cversek · 29 days ago

Correction to my third point.

I said the local block carries no error details and that nothing marks it as locally generated. That was wrong, and checkable at the time — the transcript record does mark it: the assistant turn carries model <synthetic> and an api-error flag, plus an error field. Token usage is genuinely zero, which is the part I got right.

The real issue is narrower and I'd restate it this way: the diagnostics exist, but the interactive surface doesn't show them. On the same client version, a headless run printed the estimated request size, the limit that was applied, and how much of the total was system prompt and tool definitions. The interactive path printed only "Prompt is too long".

Surfacing what the headless path already prints would make the second issue self-diagnosing, which is what I was actually asking for.

kalavt · 24 days ago

Independent confirmation of #1 from a different kind of intermediary, plus one detail that makes your suggested fix cheaper than it looks.

Setup: self-hosted LiteLLM behind an nginx ingress, ANTHROPIC_BASE_URL pointed at it. Claude Code 2.1.220 and 2.1.221, macOS arm64, model claude-opus-5[1m].

Same symptom: sessions with zero images wedged on Request too large (max 32MB). I measured the gateway's real ceiling by POSTing increasing bodies to a nonexistent path:

 256 KB -> 404   (reaches the backend)
 900 KB -> 404
1100 KB -> 413   (nginx rejects)

The ingress carried no proxy-body-size annotation, so it was running on ingress-nginx's built-in 1m default. Every request over 1 MB got a 413 and was reported to me as a 32 MB attachment problem: 32x off, and attributed to a cause that did not exist in the session.

The detail worth adding: the client already persists the upstream response. The transcript record for that synthetic turn carries it verbatim.

{
  "apiErrorStatus": 413,
  "errorDetails": "request_too_large: 413 <html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body>\r\n<center><h1>413 Request Entity Too Large</h1></center>\r\n<hr><center>nginx</center>\r\n</body>\r\n</html>\r\n",
  "message": { "model": "<synthetic>", "...": "..." }
}

<center>nginx</center> names the culprit outright. So "including the response body, or even just its origin" needs no new capability: the data is captured and then discarded at the display layer. Surfacing apiErrorStatus and errorDetails in the interactive error would have ended this in one look, for both of us.

The recovery advice compounds it: the message says to run /compact, but compact resends the same history and trips the same limit, so following the advice loops. One of my sessions accumulated 107 compact boundaries against a token peak of 377k on a 1M window. Auto-compact had no reason to fire at 37% of the window, and the manual retries could not succeed either, because the constraint was bytes at an intermediary rather than tokens in the window. The advice does not merely fail to help here, it generates the retry loop.

Incidental data point for #2: with a non-first-party base URL and the explicit [1m] model id, /context reported the full 1M window and auto-compaction did not engage early. That matches your note that selecting the 1M variant sidesteps the base-URL check, so I never hit #2 at all. Which is its own evidence for how much the selected model id changes the outcome on an otherwise identical setup.

Root cause on my side was entirely my own misconfigured ingress, same as #83898. Worth noting that this is now three independent reports where the debugging time went into the client because the client named a cause it had not verified.

Disclosure: investigated and drafted with AI tooling.

bcherny collaborator · 5 days ago

Thanks for the unusually thorough report — I was able to confirm the core of it.

1 (413 misattributed to images): reproduced on 2.1.233. With a proxy in front of the API returning 413, a session with zero images and a request body well under 1MB still reports "Request too large (max 32MB). Accumulated images and attachments in the conversation pushed the request over the limit", and the intermediary's own error body is not surfaced. Recent releases improved the copy for conversations genuinely over the 32MB limit (they now report measured sizes) and 2.1.233 forwards upstream error text for some gateway configurations, but a generic intermediary 413 below 32MB still gets the unverified images/attachments explanation.

2 (custom base URL silently reduces the context window): consistent with observed product behavior — long-context is deliberately not extended for non-default endpoints, the workaround variable you found is the intended escape hatch for endpoints that forward unmodified, and there is currently no notice at session start when the smaller window takes effect. I couldn't exercise the 1M path end-to-end in my test environment, but your evidence table is convincing.

3 (local over-length block looks like a server rejection): also consistent — the locally generated "Prompt is too long" turn is emitted in the same shape as an API error message, with no field marking it as client-originated.

Filing this as a diagnosability bug with the silent window reduction (2) as the priority, matching your assessment.

🤖 Generated with Claude Code

dkrisman · 1 day ago

Independent confirmation of item 1 on Claude Code 2.1.251, with a transcript record and a no-workload reproduction.

Setup: ANTHROPIC_BASE_URL pointing at an OpenAI-compatible gateway fronted by nginx. nginx rejected a ~2 MB /v1/messages body with a stock 413; the conversation contained zero images and zero attachments. The client had the real cause in hand and still rendered the canned message. The session jsonl records it faithfully:

{
  "type": "assistant",
  "message": {
    "model": "<synthetic>",
    "content": [{
      "type": "text",
      "text": "Request too large (max 32MB). Accumulated images and attachments in the conversation pushed the request over the limit. Run /compact, or double press esc to go back and remove attachments."
    }]
  },
  "error": "invalid_request",
  "errorDetails": "request_too_large: 413 <html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body>\r\n<center><h1>413 Request Entity Too Large</h1></center>\r\n<hr><center>nginx</center>\r\n ...",
  "isApiErrorMessage": true,
  "apiErrorStatus": 413
}

Trivial reproduction, no real workload needed: an nginx with client_max_body_size 1k; proxying to any Anthropic-compatible endpoint, ANTHROPIC_BASE_URL=http://localhost:<port> claude, send any message. nginx caps bodies at 1 MB by default and long agentic sessions routinely post multi-MB bodies, so a stock nginx in the path makes this the expected failure mode, wearing the wrong explanation.

Suggested shape for the fix: surface the actual status and responder (413 Request Entity Too Large, response served by nginx) plus a hint to check the proxy's request-size limit when a custom base URL is set, and keep the images/attachments guidance only for the client's own pre-flight check against the first-party 32 MB limit when media is genuinely present