OAuth refresh corrupts credentials state during transient upstream 5xx → persistent 401 loop across sessions

Title

OAuth refresh corrupts credentials state during transient upstream 5xx → persistent 401 loop across sessions

Body

Summary

Claude Code's OAuth refresh path appears to mishandle transient Cloudflare 5xx responses on api.anthropic.com. When the refresh fires during such a window (or a normal request retries through one), the credentials end up in a state where ~/.claude/.credentials.json retains a token that the server rejects with 401 Invalid authentication credentials. The bad token then persists across sessions — the next claude session reads it and the very first API request returns 401 before the user even finishes typing. /login is the only thing that fixes it, until the cycle repeats 4–8h later. Reproduced across two different Anthropic subscription accounts, so this is client-side, not account-side.

Environment

• Claude Code: 2.1.150
• Platform: Linux 6.8.0-107-generic (Ubuntu VPS)
• Shell: bash
• Subscription: Max plan (also reproduced on a second, different Anthropic account)
• Only one claude process at a time on this box
• No ANTHROPIC_* env vars exported in any shell rc
• No cron jobs / scheduled routines / claude --schedule running
• No project-level .mcp.json; only global plugin set (~12 plugins)
• Single working directory: /root

Reproduction (inferred chain — repeats reliably every 4–8h)

1. A long-running claude session crosses the OAuth access-token expiry (~4–8h).
2. Refresh fires during a transient Cloudflare 5xx window on api.anthropic.com (we've captured 502s repeatedly).
3. The refresh either fails silently or returns a token that the server then rejects.
4. The client retries (retryAttempt 1 → N), but eventually the server returns 401 with x-should-retry: false — the cascade ends in a fatal 401.
5. Session emits Please run /login · API Error: 401 Invalid authentication credentials and halts.
6. ~/.claude/.credentials.json is left on disk with the bad token, not cleared or marked invalid.
7. Next claude session boots, reads the file, and the very first API request returns 401 with no further activity required from the user.
8. User runs /login. New OAuth chain. Session works for 4–8h. GOTO 1.

Evidence (gathered from ~/.claude/projects/-root/*.jsonl)

• 697 prior 401 events across 48 distinct sessions, going back to 2026-04-24.
• Server response body is uniformly:

``json
{"type":"error","error":{"type":"authentication_error","message":"Invalid authentication credentials"}}
``

• Of 17 sessions where I could measure precise timing, 14 hit their first 401 within an hour of session start; 12 of those at minute zero (first request → 401). This means the credential is invalid before the user types anything.
• 14% of all 401 records are immediately preceded by a 5xx (typically 502 from cloudflare server header). Today's textbook example:

| time (UTC) | retry # | status | request-id |
|---|---|---|---|
| 2026-05-23T20:41:52 | 1 | 502 | — |
| 2026-05-23T20:41:56 | 5 | 502 | — |
| 2026-05-23T20:42:05 | 6 | 502 | — |
| 2026-05-23T20:42:22 | 7 | 401 | req_011CbL9f1GRrrouLuuybMkjV |

After that 7th attempt flipped to 401 with x-should-retry: false, the session died. The next fresh session ~2h later hit 401 on its very first request.

• Sample 401 request-ids (all authentication_error): req_011CaxA93YYVauXGq3C15cST, req_011CaxA96wSYCLHAtdqFMLNh, req_011CaxAC94GXMLYFRLRfFmAx, req_011CaxACCDXBm1m3kDckDXQ9, req_011CaxADFnriA67gn4ofz4aa.

What I ruled out

| Hypothesis | Result |
|---|---|
| Stale ANTHROPIC_API_KEY env var hijacking OAuth | Ruled out — no ANTHROPIC_* in any shell rc |
| External API key in /opt/bitmatrix/.env is bad | Ruled out — independently returns HTTP 200 against api.anthropic.com |
| Concurrent claude processes racing on credentials.json | Ruled out — ps shows only one PID at a time |
| Another device/install rotating the refresh chain | Ruled out — only this VPS |
| Cron / claude --schedule / routines | Ruled out — none exist |
| MCP OAuth flow clobbering main claudeAiOauth block | Plausible but unlikely — mcp-needs-auth-cache.json prevents re-loop |
| Project-specific config (/root vs another) | Artefact — "other project" hadn't been used since March |
| Specific Anthropic account/billing state | Ruled out — reproduced on a second account |

Suggested investigation

1. When the refresh endpoint returns 5xx (or returns 200 but the resulting token is then rejected), do not persist the unverified token to credentials.json. Leave the prior token, or clear and prompt for re-auth.
2. On 401 with authentication_error, detect that the file-on-disk token matches the in-memory token and proactively clear it, instead of writing the corrupted state back to disk for the next session to inherit.
3. Consider a "credentials sanity check" on session boot — a cheap /v1/messages ping with retry-on-5xx-only — to detect a stale credential before the user's first prompt fires.
4. The 502 → 401 mutation across retries (status code changes mid-retry-chain) suggests the retry logic may be substituting headers or stripping auth on retry. Worth tracing.

Workarounds (for affected users)

• Run /login proactively after any break > 4h.
• Close claude and restart before crossing ~4h continuous use.
• Watch ~/.claude/.credentials.json mtime — if it hasn't been updated in 4h+ and you're still in-session, you're about to hit it.

Repro environment can be made available

I have ~700 logged 401 events across 48 sessions in ~/.claude/projects/-root/*.jsonl and two live instrumentation watchers running. Happy to share sanitized timelines or specific session JSONLs on request.