Custom HTTP MCP server: OAuth authorization does not persist across launches, and server visibility is inconsistent across surfaces

Custom HTTP MCP server: OAuth authorization does not persist, and server visibility is inconsistent across surfaces

Summary

A custom HTTP-type MCP server (21st, https://21st.dev/api/mcp, header-based x-api-key auth plus an OAuth authorization step surfaced via /mcp) exhibits two related problems:

1. OAuth authorization does not persist across separate claude process launches in the terminal, even though it is stored at user/global scope (~/.claude.json).
2. Server visibility is inconsistent across different Claude surfaces (terminal CLI vs. a separate hosted/managed session type) in ways that don't correlate with configuration changes.

Environment

• macOS (Darwin 25.5.0), zsh
• Claude Code CLI (terminal) + a separate hosted Claude session surface (non-interactive, curated tool set — Gmail/Drive/Office connectors, computer-use, DesignSync, etc.)
• MCP server config: type: "http", url: "https://21st.dev/api/mcp", headers: { "x-api-key": "${API_KEY_21ST}" }
• API_KEY_21ST exported via ~/.zshrc

Chronology / reproduction steps

1. Installed the server at project scope via the vendor's own installer (21st init --client claude --write), which wrote mcpServers.21st into <project>/.mcp.json.
2. In the terminal, /mcp showed 21st under Project as needs authentication. Selected it → "Authenticate" → browser OAuth flow → "Authentication successful" → /mcp showed ✔ connected.
3. Opened a new conversation on the separate hosted surface, in the same project directory. 21st did not appear in that session's tool list at all (confirmed via a tool search / diagnostic call — no 21st-prefixed tools existed, whereas another user-scope stdio MCP server, firepuppet, configured in ~/.claude.json, was available and functional in that same hosted session).
4. Hypothesis: the hosted surface only loads user/global-scope (~/.claude.json) MCP servers, not project-scope (.mcp.json) ones — plausibly because project-scope servers require an interactive one-time trust prompt that a non-interactive surface can't satisfy.
5. Copied the same server definition into ~/.claude.json under mcpServers.21st (global/user scope), alongside the already-working firepuppet entry.
6. Opened another new conversation on the hosted surface: 21st now did appear in the tool list, but as needs authentication (progress — confirms the scope hypothesis above).
7. Went back to the terminal, ran /mcp: it now listed 21st twice — once under Project (✔ connected, from step 2) and once under User MCPs (needs authentication, the new global entry). Selected the User MCPs entry, authenticated via browser again → "Authentication successful."
8. Asked the user to open yet another new conversation on the hosted surface to confirm persistence. Before that could be tested, the user separately closed their terminal and opened a brand new terminal, ran claude, and /mcp → User MCPs 21st again showed needs authentication, despite having just been successfully authenticated in step 7.

• Inspected ~/.claude/mcp-needs-auth-cache.json: contained {"21st":{"timestamp":<matches the moment of this /mcp check>}} — confirms this is a fresh negative result, not a stale cache artifact.
• No token/credential file was found for this server elsewhere under ~/.claude/; a keychain search was attempted but abandoned (see Notes) since it started prompting for the user's macOS login/keychain password mid-search, which should not be necessary to diagnose an application-level config issue.
• Conclusion: the OAuth authorization for this HTTP MCP server does not persist across separate claude process launches, even at user/global scope. Contrast with firepuppet (stdio server, no OAuth step) and with claude.ai-linked connectors (Gmail/Drive/Indeed), which stay connected indefinitely without re-authorization.

1. Re-authenticated again in that terminal session (/mcp → Re-authenticate → succeeded: "Authentication successful. Reconnected to 21st.").
2. User then re-authenticated inside the hosted-surface conversation itself (not the terminal) that had been showing needs authentication. Result, per that session's own tool search: 21st had disappeared from the tool list entirely — no longer listed even as "needs authentication." Two hypotheses proposed by that session (unverified): either the authorization succeeded but that session doesn't hot-reload its tool list, or the connector simply isn't attached to that project/session.

Why this matters

Step 8 alone (auth not persisting across terminal launches) makes any HTTP-type custom MCP server with an OAuth step effectively unusable without re-authenticating every single session — a materially worse experience than stdio servers or claude.ai-native connectors. Step 10 suggests the problem compounds on hosted/managed surfaces, where the failure mode isn't even consistent (auth-required vs. server-absent) across attempts, making it unclear to an end user what state the connector is actually in or what action would fix it.

What would help

• Confirmation of whether OAuth tokens for user-scope HTTP MCP servers are intended to persist across claude process restarts (if yes: this is a bug in token storage/retrieval; if no: the /mcp UI should say so instead of implying a normal one-time authorization).
• Clarity on whether project-scope (.mcp.json) MCP servers are intentionally excluded from non-interactive/hosted session surfaces, and if so, documentation of that behavior so users aren't led to duplicate config across scopes as a workaround.
• Investigation into why the hosted surface's reported state for the same server flipped from "needs authentication" to "absent from tool list" between two consecutive conversations with no configuration change in between.

Notes

• A security dump-keychain command run during diagnosis triggered a macOS keychain password prompt on the user's screen. This was an unnecessary diagnostic step and was aborted without the user entering their password — flagging only in case it's relevant to how/where this type of credential should be stored (it apparently isn't using the keychain today, based on the empty search results before it was aborted).