Random Timeout with MCP Servers

There seem to be an intermittent problem with some MCP servers.

For example, when I add the brave search mcp server:

$ claude mcp add brave_search --env "BRAVE_API_KEY=$(skate get brave_search_api)" -- npx -y @modelcontextprotocol/server-brave-search
Added stdio MCP server brave_search with command: npx -y @modelcontextprotocol/server-brave-search to project config

It says it failed to connect:

╭─────────────────────────────────────────────────────╮
│ ✻ Welcome to Claude Code research preview!          │
│                                                     │
│   /help for help                                    │
│                                                     │
│   cwd: /home/christian/dev/oss/awesome-mcp-servers  │
│                                                     │
│   ───────────────────────────────────────────────── │
│                                                     │
│   MCP Servers:                                      │
│                                                     │
│   • brave_search                             failed │
╰─────────────────────────────────────────────────────╯

And the logs says:

$ cat ~/.cache/claude-cli-nodejs/-home-christian-dev-oss-awesome-mcp-servers/mcp-logs-brave_search/2025-02-28T13-08-47-819Z.txt
[
  {
    "error": "Connection failed: Connection to MCP server \"brave_search\" timed out after 5000ms",
    "timestamp": "2025-02-28T13:08:52.985Z",
    "sessionId": "2d2f8a48-c639-4c2d-bee6-357225021fb1",
    "cwd": "/home/christian/dev/oss/awesome-mcp-servers"
  }
]

But then I try again and it works.

Could it be that 5000 ms timeout is too short? It needs enough time to download the code and start it up, right?
Maybe it could do that in the background after 5000ms rather than timing out?