[BUG] CLOUD_ML_REGION env variable is ignored for the Claude 3.5 Haiku POST when using LiteLLM+GoogleVertexAI

Resolved 💬 3 comments Opened May 23, 2025 by andreweskeclarke Closed May 28, 2025

Environment

  • Platform (select one):
  • [ ] Anthropic API
  • [ ] AWS Bedrock
  • [ ] Google Vertex AI
  • [x] Other: LiteLLM Proxy in front of Google Vertex AI
  • Claude CLI version: 1.0.2
  • Operating System: Linux
  • Terminal: iTerm2

Bug Description

I am using LiteLLM proxy in front of Google Vertex AI and the env variables are not quite meshing well:

command:
ANTHROPIC_VERTEX_PROJECT_ID="xxx-xxx-xxx" CLOUD_ML_REGION="us-east5" ANTHROPIC_VERTEX_BASE_URL="http://localhost:4000/vertex_ai/v1" ANTHROPIC_AUTH_TOKEN="sk-xxxxxxxxxxx" CLAUDE_CODE_SKIP_VERTEX_AUTH=1 CLAUDE_CODE_USE_VERTEX=1 claude

litellm logs a 500 error because the 3.5 Haiku URL has ignored my CLOUD_ML_REGION env variable and replaced it with a URL encoded space:
POST /vertex_ai/v1/projects/xxx-xxx-xxx/locations/%20/publishers/anthropic/models/claude-3-5-haiku%4020241022%3AstreamRawPredict HTTP/1.1" 500 Internal Server Error

See the %20 where it should say us-east5?

BUT! The normal model call is fine
"POST /vertex_ai/v1/projects/xxx-xxx-xxx/locations/us-east5/publishers/anthropic/models/claude-opus-4%4020250514%3AstreamRawPredict HTTP/1.1" 200 OK

Steps to Reproduce

Described above

Expected Behavior

I'd like the CLOUD_ML_REGION to be used in all URLs

Actual Behavior

It isn't used in all URLs

Additional Context

n/a

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗