[BUG] CLOUD_ML_REGION env variable is ignored for the Claude 3.5 Haiku POST when using LiteLLM+GoogleVertexAI

Status Fixed / completed
Maintainer reply ✓ Yes — ant-kurt
Activity 3 comments · opened May 23, 2025 · closed May 28, 2025
💡 Likely answer: A maintainer (ant-kurt, collaborator) responded on this thread — see the highlighted reply below.

Environment

  • Platform (select one):
  • [ ] Anthropic API
  • [ ] AWS Bedrock
  • [ ] Google Vertex AI
  • [x] Other: LiteLLM Proxy in front of Google Vertex AI
  • Claude CLI version: 1.0.2
  • Operating System: Linux
  • Terminal: iTerm2

Bug Description

I am using LiteLLM proxy in front of Google Vertex AI and the env variables are not quite meshing well:

command:
ANTHROPIC_VERTEX_PROJECT_ID="xxx-xxx-xxx" CLOUD_ML_REGION="us-east5" ANTHROPIC_VERTEX_BASE_URL="http://localhost:4000/vertex_ai/v1" ANTHROPIC_AUTH_TOKEN="sk-xxxxxxxxxxx" CLAUDE_CODE_SKIP_VERTEX_AUTH=1 CLAUDE_CODE_USE_VERTEX=1 claude

litellm logs a 500 error because the 3.5 Haiku URL has ignored my CLOUD_ML_REGION env variable and replaced it with a URL encoded space:
POST /vertex_ai/v1/projects/xxx-xxx-xxx/locations/%20/publishers/anthropic/models/claude-3-5-haiku%4020241022%3AstreamRawPredict HTTP/1.1" 500 Internal Server Error

See the %20 where it should say us-east5?

BUT! The normal model call is fine
"POST /vertex_ai/v1/projects/xxx-xxx-xxx/locations/us-east5/publishers/anthropic/models/claude-opus-4%4020250514%3AstreamRawPredict HTTP/1.1" 200 OK

Steps to Reproduce

Described above

Expected Behavior

I'd like the CLOUD_ML_REGION to be used in all URLs

Actual Behavior

It isn't used in all URLs

Additional Context

n/a

View original on GitHub ↗

3 Comments

gboston · 1 year ago

Experiencing the same issue. CLOUD_ML_REGION isn't used and just seems to be empty on LiteLLM

ant-kurt collaborator · 1 year ago

Thanks for reporting this - will be resolved in the next release.

EDIT: v1.0.4 is now released

github-actions[bot] · 1 year ago

This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.