[BUG] CLOUD_ML_REGION env variable is ignored for the Claude 3.5 Haiku POST when using LiteLLM+GoogleVertexAI
Environment
- Platform (select one):
- [ ] Anthropic API
- [ ] AWS Bedrock
- [ ] Google Vertex AI
- [x] Other: LiteLLM Proxy in front of Google Vertex AI
- Claude CLI version: 1.0.2
- Operating System: Linux
- Terminal: iTerm2
Bug Description
I am using LiteLLM proxy in front of Google Vertex AI and the env variables are not quite meshing well:
command:ANTHROPIC_VERTEX_PROJECT_ID="xxx-xxx-xxx" CLOUD_ML_REGION="us-east5" ANTHROPIC_VERTEX_BASE_URL="http://localhost:4000/vertex_ai/v1" ANTHROPIC_AUTH_TOKEN="sk-xxxxxxxxxxx" CLAUDE_CODE_SKIP_VERTEX_AUTH=1 CLAUDE_CODE_USE_VERTEX=1 claude
litellm logs a 500 error because the 3.5 Haiku URL has ignored my CLOUD_ML_REGION env variable and replaced it with a URL encoded space:POST /vertex_ai/v1/projects/xxx-xxx-xxx/locations/%20/publishers/anthropic/models/claude-3-5-haiku%4020241022%3AstreamRawPredict HTTP/1.1" 500 Internal Server Error
See the %20 where it should say us-east5?
BUT! The normal model call is fine"POST /vertex_ai/v1/projects/xxx-xxx-xxx/locations/us-east5/publishers/anthropic/models/claude-opus-4%4020250514%3AstreamRawPredict HTTP/1.1" 200 OK
Steps to Reproduce
Described above
Expected Behavior
I'd like the CLOUD_ML_REGION to be used in all URLs
Actual Behavior
It isn't used in all URLs
Additional Context
n/a
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗