[BUG] Fine-grained tool streaming cannot be turned on when experimental betas are off
Preflight Checklist
- [x] I have searched existing issues and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
What's Wrong?
When I run the latest Claude Code (2.1.296) with the following env vars, for a custom base URL (LLM gateway) and with telemetry off, experimental betas disabled, and trying to enable the fine-grained tool streaming:
ANTHROPIC_BASE_URL=https://... (not api.anthropic.com)
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1
CLAUDE_CODE_ENABLE_FINE_GRAINED_TOOL_STREAMING=1
DISABLE_TELEMETRY=1
Per the current Claude Code environment variable docs, this fine-grained tool streaming is currently off by default when using a custom base URL, and is supposed to be able to be turned on with this env var:
Set to 1 to force on when routing through a proxy via ANTHROPIC_BASE_URL, ANTHROPIC_VERTEX_BASE_URL, or ANTHROPIC_BEDROCK_BASE_URL.
Problem: In the requests to the messages endpoint, the tools are not marked with eager_input_streaming: true.
This is verified by running mitmproxy inline on the connection to the custom base URL and inspecting the request body.
As a result, when thinking and output time to produce a tool call exceeds 10 minutes without any streaming result, various timeouts on our inference path are hit and the request is treated as errored out.
API Error: Connection lost mid-response. The response above may be incomplete.
What Should Happen?
In the requests to the messages endpoint, the tools should be marked with eager_input_streaming: true
Error Messages/Logs
[DEBUG] [betas] Stripped from tool schemas: [eager_input_streaming] (experimental betas disabled)
Steps to Reproduce
- Start Claude Code with settings to connect to a custom base URL and fine-grained tool streaming on
````
ANTHROPIC_BASE_URL=https://... (not api.anthropic.com)
CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1
CLAUDE_CODE_ENABLE_FINE_GRAINED_TOOL_STREAMING=1
DISABLE_TELEMETRY=1
- Send a prompt
- Observe using a mitm proxy on the connection to the custom base URL that the tools are not marked with
eager_input_streaming: true, or observe in the Claude Code debug log that log message.
Claude Model
Other
Is this a regression?
No, this never worked
Last Working Version
_No response_
Claude Code Version
2.1.296
Platform
Other
Operating System
macOS
Terminal/Shell
iTerm2
Additional Information
This is using a custom base URL that points to a LLM gateway (Anthropic messages API) that connects to Bedrock models like Sonnet 5.5 and Opus 5.5, but I believe that is not material to this discussion as the issue here is identified to be due to the request that is generated by Claude Code, and nothing to do with the subsequent handling.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗