[BUG] Claude Code is not local inference friendly
Preflight Checklist
- [x] I have searched existing issues and this hasn't been reported yet
- [x] This is a single bug report (please file separate reports for different bugs)
- [x] I am using the latest version of Claude Code
What's Wrong?
When Claude Code is calling the model endpoint, 5 minutes timeout limit is hard-coded.
If the model is not light speed and requires more than 5 minutes thinking on a long and complex context, then Claude Code fails after 2 attempts with error message "API Error: terminated".
What Should Happen?
Claude Code should wait long enough to give a chance for the model to produce some response.
Please make this timeout value configurable with an environment variable.
API_TIMEOUT_MS variable is used for prompt processing timeout. For prompt processing, there are 10 attempts giving a chance to finish PP phase.
The main problem is that this variable maximum value is 5 minutes, any bigger value is clamped.
The thinking timeout could use the same variable, while at the same time removing the 5 minutes clamping. At this time, API_TIMEOUT_MS is not used for the A
Error Messages/Logs
API Error: terminated
Steps to Reproduce
Connect Claude Code to a thinking model with very low performance. After 2 attempts (5 minutes each) the message appears.
Claude Model
None
Is this a regression?
I don't know
Last Working Version
_No response_
Claude Code Version
2.1.77 (Claude Code)
Platform
Other
Operating System
Ubuntu/Debian Linux
Terminal/Shell
Xterm
Additional Information
_No response_
This issue has 4 comments on GitHub. Read the full discussion on GitHub ↗