Tokens per second need to be improved
Resolved 💬 3 comments Opened May 4, 2025 by rivershah Closed Jan 3, 2026
For agentic workflows, the standard gpu based inference token per second speeds that anthropic are running is not going to cut it. mistral and meta are partnering with cerebras to get token speeds to great level.
Will anthropic please consider at least groq level speeds?
cline does have sambanova inference with deepseek-r1-671b but the user experience is not as strong as claude-code.
Please either move to these specialist inference providers or give max users inference speeds that are 5-10x current token per seconds via dedicated gpu clusters.
Want to stick with cluade-code and anthropic and not move to cline
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗