New Features: Decrease cost by using different LLM

Feature Description

I have been extensively used the tools for couple weeks now, and one of the biggest negative is the cost that scale quickly compared to other tools.

Pros: The cost is usually compensated by providing better value overall most of the time.
Cons: When facing situations which are inevitable with agentic AI, unproductive code can be very expensive.

For example:
We lost 100$ yesterday on simple tasks that were costly in tokens. During refactoring code to regress a bug, the agentic AI made multiple mistakes generating deprecated code or breaking syntax between library versions, and circle back to the initial state of the code. than circling back to initial stage. because the agentic AI for some reason became confused and start to perform circular logic behaviours that were costly in tokens.

Anthropovic doesn't offer any way to report the waste of tokens usage, like in the logs were we could report a tokens session to be investigate by Anthropovic staff and refund based on decisions. And also allow the Anthropovic team to review the session and use it as telemery to improve the behaviours of the AI.

Solution:

Support for multiple LLM interface that can be changed per context.

Example 1: Normal prompt chat doesn't require a powerful AI to understand user intent and translate it into tasks. A cheaper AI or even free Ollama could be used, discretionary to the user preferences.

Example 2: Committing code doesn't require powerful AI to create a meaningful message and sending the tool agent to perform the task and receiving the state of the task. Again a cheaper LLM or a local Ollama could do it.

Example 3: Sometime we just want to explore small variation of an idea or something that doesn't require a large context windows to which a cheaper AI could perform sufficiently good. Using a cheaper LLM would be beneficial and allow user to confirm an idea before asking the powerful AI to do the real task.

Example 4: Managing small tasks like documenting code, extracting data, could be perform by cheaper AI, while letting the bigger AI do the orchestration that require a larger windows context. Like having an army of dumb AI to do simple tasks, while letting the overall big picture being managed by the AI driving them.

Having possibility to integrate different LLM, especially free LLM that can be run locally, would be very welcome by every users. I don't think it will affect your monetization. The value of the more powerful AI is still significant that it creates an organicdependency.

It would be acceptable to not provide access to competing LLM API by restricting an interface only to local LLM on Ollama or a proprietary implementation that support local LLM run. It would add incredible value to Claude Code.

Market Fit

From my discussion on many forums, and for the MAIN reason why people had attempted to reverse engineer Claude Code was to implement that capability. This is in my opinion the most SEEK feature requested in public space.