[FEATURE] Support indexed code search tools for reduced token consumption
Preflight Checklist
- [x] I have searched existing requests and this feature hasn't been requested yet
- [x] This is a single feature request (not multiple features)
Problem Statement
Claude Code's Grep tool uses ripgrep for every code search. This works, but in a
typical session the agent makes 50-200 searches, and each one returns raw text lines
with no structural context. The agent then has to read full files to understand what
the results mean.
I measured this on a real codebase (ripgrep's source, 208 files): a 3-query
investigation task consumed 20,580 tokens -- 45% of that was file reads just to
understand grep results.
By replacing grep with an indexed search that returns function signatures + call
graph context (using trigram indexing + tree-sitter), the same task took 2,814
tokens. 87% reduction, measured.
The core issue: the Grep tool returns lines. Agents need structure -- function
bodies, who-calls-what, what-breaks-if-I-change-this. That understanding currently
requires multiple tool calls (grep + read + grep + read), which burns tokens and
adds latency.
Proposed Solution
Allow Claude Code to use indexed/structural search tools alongside or instead of
ripgrep. This could be:
- A hook or config that lets users specify an alternative search command
(e.g., a tool that returns JSON with function signatures + call graph)
- Native support for search results that include structural context
(enclosing function, callers, callees)
- MCP server integration for code intelligence tools
I built an open-source prototype (Hypergrep) that demonstrates the approach:
- Trigram-indexed search (4.4ms warm vs ripgrep's 31ms)
- Returns full function bodies instead of lines (tree-sitter, 16 languages)
- Call graph queries (--callers, --impact) in microseconds
- Semantic compression: 3 output layers with token budget fitting
- 120 tests, MIT license
GitHub: https://github.com/marjoballabani/hypergrep
Benchmarks: https://github.com/marjoballabani/hypergrep/blob/main/BENCHMARKS.md
Not suggesting Claude Code adopt Hypergrep specifically -- just that the
architecture would benefit from supporting structured search results rather
than raw grep lines.
Alternative Solutions
_No response_
Priority
Critical - Blocking my work
Feature Category
CLI commands and flags
Use Case Example
_No response_
Additional Context
_No response_
This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗