[FEATURE] Lazy context loading: extend the ToolSearch pattern to all context components

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

Claude Code loads ALL context at startup: CLAUDE.md files, memory, MCP servers, skills, rules, tool definitions — regardless of whether the session needs any of it. There's no way to load context on demand.

The only alternative is --bare, which strips everything (and is currently buggy — #38676 still loads user configuration including MCP servers). It's all-or-nothing: either every component loads, or none do.

**Startup token overhead (out of 200K default context window):*

• MCP tool definitions (7 servers): 67,300 tokens / 33.7%
• Skills (20+ plugins): 30,000–40,000 tokens / 15–20%
• Tool definition behavioral instructions: ~15,000+ tokens / ~7.5%+
• CLAUDE.md files: 5,000–15,000 tokens / 2.5–7.5%
• Rules (re-injected per tool call): 6,200+ tokens / 3%+
• System prompt infrastructure: ~10,000 tokens / 5%
• Total startup overhead: ~130,000–150,000 tokens / 65–75%

_*Some model tiers (e.g., Opus) support up to 1M tokens, but the proportional overhead remains significant._

Users lose up to 75% of their context window before typing anything. On Max plans, typing "hi" consumes 4% of the usage budget (#42483). Session resume incurs significant overhead (#42009). Rules re-inject on every tool call, consuming 46% of context over a session (#32057).

ToolSearch already proved that lazy loading works: 85% token reduction for tool schemas. The single feature request here is: extend this same pattern to all context components (skills, MCP connections, rules, instructions), rather than only tool schemas. The phases below are an incremental implementation path for that one capability, not separate features.

Proposed Solution

Claude Code should extend the existing ToolSearch pattern to all context components: load them on demand instead of all at startup. ToolSearch already proves the pattern works (85% token savings for tool schemas). The same "register metadata, load on demand" approach should apply to skills, MCP connections, rules, and instructions.

Here's an incremental implementation path:

Phase 1: Fix what's broken (low effort, high impact)

1. Fix --bare mode: actually skip MCP servers as documented (#38676)
2. Stop re-injecting rules on every tool call: inject once, re-inject only when the relevant file set changes (#32057). This alone saves 46% of context over a session.
3. Move behavioral instructions out of tool definitions: thousands of tokens of git commit protocols, dialogue examples, etc. are embedded in Bash/TodoWrite tool descriptions (#26158). Move to system prompt where they can be compressed.

Phase 2: Extend the ToolSearch pattern (medium effort)

1. SkillSearch: same pattern as ToolSearch for skills. Name + one-line description at startup; full body loaded on invoke (#43816).
2. Lazy MCP connections: register servers at startup (name + tool list), but don't connect or load full schemas until a tool from that server is actually requested (#40019).

Phase 3: Context profiles (medium-high effort)

1. Named profiles: claude --profile minimal, claude --profile dev, claude --profile pm. Each profile specifies which skills, MCP servers, rules, and memory files to load. Config in settings.json or .claude/profiles.json, with glob patterns for skill/rule selection and mid-session switching via /profile.

Phase 4: Action-triggered loading (high effort, highest long-term value)

1. Context-aware instructions: instructions load based on what Claude is about to do, not what's configured globally. Example: .claude/instructions/on-test.md loads when about to run tests (#39854).
2. MCP context isolation: MCP servers load into forked agent contexts instead of the main context, using existing context: fork infrastructure (#17668).
3. Dynamic unloading: tools loaded via ToolSearch that haven't been used for N turns return to deferred state. Context self-cleans over long sessions.

Risks and mitigations

Based on real problems already observed with ToolSearch:

• Latency spikes: lazy MCP connections could add seconds. Mitigate with warm-start pools for recently-used servers.
• First-turn unavailability (#42148): deferred tools break automated tasks. Mitigate with --eager-tools / --eager escape hatch.
• Context compression loses references (#42835): deferred names vanish after /compact. Mitigate by marking deferred references as non-compressible.
• Hook conflicts (#33073): PreToolUse hooks hang after deferred loading. Needs architectural fix.
• --print mode deadlocks (#35262): CI pipelines need predictable, eager behavior by default.
• Clear visibility: a /status command showing what's loaded, what's deferred, and what's available.

Alternative Solutions

What exists today:

• ToolSearch (shipped): defers MCP tool schemas until needed, 85% token reduction. But: first-turn unavailability (#42148), compression loses refs (#42835), hook conflicts (#33073).
• --bare flag (shipped): skips all auto-discovery. But: all-or-nothing; still loads MCP servers (#38676); no auth (#38022).
• context: fork (shipped): skills run in isolated sub-contexts. But: only for skills, not MCP or rules.
• /compact (shipped): compresses context mid-session. But: lossy workaround, not prevention.

Community workarounds:

• skills-inactive directories to manually disable skills
• [lazy-mcp](https://github.com/voicetreelab/lazy-mcp): open-source MCP proxy providing lightweight tool summaries with full schemas on demand
• Manual MCP server toggling per session

Prior art in other tools:

• VS Code Extension Activation Events: extensions declare triggers (onLanguage:python, onCommand:X) and don't load until fired
• IntelliJ Plugin Lazy Loading: plugins declare extension points; only initialize when the platform requests their contribution
• Webpack Code Splitting: dynamic import() loads modules on demand
• LSP: servers declare capabilities upfront; clients only send requests for supported features

Priority

High - Significant impact on productivity

Feature Category

Performance and speed

Use Case Example

Quick question, no project context needed:

claude --profile minimal
> "What's the difference between useMemo and useCallback?"
# Zero MCP connections, zero skills loaded, zero rules injected

Starting a research session, then switching to coding:

> "Research how other tools handle rate limiting"
# Only web search MCP loads
> "Now let's implement it in our API"
# Project CLAUDE.md, relevant skills, dev server rules load on first file edit

CI/automation (predictable, no lazy loading):

claude --bare --eager-tools -p "Run the test suite and report failures"
# Everything needed is explicit; no deferred loading surprises

Multi-role team using profiles:

claude --profile pm    # loads PM skills, hides dev tools
claude --profile dev   # loads dev skills, MCP servers, testing rules
claude --profile ops   # loads infra skills, monitoring MCP

Additional Context

Related issues (19 total):

Primary:

• #17668: MCP Context Isolation (detailed feature request with architecture proposal)
• #43816: SkillSearch (feature request with design rationale and metrics)
• #14882: Skills load full content at startup despite docs claiming otherwise
• #32057: Rules re-injected every tool call (46% context waste)
• #26158: Behavioral instructions embedded in tool definitions (significant token overhead)
• #38930: --eager-tools request

Supporting evidence:

• #40019: Lazy MCP connections
• #44371: Plugin skills load full content at startup
• #41812, #42483, #42009: Token waste user reports
• #30423: Skill profiles
• #38676: --bare mode broken
• #42835, #42148, #33073, #35262: Existing deferred loading bugs

Summary: The ToolSearch pattern already proved that lazy loading works and saves 85% of tokens for tool schemas. This proposal extends that pattern systematically to all context components. The community has already built workarounds (skills-inactive directories, lazy-mcp proxies, manual MCP toggling). The question is whether to let the ecosystem fragment around workarounds or ship a first-party solution.