[BUG] Capabilities behind ToolSearch are reached for less reliably than loaded tools, producing unnecessary user prompts and invented workarounds

Status Open
Maintainer reply None cached
Activity 0 comments · opened Aug 5, 2026

Preflight Checklist

  • [x] I have searched existing issues and this hasn't been reported yet
  • [x] This is a single bug report (please file separate reports for different bugs)
  • [x] I am using the latest version of Claude Code

What's Wrong?

In a Cowork session with a large deferred-tool set, the assistant repeatedly failed to reach for capabilities that were available but not loaded. Rather than running ToolSearch to acquire a relevant tool, it either asked the user a question the deferred tool could have answered, or invented a workaround for a constraint the deferred tool would have removed.

Both outcomes look like reasonable behaviour. Nothing errors. The failure is only visible to a user who happens to know the capability existed.

Worked instance, 2026-08-05.

Task: determine whether a particular GitHub issue had been filed by the user.

Available but deferred: mcp__session_info__list_sessions and mcp__session_info__read_transcript, which read prior session transcripts on the same machine.

What happened: the assistant classified the question as not answerable locally and asked the user directly. It did not run ToolSearch. The user pushed back. On the second attempt the assistant loaded both tools in a single ToolSearch call and answered from local records in under a minute, including a full-text search of the local document store that returned a clean negative.

The rule requiring a local-source check before asking was already present in the project's always-loaded CLAUDE.md, and separately in a persistent memory file. Neither prevented the failure.

Precedent in the same setup, 2026-05-17. From the user's own dated change log: a session hit a filesystem permission error, did not search for mcp__cowork__allow_cowork_file_delete, and worked around it by truncating a file into a stub, which then had to be cleaned up in a second pass. The user subsequently wrote a persistent memory reading, in substance, "on bash Operation not permitted, look for mcp__cowork__allow_* before working around it."

That memory exists because of this failure mode. It did not prevent the 2026-08-05 recurrence.

What Should Happen?

A capability that is available should be reached for at approximately the rate it would be if it were present in the loaded tool list.

Concretely: when the model is about to tell the user that something cannot be determined, or is about to route around a blocked operation, the deferred-tool space should be searched first.

Error Messages/Logs

Steps to Reproduce

  1. Configure a Cowork project with a large deferred-tool set (see Additional Information for the regime where this was observed).
  2. Give the assistant a task whose answer requires a deferred tool that is not obviously implied by the request, for example a question about prior sessions when session_info is deferred.
  3. Do not name the tool.
  4. Observe whether the assistant runs ToolSearch, or instead asks a clarifying question or states that the information is not available.

Not deterministic in our experience. This is reported as a rate question rather than a hard reproduction, which is why the suggested eval in Additional Information is framed as two arms.

Claude Model

Opus

Is this a regression?

I don't know

Last Working Version

Claude 1.25927.0 (003700) 2026-08-04T05:34:55.000Z

Claude Code Version

Claude 1.25927.0 (003700) 2026-08-04T05:34:55.000Z (desktop app, Cowork mode)

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

Terminal.app (macOS)

Additional Information

Hypothesis

Offered as testable, not as established.

A capability behind a search step is acquired less reliably than one present in the tool list, because the two branches have asymmetric cost. Noticing that a deferred tool might exist, forming a query, and spending a round trip is a real cost. Concluding "I cannot determine this, I will ask" costs nothing and produces output that reads as careful. Under context pressure the asymmetry may bias toward the second branch.

If that is right, the effect should scale with the size of the deferred set and with how full the context already is when the decision is made.

The eval that would settle it

A fixed task battery where each task requires exactly one non-core tool. Two arms, everything else held constant (model, effort, harness build, instruction file, task order):

  • Arm A: the required tool is present in the loaded tool list.
  • Arm B: the identical tool is behind ToolSearch.

Measure, per task: whether the model acquired and used the tool, versus asked the user, versus produced a workaround, versus asserted the task was not possible. A materially lower acquisition rate in Arm B would confirm the effect. A flat result would refute it, and reports of this shape should then be read as ordinary instruction-following variance instead.

A useful secondary cut: hold the arm constant and vary context fill, to test whether the gap widens as the window fills.

What we ruled out first

Recorded because triaging a complaint wave presumably means separating real regressions from baseline variance, and this is a worked example of a heavy user looking hard and finding nothing.

The user's initial read was that quality had regressed with the Opus 5 default switch on 2026-07-24. We tested that against a dated, first-party failure log maintained continuously since 2026-02, 16 numbered entries, written before this question was asked.

The naive comparison supports a regression: 8 dated entries across the 68 days before 2026-07-24 (0.119/day) against 5 across the 13 days after (0.385/day), a 3.3x increase, roughly p=0.02 under a Poisson null.

It is an artifact. The 68-day baseline includes a period of near-dormancy: four consecutive weeks in June show 1, 1, 4 and 4 file modifications. Comparing intense use against near-disuse manufactures the spike. Matched on activity instead, using the busy stretch immediately prior (2026-07-13 to 07-23) against 07-24 to 08-05: 0.364/day against 0.385/day. No signal. With 4 and 5 events nothing smaller than roughly a doubling would be detectable, so this is "no signal", not "proven flat".

Independent throughput check over the same boundary, using per-week volume of the user's weekly log: 14.9k/day, 16.9k/day, 26.2k/day across the three weeks spanning the switch. Activity rose after it, which inflates raw failure counts at a constant error rate.

Four of the eight pre-Opus-5 entries are the same failure class described in this report, the earliest dated 2026-05-17.

One methodological warning, in case it is useful to whoever triages these: a files-modified-per-week metric built on mtime is unusable as a time series, because mtime records only the most recent touch, so any file edited twice lands entirely in the later bucket. It showed a 4x spike here and was discarded.

Environment detail

Sanitized. No file contents, names, or personal data.

  • Cowork mode, Claude desktop app, macOS. Models observed: Sonnet 5 and Opus 5.
  • Always-loaded project instruction file: 42.2 KB, 295 lines, roughly 10.5k tokens. Note this is well past the under-200-lines guidance in the memory docs, and is itself a plausible contributor.
  • A second always-loaded CLAUDE.md from another mounted folder: 4.3 KB.
  • 7 mounted folders.
  • Roughly 46 MCP servers configured, a substantial share unauthenticated or slow to connect, with connect and disconnect notices arriving mid-session.
  • Roughly 180 skills across about 20 plugins.
  • An index file read on most tasks in this project: 64 KB, roughly 16k tokens.

What is not claimed

  • Not claimed that this is new, or that it began with any particular release.
  • Not claimed that the mechanism is proven. The instance count is small, and the assistant's own account of why it failed is post-hoc and should carry no evidentiary weight.
  • Not claimed that the user's configuration is blameless. The instruction file is oversized by Anthropic's own published guidance, and the MCP roster is large. Those are plausible contributors and the eval design controls for neither.

The narrow ask is the two-arm eval. If the gap is real it is a harness property that no amount of user-side rule-writing can fix, which is the practical reason for filing rather than adding another rule.

Related

Two open reports filed in the last 48 hours describe different failures in the same subsystem. Taken together with this one they suggest deferred tools have a lifecycle problem rather than three unrelated bugs:

  • #83756 (2026-08-04), acquisition is expensive. Telemetry showing that a single ToolSearch loading roughly 6 or more MCP schemas triggers a full prompt-prefix rebuild, against three separate documentation statements that tool search preserves the cache.
  • #84189 (2026-08-05), retention is unreliable. Deferred schemas are not restored after compaction, so the model reconstructs the call shape from memory and fails validation repeatedly. That report's own diagnosis is "the failure mode is not knowing the schema is gone."
  • This report, acquisition is not attempted. The tool is never searched for, so the model asks the user or works around the gap.

This report does not claim that #83756's cache cost explains the behaviour here. That cost is borne by the user and is not visible to the model, so it cannot be a behavioural driver. It is cited only as independent evidence that the deferred path is not a neutral no-op and that its properties are worth measuring.

Cowork instruction-adherence reports in the same window, listed for pattern-matching rather than as duplicates: #84229, #84070, #84197, #84148.

View original on GitHub ↗