Opus 4.6 regression: LLM guesses instead of using tools, wastes 4 round trips on trivially answerable question

Summary

Opus 4.6 repeatedly guesses answers to codebase questions instead of using available tools, wasting 4 round trips on a question answerable in one. This pattern was not present in earlier Opus 4.6 configurations and represents a performance regression.

Reproduction (anonymized)

Context: After a test run, 2 tests failed due to external API timeouts on backend endpoints.

User: "which ones didn't work due to timeout?"
LLM: Correctly lists the 2 failing backend API endpoints.

User: "not the API, the frontend link to these funnels"
LLM (attempt 1): Still gives the backend API endpoints, just reformatted. Ignores "frontend" which clearly refers to the frontend app on a different port.

User: "no, /api/ is not a frontend link"
LLM (attempt 2): Guesses frontend URLs without checking the codebase. Gets them wrong (404).

User: "no they 404"
LLM (attempt 3): Instead of searching the codebase (trivial: list the pages directory), asks the user for the answer.

User: "you know, stop being lazy"
LLM (attempt 4): Finally searches the pages directory and finds the correct paths in seconds.

Regression

This behavior was not present in earlier Opus 4.6 configurations. Previously, when asked a codebase question the model would immediately use tools (Glob, Read, Bash) to find the answer. Now it:

1. Tries to answer from memory (wrong)
2. Guesses again (wrong)
3. Asks the user instead of searching (lazy)
4. Only uses tools when explicitly told to

The model had full codebase access with Bash, Glob, Grep, and Read tools available the entire time.

Expected Behavior

User: "not the API, the frontend link to these funnels"
LLM: Searches the pages directory → finds the page files → returns correct URLs. One step.

Why This Matters

• Each round trip costs tokens (input context grows with every message)
• By attempt 4, the full conversation history is re-sent, multiplying the waste
• The pattern (guess → fail → guess → fail → ask user → finally search) burns tokens at every step
• Token waste ratio here: ~5x what was needed

Suggested Benchmark Dimension

This could be tracked as: "codebase-answerable questions where the LLM has tool access but fails to use tools."

Metrics:

• Round trips to reach correct answer (optimal: 1, actual: 5)
• Token waste ratio (tokens spent / tokens for optimal path)
• Rate of "ask user" fallback when answer is discoverable in codebase
• Comparison across model versions to detect regressions