Search output is substituted for reading, and the verdict still covers files that were never opened

Summary

Claude Code substitutes targeted searches for reading, then issues a conclusion whose scope implies the reading was done. Grep answers only the questions the model already knew to ask; the purpose of reading a diff is to find the problems it did not know to look for.

What happened

Asked to review an 827-line pull request across seven files, the model read:

• the PR metadata and the comment thread
• one two-line hunk in a shared core file
• the output of two grep passes across the diff, for enum strings and environment variable names

It did not read the 534-line main implementation, the 158 lines of new tests, or a 39-line service the change introduces. It then issued a Reject verdict covering the whole change.

The verdict agreed with the maintainer's own earlier review, which is what makes it worthless: the thread already contained his "changes requested" and the contributor's acknowledgement. He had asked for an independent check precisely so he would have something other than an echo. One question — "did you actually read what he's doing, or just based in the comments?" — collapsed it.

The change in question builds container images from user-supplied archive artifacts and runs containers from them. The defects that matter in code like that are not greppable.

Why this matters

Grep is a confirmation tool. It can verify a hypothesis and it cannot generate one. When the model reaches for it in place of reading, the resulting review is structurally incapable of finding anything the model had not already guessed, while the output is indistinguishable from a review that read everything.

This is a distinct failure from skipping a step outright. The model performs a form of inspection, which makes the shortcut invisible: the message can cite line numbers and symbol names it never read in context.

It compounds with an availability effect. In the same session the model reviewed three changes and skipped a different required reading step in each: reviewing from the diff without opening the touched files; verdicting before reading the linked issue; and this one, verdicting without reading the implementation. In every case the skipped step was the one requiring sustained reading with no intermediate output.

Expected

1. A verdict covering N files requires having read N files. If the model has read a subset, the output must say which subset and must not carry a conclusion about the rest.
2. Do not accept search output as a substitute for reading when the task is "review this change." Searches supplement a read; they do not constitute one.
3. When a conclusion is already present in the thread, treat agreement with it as the weakest possible signal and state explicitly what was independently verified.
4. Track which files were actually opened during a review and surface that alongside the verdict, so the reader can see the basis rather than infer it.

Environment

• Claude Code CLI, Opus-class model, paid subscription
• Long code-review sessions on an open-source project with a written review procedure supplied via a project instruction file and a persistent memory directory
• Related: #96416, #96420, #96423, #96424

---

Co-authored with Claude Code.