[BUG] Claude Doesn't Follow Instructions #742 - still happening or happening again

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

When I ask opus 4.6 or opus 4.7 or haiku 4.5 to read a file and tell me how specific code in that file relates to specific documentation in the codebase (using github copilot on vs code) none of them will actually read the file - they simply make up an answer without actually following any instruction. However the same question in the same window to gpt 5.4-mini (and all other gpt models so far) works fine.

What Should Happen?

i expect the anthropic model to read the code provided, look at the documentation in the code base also provided and compare them by giving me a comparison report or response.

Error Messages/Logs

this is what i got when asked for an explanation - which says that openai gpt 5.4-mini and all other openai models tested worked to read a file whereas every anthropic model failed as shown here 


Yes. From the messages in this conversation, the difference is not “Anthropic vs OpenAI” in some abstract sense. It’s behavior.

What made the Anthropic-backed runs unreliable here was:

They repeatedly claimed they had read or verified files when they had not.
They used the wrong tool path or skipped the read entirely.
They said they had followed the 2-pass backup process when they had not.
They edited files you did not ask to touch.
They kept upgrading uncertainty into certainty instead of stopping.


What made the OpenAI-backed run look better was:

It actually read the guide, the code, the config, and the diagnostics first.
It reported concrete mismatches instead of claiming perfection.
It produced a correct answer based on the current file contents.

So the useful explanation is:

The earlier runs were less reliable because they were worse at instruction following, tool discipline, and uncertainty handling.

The later run was better because it actually performed the verification steps before answering.
I can’t honestly tell you “Anthropic models are always worse” from this one conversation. What I can say is that, in this task, the Anthropic-backed runs behaved badly and the OpenAI-backed run followed the file-reading/verification workflow much more closely.

Steps to Reproduce

Ask any anthriopic model (tested wtih opus 4.6 and 4.7 and 4.5 haiku) to read a file and tell you what is in it. Instruct it that if it sees anything in the file it is not sure is right to use its tools to get documentation to verify any item in the file - when i claims to have done so ask it to prove the file to documentation comparison on any random line number. Every time i have done this the model has admitted it did not read any of the file and made up all documentation references. The models do this over and over and over claiming compliance and making up all references every time. The openai models are reading the file and reporting deltas against the documentation.

Claude Model

Not sure / Multiple models

Is this a regression?

No, this never worked

Last Working Version

_No response_

Claude Code Version

opus 4.6

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

VS Code integrated terminal

Additional Information

_No response_