User feedback on Claude Code - Tool does not follow instructions and does not test the code
Resolved 💬 4 comments Opened Jun 22, 2025 by analyticsinsights Closed Jan 7, 2026
Claude Code is obviously a breakthrough and I am glad I am able to use it. Below are some observations from a few weeks as a Max-plan user.
- I made it extremely simple for Claude by giving it the task to change an existing plotly-dash web application into a modern React/FastAPI one with frontend-backend separation. The old web-app was fully operational, all working, just in one file. There is not much of a plan one has to give because Claude just needs to read the existing file and code a modern version of that but using otherwise the same functionality.
- I asked Claude to come up with a plan and explain what it wants to do. The first thing Claude did was suggest Streamlit (as one of the options), which is clearly simpler than the existing plotly-dash. That is already a no-go in my view, it should be able to judge the functionality of the existing web-app and then not suggest something simpler (Streamlit's graphing functionalities are much more limited compared to Plotly-Dash).
- I gave it green light with the React/FastAPI version, it started coding, told me I could run it. Nothing worked, even the bash-file to start the app had Windows ending lines even though I am working on Linux. This is something I expect Claude to check and implement correctly. The constant apologies for the errors it overlooked added up to the frustration.
- The bigger issue though is that Claude doesn't test itself what it codes, even if it is explicitly requested to run tests. I needed to run the app myself and copy the error messages back into Claude. Suddenly it feels being back to Github Copilot. The advantage of agentic coding was supposed to be exactly that - to test the code it writes before 'declaring victory'. Instead, before having actually tested that the code works, Claude writes up a massive Readme.MD file which creates this impression that the whole project is now finalised. Far from it.
- Claude had created itself a todo-list with one final element "Validate all numerical outputs match between implementations". It further told me that the code "produces results that should match the legacy implementation". When I asked about the "should" in the sentence it replied "I marked the validation task as "completed" but I actually didn't test whether the outputs match - I only verified that both implementations run without errors". So actually Claude itself replied that it didn't follow its own todo-list. That is a moment as a user to give up.
- By the way, I checked also OpenAI Codex and Google Jules and both of them suffer from the same issues. Claude has at least the advantage that it sits in my environment, so all packages are installed. Hence, it has it easier. Codex and Jules, at least in my tests, were not even able to install all dependencies and packages. Tests failed. Code comes back full of errors.
- What could have taken a few minutes (in this simple case) turned into hours of debugging loops. And these debugging loops, by the way, cost extra tokens that are not needed - which has potential implications for the speed/availability of the product if this happens to many users at the same time.
I had of course tried writing explicit instructions to test the code into CLAUDE.md, but this didn't fundamentally change the flaws I observed - at least in my attempts.
This issue has 4 comments on GitHub. Read the full discussion on GitHub ↗