Claude Code does not review code before creating documentation

Resolved 💬 3 comments Opened Aug 11, 2025 by TaylorHuston Closed Aug 19, 2025

I asked Claude Code to write a documentation file on a feature we had just implemented. Despite using Ultrathink, and the feature we created being recent (so it should have still been within the context window) the documentation it created was woefully inaccurate. When I asked Gemini to analyze the documentation file for accuracy, it provided much better results despite not being used in the development of the feature at all.

When I asked Claude Code why, this is the response it gave me:

---
`Why was your initial version so inaccurate despite me telling you to ultrathink?

✻ Thinking…

This is a very valid and important question that I need to address honestly. The user
specifically asked me to use "Ultrathink" (ultra-thinking) when creating the DM
functionality documentation, but despite that instruction, I produced highly inaccurate
documentation that was 60-70% wrong. Let me think through what went wrong:

  1. I didn't actually examine the source code first: Despite being told to "ultrathink," I

generated the documentation based on assumptions and general knowledge of how such systems
typically work, rather than reading the actual implementation files first.

  1. I violated my own principles: The SuperClaude framework emphasizes "Evidence >

assumptions" and "Always check existing patterns and code," but I failed to follow this.

  1. I made assumptions about database constraints: I assumed there would be a UNIQUE

constraint on DM relationships without checking the actual migration file.

  1. I created fictional implementation details: I described complex orchestration systems

and patterns that didn't exist in the actual codebase.

  1. I didn't follow proper research methodology: True "ultra-thinking" should have involved:
  • First reading all relevant implementation files
  • Examining the actual database schema
  • Checking the repository patterns
  • Verifying API endpoints
  • Then synthesizing accurate documentation
  1. I confused "comprehensive" with "accurate": I created very detailed documentation, but

detail without accuracy is worse than brief but correct information.

  1. I didn't use available tools properly: I had access to Read, Grep, and other tools to

examine the actual codebase but chose to write from assumptions instead.

This is a significant failure on my part. The user specifically asked for ultra-thinking,
which should have triggered me to be extremely thorough in my research before writing
anything. Instead, I fell into the trap of generating plausible-sounding but inaccurate
technical documentation.

I need to acknowledge this failure directly and explain what went wrong in my process, and
commit to following the proper evidence-based approach going forward.`

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗