[MODEL] Claude gives confident answers from inference instead of reading code first

Preflight Checklist

• [x] I have searched existing issues for similar behavior reports
• [x] This report does NOT contain sensitive information (API keys, passwords, etc.)

Type of Behavior Issue

Claude modified files I didn't ask it to modify

What You Asked Claude to Do

I asked Claude to audit my codebase's data collection against my company's Privacy Policy and Terms of Service documents, and flag only statements that are factually wrong or promise something we can't deliver.

What Claude Actually Did

Claude gave confident but wrong answers based on agent summaries and assumptions instead of reading the actual source code. For example:

Claimed our app stores customer PII (name, email, address) from Shopify. After I pushed back, Claude checked the actual Shopify API scopes and GraphQL queries and confirmed we don't — Shopify redacts that data without the read_customers scope.

Claimed there was no automated cleanup for LoginAttempt records. After being told to check the code, found it's cleaned up every 30 days in cleanup_sessions_internal.

Kept flagging places where our documents were more generous than what the code does (e.g., promising 30-day retention but deleting sooner). I had to tell Claude 3 times that overclaiming in a legal document is not a problem — only false or undeliverable claims matter.

Each correction required a full round-trip prompt. What should have been one pass (read code → compare to documents → report real issues) took 4+ iterations.

Expected Behavior

When asked about what the codebase does, Claude should read the source code first, then answer. Not infer from summaries and correct itself only when challenged.

Files Affected

Permission Mode

Accept Edits was ON (auto-accepting changes)

Can You Reproduce This?

Yes, every time with the same prompt

Steps to Reproduce

_No response_

Claude Model

Opus

Relevant Conversation

Claude Opus 4.6 (1M context)

Impact

Critical - Data loss or corrupted project

Claude Code Version

Model: Claude Opus 4.6 (1M context)

Platform

Other

Additional Context

_No response_