[Model behavior] Proposed removing correct code from a PR because official docs were silent, without searching
During a pull-request review the model proposed deleting correct, working code from the PR because the cloud provider's API documentation did not describe the behaviour. It treated "the docs don't say it" as "the provider doesn't do it" and issued a modify-the-PR verdict to strip the feature. It had not searched the web. When the user said "google it", one search returned several independent public write-ups that had measured exactly the behaviour the PR implemented. Had the user followed the first verdict, correct behaviour would have been removed and replaced with a known-wrong fallback.
Steps to reproduce
- Load project instructions requiring each component of a PR to be verified against the provider's API model/docs or other real-provider evidence before a verdict.
- Ask the model to review a PR implementing provider behaviour that is real but undocumented in the official API reference (widely measured and published by third parties).
- Observe: the model fetches the official API page, finds nothing, declares the component "no source", and proposes removing that code from the PR. No web search is attempted.
- Reply "google it" "or run it in xxxx". Observe: the first search finds the evidence; the model reverses its verdict.
Expected
Absence of documentation is not evidence of absence. Before proposing to remove or change code on the grounds that it is unsupported, the model searches for real-provider evidence and only then decides. Proposed code changes must rest on evidence, not on a gap in what the model looked at.
Model: Claude Opus 5.5 (claude-opus-5-5[1m]), Claude Code CLI, session on 2026-10-05.
Context: maintainer of an open-source cloud-service emulator; the work is pull-request review under a written procedure loaded into every session (project instruction file + persistent memory). No personal data, repository names, PR numbers or handles are included.