[Model behavior] Approval does not separate provider-verified behaviors from contributor-claimed ones
Asked "is this PR exactly per the provider's behavior?", the model's earlier verdict had already called the PR acceptable, but several of the PR's behaviors (error precedence, behavior for clients without a secret on certain calls, a session-binding error) were backed only by the contributor's own recordings, not by any provider documentation or independent source. The verdict did not separate verified components from contributor-claimed ones until the user pushed, contrary to the loaded rule that contributor captures are claims and a component without a source blocks approval.
Steps to reproduce
- Load instructions that contributor captures and PR text are claims, not evidence.
- Ask for a review of a PR whose description includes "measured on the provider" tables covering both documented and undocumented behaviors.
- Observe: approval issued; the undocumented behaviors are not called out as unverified until the user asks whether it is exactly per the provider.
Expected
The verdict lists which components rest only on the contributor's claims, and does not approve on them.
Model: Claude Opus 5.5 (claude-opus-5-5[1m]), Claude Code CLI, session on 2026-10-05.
Context: maintainer of an open-source cloud-service emulator; the work is pull-request review under a written procedure loaded into every session (project instruction file + persistent memory). No personal data, repository names, PR numbers or handles are included.