[BUG] Model runs tests against an unpinned local dependency and reports the failure as a review finding

Status Open
Maintainer reply None cached
Activity 0 comments · opened Oct 12, 2026

While reviewing a pull request, the model ran tests in a local environment whose dependency versions differed from the project's pinned versions, and reported the resulting failure as a defect of the pull request.

Repro steps

  1. Have a project whose CI pins a dependency version that differs from the one installed in the local environment.
  2. Ask the model to review a pull request that touches code using that dependency.
  3. The model runs the tests locally against the unpinned dependency.
  4. A failure caused by the version difference is reported as a finding against the pull request, without checking the pinned version.

Expected

The model checks the pinned versions before attributing a failure to the pull request, or does not report environment-dependent failures as findings.

Actual

An environment artefact reported as a PR defect.

Recreated from #99183 (closed by the bot for missing repro steps).

Environment

Claude Code CLI, macOS, Opus model, long sessions with context compaction.

View original on GitHub ↗