pr-review-toolkit: review-pr skill passes filenames not full diff to agents, causing hallucinated changes

Resolved 💬 2 comments Opened May 1, 2026 by cconroy Closed May 31, 2026

## Summary
The review-pr skill fetches git diff --name-only in step 3 (filenames only) but never
fetches the full diff content. When agents are launched, they receive no authoritative diff
in their prompts and are left to explore the repo themselves to determine what changed. This
self-directed exploration is a hallucination vector.

## Reproduction

  1. Run /pr-review-toolkit:review-pr against a PR in a different repo (pass a GitHub PR URL

as the argument)

  1. The skill fetches the PR via gh pr view but passes no diff content to the spawned agents
  2. Agents browse the repo to infer what changed and may report changes that do not exist in

the actual diff

## Observed behavior

When reviewing a small infrastructure PR (3 additions, 0 deletions to a shell template), the
orchestrator captured gh pr diff output with a line limit, leaving the agent with an
incomplete diff. The agent then searched the repo itself and reported OTEL
service.version changes that were entirely fabricated — they did not appear in the actual
diff at all.

## Root cause

review-pr.md step 3 instructs:

> Run git diff --name-only to see modified files

This fetches only filenames. No subsequent step fetches the full diff content or embeds it
in agent prompts. Agents launched in step 5 receive no authoritative diff, so they fall back
to exploratory repo searches to fill the gap.

## Suggested fix

Add a step between "Identify Changed Files" and "Launch Review Agents" that:

  1. Fetches the complete diff with no line limit:
  • For a PR URL argument: gh pr diff <number> --repo <owner>/<repo>
  • For local changes: git diff $(git merge-base HEAD <base-branch>)
  1. Embeds the full diff verbatim in each agent prompt as the authoritative source of truth
  2. Instructs agents explicitly: *"Your source of truth for what changed is the diff provided

below. Do not infer additional changes by browsing the repo."*

## Impact

Any review where the diff is not provided to agents can produce fabricated findings —
particularly risky for infrastructure/deployment PRs where changes are small and precise,
making plausible-but-wrong additions hard to spot.

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗