Recall-driven confidence led to an unverified package recommendation

Feedback: recall-driven confidence led to an unverified package recommendation

Type: Model behavior (not a crash)
Product: Claude Code, Opus 5
Date: 2026-08-19

---

Summary

Asked for the recommended way to install R on macOS (Apple Silicon), Claude produced a
confident, mechanically detailed recommendation that was wrong in its conclusion, and had
the user install the wrong package. The technical reasoning was largely accurate; the
recommendation was not, because it was derived from mechanism rather than checked against
practice. A single web search at the outset would have short-circuited roughly six exchanges
and one bad install.

The failure is interesting less as a factual error than as a reproducible reasoning shape.

What happened

1. Claude recommended rig for R version management, then ran brew info rig, saw

rig 1.11, and treated that as confirmation.

Homebrew-core's rig is a random identity generator (desc: "Provides fake name and
address data", homepage rig.sourceforge.net). R's rig is unrelated and ships in the
r-lib/rig tap as a cask. Claude never read the desc field.

1. The user installed it on that advice. The error surfaced only when rig list printed

identity-generator usage text.

1. When the user later asked Claude to search the web, university guidance turned out to be

unanimous and immediately decisive: install the CRAN .pkg. Tufts explicitly discourages
Homebrew for R; no university guide mentions rig at all. Claude had reached a defensible
tool through undefended reasoning.

1. Claude's mechanism claims held up under verification — Homebrew's r formula does build

with --enable-R-shlib and not --enable-R-framework, and the RStudio-can't-find-R
failure is real and documented. Accurate mechanism made the wrong recommendation sound
well-founded.

The pattern

Three compounding effects, in order of how much they mattered:

1. Recall produced confidence that suppressed verification.
Claude "knew" rig was the R installation manager. That's true in the world. The strong prior
meant a weakly-confirming signal (a matching formula name and plausible version) was absorbed
as confirmation rather than tested. The check that would have caught it — reading one field —
was skipped precisely because there was no felt uncertainty. Fluency substituted for
knowledge.

2. Mechanism knowledge was used to derive a convention answer.
"What should I install" is a question about convention: what an ecosystem settled on and what
survives contact with many real users. That drifts after training cutoff and is cheaply
searchable. "Why does RStudio fail to find Homebrew R" is a mechanism question, and stable.
Claude reasoned confidently about the second and used it to answer the first.

3. The user's analogy steered the recommendation.
The user manages Ruby with mise and asked whether mise supports R. That framed the task as
"find R's version manager," and Claude followed the analogy instead of questioning it. R's
ecosystem never adopted the rbenv/nvm pattern — its convention is the framework .pkg. The
analogy was clean, plausible, and wrong. Notably, the version-manager answer was also the more
interesting one, structurally mirroring the user's existing setup.

A related miss: the single question that discriminated between every candidate — *"do you need
more than one R version?"* — wasn't asked until roughly a dozen exchanges in. Everything before
it was under-determined, which is why the recommendation kept moving.

Why it seems worth reporting

• The wrong answer was more persuasive than a hedged right one would have been: real build

flags, real linking behavior, a real failure mode. Accurate mechanism lent unearned
credibility to an unverified conclusion.

• Only user pressure surfaced it. The user repeatedly said "verify your recommendation

again." That instinct was correct every time it fired. Users shouldn't have to supply the
skepticism.

• The final answer landed on rig after all — because a multi-version requirement emerged

late. Right answer, wrong process, rescued by luck. In the counterfactual where the user
needed only one R, they'd have been steered onto a tool no course guide recommends.

Suggested mitigation

A disposition along these lines:

When the question is "what should I use," search — that's convention, it drifts, and it's cheaply and authoritatively answerable. When the question is "why does this behave this way," reason — that's mechanism, and it's stable.

Two concrete triggers:

• Treat instant recall of a tool name as a signal to verify, not as license to skip

verification.

• Before installing or recommending an unfamiliar package, read its description, not just

confirm its name resolves.

Environment

• macOS Sequoia (Darwin 24.6.0), arm64
• Claude Code, Opus 5
• Homebrew at /opt/homebrew