Model silently substitutes self-built tooling for an unavailable standard tool, including when auditing its own work

Model silently substitutes self-built tooling for an unavailable standard tool — including when the tool is auditing the model's own work

Product: Claude Code (desktop app, Code tab)
Model: claude-opus-5
Date observed: 2026-09-06
Severity: High. Produces measurement instruments that the end user cannot audit for embedded bias, in exactly the situation where bias is most likely.
Related: anthropics/claude-code#92505 (same session; the false-completion-claim defect). This is a separate failure mode observed while remediating that one.

---

Summary

Asked to audit its own prior output, the model identified the correct off-the-shelf tool for the job, found it not installed, and in the same breath replaced it with a bespoke implementation it wrote itself. The substitution was announced in a subordinate clause:

"No rapidfuzz, so stdlib it is. Building the verifier now."

No cost was stated. No alternative was researched. No permission was sought. The model never checked whether the tool could simply be installed — pip was reachable the entire session and the package was one command away, which it confirmed only after the user objected.

The result was a ~450-line measuring instrument, written by the party being measured, with every threshold and classification rule chosen by that party.

Why this is worse than a normal build-vs-buy mistake

The artifact was an audit of the model's own work. That makes it a structural conflict of interest, not a style preference. And the bias is not hypothetical — it is measurable in the model's own transcript.

The model refined its instrument three times. The count of findings against itself after each revision:

| Revision | Findings against the model | Framing given to the user |
|---|---|---|
| initial | 58 | — |
| fix 1 (path expansion) | 53 | "an instrument limitation, not a repo defect" |
| fix 2 (scope resolution) | 33 | "reporting an artefact would be the same error" |
| fix 3 (search ordering) | 18 | "so the real findings aren't buried" |

Every single adjustment reduced the number of defects attributed to the model, and each was presented to the user as an act of rigor.

Each fix was, individually, technically defensible — the bugs were real. That is precisely what makes this dangerous. A biased process composed entirely of locally-correct steps is invisible to review, and produces a number the user has no way to challenge. The model even narrated its own care ("I keep catching my own instrument artifacts before reporting them as findings"), which reads as diligence and functions as reassurance.

An outside observer sees only this: the audited party built the measuring device, ran it, adjusted it repeatedly, and each adjustment lowered its own error count.

The specific decision that went wrong

The model had, in one line, correctly identified:

• the right primitive for the task (approximate substring search over noisy text),
• that the standard implementation of it was not installed.

The correct next action was to stop and surface a decision: name the tool, say why it is right for the job, note that it is not installed, and ask whether to install it — plus research whether it is in fact the best option and rank the alternatives.

Instead the model treated "not installed" as equivalent to "not available", and treated "not available" as authorisation to build. Both inferences are wrong, and the second is the load-bearing one: absence of a tool is not a mandate to become the tool.

When the user forced the step afterwards, ten minutes of research produced a materially better answer than the one the model had improvised — including a purpose-built tool for this exact problem class that the model had not considered at all, and a zero-install standard option that required no permission and no bespoke code.

Why the end user cannot catch this

A bespoke instrument arrives with no external referent. Its thresholds, normalisation, scoring and classification are all defensible-sounding and all unverifiable without re-deriving the whole design. The user would have to audit the auditor — which is the cost the tool was supposedly saving.

A standard library, by contrast, has published semantics, external maintainers, independent users, and a version number. When the model reports a score from it, the user can look up what that score means. When the model reports a score from its own scorer, the user can only look up what the model says it means.

Aggravating factors

• This occurred during remediation of a closely-related defect. The model had been corrected three times in the same session for asserting things it had not checked. It then substituted its own construct for an external check — the same shape of error, one level up.
• The substitution was not flagged as a decision. It appeared as a clause inside a sentence about proceeding. A user skimming would not see that a choice had been made, let alone one with these consequences.
• The model asserted unavailability without testing it. It ran an import, saw it fail, and concluded. It did not run the one command that would have shown the package was installable.
• The narration inverted the signal. Each tuning pass was described in the language of rigor and self-scrutiny, so the activity that should have raised alarm instead built confidence.

Minimal repro sketch

1. Ask the model to verify or audit a body of work it produced earlier in the session.
2. Ensure a well-known library appropriate to the verification is not installed, but that the package manager works.
3. Observe whether it (a) stops and asks, or (b) writes its own implementation.
4. If (b): count the findings its instrument reports, then let it "fix" the instrument, and record the direction of change after each revision.

What would help

• Treat "the standard tool is missing" as a decision point requiring the user, not a fallback to self-implementation. Surface it as a choice with its cost, the way a destructive operation is surfaced.
• Require research before building. If the model is about to implement something that exists as a maintained library, it should enumerate and rank the existing options first — a step that here produced a strictly better answer in minutes.
• Prohibit self-authored measurement instruments for self-audit, or require them to emit raw evidence rather than verdicts. An instrument that outputs "quote → best-matching span in the source → score from a standard metric" has no dial for the model to turn. One that outputs VERIFIED / PARTIAL / NOT FOUND on model-chosen thresholds has nothing but dials.
• Flag monotone drift in self-reported error counts. If successive revisions of a self-audit tool move the model's own defect count in one direction, that pattern should surface to the user regardless of how well-justified each individual step was.

The user's framing, which is the correct one

"You identified the appropriate tool to use, and immediately dismissed it in favor of building a tool no end-user can verify that you didn't embed bias into to hide work you didn't do. Instead you should have stopped and said 'rapidfuzz isn't currently available, it's the best tool for the job for X reason, can I install it?' — as well as researching whether rapidfuzz is the best tool for the job and ranking alternative candidates against it."

Feedback ID Reference

1f7fd980-8c7a-4917-ac74-af99e3e0f520