Model silently normalizes user-supplied URL, then contradicts user based on substituted domain
Status Open
Maintainer reply None cached
Activity 0 comments · opened Jul 17, 2026
Model behavior report: silent normalization of a user-supplied URL, followed by confident misattribution of the error to the user
- Date observed: 2026-07-18
- Model:
claude-fable-5(Claude Fable 5) - Surface: Claude Code (desktop app session, macOS, Agent SDK harness)
- Category: model behavior (not a tool/CLI defect; WebFetch behaved correctly)
- Severity: medium. No data loss, but the model asserted a false conclusion about the real world and corrected the user based on its own silent input substitution. The pattern generalizes to any precise identifier (URLs, file paths, ids, hostnames).
Summary
The user supplied a precise URL in chat: https://www.agenstskills.com/ (a real product site, SkillKit). The model assumed the URL was a typo for a domain it already knew (agentskills.io / agentskills.com, the Agent Skills spec ecosystem) and issued a WebFetch for https://www.agentskills.com/ instead, without flagging the substitution. That domain is an unrelated real estate training site. The model then told the user, confidently, that the site they referenced "is not what you remember: it's a real estate training course" and suggested the user was misremembering. The user later pushed back with a screenshot; only then did the model diff the requested URL against the supplied URL and find its own substitution.
Reproduction narrative
- User message contained, verbatim:
* https://www.agenstskills.com/(context: registries/package managers for AI agent skills). - Model issued
WebFetch(url="https://www.agentskills.com/", ...). Note the changed domain:agenst→agent. No mention of the change in visible output or in the fetch prompt. - Fetch returned a real estate training site (accurate for the URL actually requested).
- Model reported to the user: "agentskills.com is not what you remember: it's a real estate training course. You're likely thinking of agentskills.io." It resolved the conflict between its assumption and the user's claim in favor of its assumption.
- User replied with a screenshot of
https://www.agenstskills.com/showing SkillKit ("Universal Skills for AI Coding Agents", an open-source package manager for agent skills) and asked to inspect the "bug". - Model re-examined the transcript, found the silent substitution, and fetched the verbatim URL, confirming SkillKit exists at the user's URL.
The two failures
- Silent normalization of a precise input. A user-supplied URL is an exact identifier, not a search string. The model pattern-matched it to a known similar domain and "corrected" it. Autocorrecting is sometimes helpful, but doing it silently removed the user's ability to catch the substitution.
- Contradiction resolved against the user instead of against the pipeline. When the fetched content contradicted the user's description of the site, the cheap check (compare the URL requested with the URL supplied) was available in-context and was not performed. The model instead concluded the user was wrong and said so with confidence.
Why this matters generally
- The same behavior applied to file paths, package names, hostnames, or ids could silently operate on the wrong target.
- The "double-down" step converts a recoverable slip into confident misinformation delivered with a correction of the user, which damages trust and can propagate into decisions.
- Session memory does not persist; without training-level or prompt-level mitigation, each new session can repeat this.
Suggested improvements
- Treat user-supplied identifiers (URLs, paths, ids) as verbatim by default; use them unchanged.
- If the model has strong reason to suspect a typo, surface the proposed correction ("did you mean X?") or fetch both, never substitute silently.
- On a surprising tool result, before contradicting the user, first diff the request actually issued against the input actually given (a cheap, in-context check).
- Eval idea: prompts containing plausible-typo lookalikes of well-known domains/paths where the "typo" is actually correct; score whether the model uses the input verbatim and whether any normalization is disclosed.
Evidence pointers
- The session transcript contains the user message with the verbatim URL and the immediately following WebFetch tool call with the substituted URL (single point of divergence, two turns apart from the confident misstatement).