WebFetch fabricates data from paginated JSON/API pages without signaling uncertainty

Summary

WebFetch returned fabricated data when summarizing a paginated JSON/API page, with no uncertainty signal. It invented Docker image tags that do not exist, which directly led to incorrect, action-guiding output during a live database upgrade.

Severity

High — produced confidently-wrong output that was acted upon (recommended a Docker image tag that does not exist), causing repeated not found failures.

What happened

WebFetch was used to enumerate Docker Hub image tags from https://hub.docker.com/v2/repositories/library/sonarqube/tags. The tool confidently returned a list of float tags including 26-community, 25-community, and lts-community.

Acting on that output, sonarqube:26-community was recommended as a pinned image. It does not exist:

$ docker manifest inspect sonarqube:26-community
no such manifest: docker.io/library/sonarqube:26-community

$ for t in 26-community 26.5-community community lts-community 25-community; do \
    printf "%-18s " "$t:"; docker manifest inspect sonarqube:$t >/dev/null 2>&1 \
    && echo EXISTS || echo "not found"; done
26-community:      not found
26.5-community:    not found
community:         EXISTS
lts-community:     EXISTS
25-community:      not found

Only community and lts-community floats actually exist. The user hit not found errors twice during a production-data upgrade before deterministic verification surfaced the truth.

Root cause (as understood)

WebFetch summarizes fetched content with a separate fast model rather than returning parsed/raw content. For large, paginated JSON (Docker Hub returns 100 tags/page), the summarizer confabulated plausible-but-nonexistent entries instead of reporting only what was present — and gave no uncertainty signal distinguishing "read from the page" from "inferred."

Requested improvements

1. For structured responses (JSON/API), expose a raw/parsed mode that does not route through a summarizer model.
2. When content is paginated or truncated, WebFetch should explicitly state the view was partial rather than silently filling gaps.
3. Calibrate the summarizer to refuse enumeration ("I can only see X of N items") instead of inventing list members.
4. Steer exact-existence questions (does tag/version/file X exist) toward deterministic tools in guidance.

Workaround

Verify existence with deterministic tools (docker manifest inspect, gh api, curl | jq) instead of WebFetch.