[FEATURE] retrieve the search index's cached/extracted copy of a page

Status Open
Maintainer reply None cached
Activity 0 comments · opened Jul 20, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

Summary

WebSearch demonstrably holds extracted body text from pages it indexes — it
quotes them, including passages absent from titles and URL slugs. There is
currently no way to ask for that held text. The only retrieval tool,
WebFetch, requires live DNS and a reachable origin, which is precisely what
is unavailable in the cases where a cached copy matters most.

Request: a tool (or a WebFetch flag, e.g. allow_cached: true) that returns
the index's stored copy of a URL when the live origin is unreachable.

Known counterarguments

Worth stating plainly, since they likely explain why this does not already
exist:

  • Licensing. Index content may be licensed for generating answers but not

for redistribution verbatim. This is probably the binding constraint, and it
may be dispositive.

  • Copyright. Serving stored copies of third-party pages raises questions

distinct from summarizing them.

  • Staleness. Cached content presented without clear labeling is actively

harmful during a live incident, where the whole question is what is true
now.

  • Publisher expectations. Sites may object to cache exposure; Google

retiring its cache: operator is the relevant precedent.

A narrower version may thread these: expose only the extract already surfaced
in summaries, always labeled with a timestamp — no new content leaves the
index, it merely becomes directly quotable and attributable rather than
laundered through a paraphrase.

Proposed Solution

Proposed shape

WebFetch(url, prompt, allow_cached=true)
-> live fetch; on failure, index's stored copy
-> response labels which was returned, plus retrieval timestamp

Or a distinct WebCache(url). Requirements that matter:

  • Explicit provenance. Must state clearly that content is cached and when

it was retrieved, so stale data is never mistaken for live.

  • Partial is fine. Even the extract already used to generate summaries is

far better than nothing, provided its partiality is labeled.

  • Failure should be honest. "Not in cache" must be distinguishable from

"cache not consulted."

Alternative Solutions

_No response_

Priority

Medium - Would be very helpful

Feature Category

CLI commands and flags

Use Case Example

Concrete motivating case

2026-07-20. Investigating a hosting provider network outage. All details below
are public record — BGP data from RIPEstat, and a public incident post.

Both of the provider's domains were unresolvable: all three authoritative
nameservers unreachable, both zones SERVFAIL'ing globally. The provider's own
incident post — the single most relevant document for the diagnosis — was
therefore unreachable, despite being indexed.

Every retrieval path failed:

| Path | Result |
|---|---|
| WebFetch on the URL | getaddrinfo ETIMEOUT — dies at DNS |
| Wayback Machine | ZERO snapshots of that blog path, any year |
| DuckDuckGo | no results; site: query empty |
| An independent index (Mojeek) | has the domain, not the post |
| Marginalia | nothing |
| Bing | inconclusive (bot-blocked) |
| Direct fetch by IP | traffic dropped upstream during mitigation |

Yet WebSearch returned substantive body text across four separate queries,
including operator voice — a first-person phrase ("every subnet we
advertise") appearing in neither the title nor the slug. The index plainly
held more of the document than it surfaces.

Asked directly for mitigation details, the search layer replied that results
"don't contain detailed information about specific lessons learned" and that
the post "appears to contain more information than what was captured." The
tool itself reported holding a partial extract, with no mechanism to request
the remainder.

Why this matters beyond one outage

The failure mode is systematic, not incidental:

  1. Outage diagnosis is the worst case. The definitive source is hosted on

the infrastructure that is down. Status pages, incident posts, and provider
blogs are exactly the documents that vanish when needed.

  1. Archives don't cover the gap. Archive.org had never crawled that blog

path in any year, and could not crawl it during the outage — the site was
unresolvable. Recent pages are the least likely to be archived and the most
likely to be urgent.

  1. Verification is blocked. The incident's stated cause was reasonably

challenged during the investigation, and the source document could not be
produced. The claim rested on a single index reachable only through the
assistant's tooling, which the person asking cannot query independently.
A cached-copy tool would have converted an unverifiable assertion into a
checkable one.

  1. It drives error. Unable to read the post, the assistant leaned on

inference and got the central finding wrong: it queried an inferred CIDR
(a /22 that was not an actual announcement), read the resulting zero-peer
response as "prefix withdrawn," and reported a BGP withdrawal. Routing was
in fact up — 20 prefixes announced, 324/325 peers — and the real mechanism
was upstream filtering during mitigation. The primary document would
likely have stated this directly.

Note the shape of this error: querying an exact CIDR that is not itself an
announcement returns zeros indistinguishable from a genuine withdrawal.
That is a sharp edge worth knowing about independently of this request.

Additional Context

_No response_

View original on GitHub ↗