[FEATURE] deep-research workflow: ask blocked-source fallback before fan-out

Resolved 💬 1 comment Opened Jun 2, 2026 by cameronsjo Closed Jul 6, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

The built-in deep-research workflow fetches sources during its Fetch phase, and bot-blocked sources (retailers, review sites, paywalls) silently yield zero claims and get scored "unreliable." The user has no input into what happens to those sources — the workflow just drops them and moves on.

In a recent product-research run (111 agents, 28 sources fetched), 5 sources were bot-blocked: two Best Buy product pages, a Home Depot page, a Whirlpool article, and a Consumer Reports ratings page. Two of those five turned out to contain decision-changing information (one ruled out a recommended accessory; another contained the highest-customer-rated product in the entire research). Recovery only happened because the user noticed the gap mid-session and improvised a manual pull.

The fallback for blocked sources is a user decision, but the workflow never asks.

Proposed Solution

In the Scope phase (before fanning out searches), the deep-research workflow should establish a blocked-source fallback via AskUserQuestion:

  1. Open blocked links in default browser — sane only (apply a URL sanity filter before auto-opening)
  2. Open all in browser — no filter, user accepts the risk
  3. Save the blocked-link list to a markdown file for manual pull later
  4. Skip blocked sources — note the gap and move on

The sanity filter test: "does this domain have any business being in this research?" — auto-open HTTPS URLs whose domain matches the research subject (manufacturer/retailer/publication surfaced by a reputable search); never auto-open URL shorteners, IP-literal URLs, punycode/homoglyph domains, or off-context TLDs.

Regardless of the chosen fallback, the final synthesis should list blocked sources alongside what each would have answered, so the user can judge whether the gaps matter.

Alternative Solutions

Current workarounds:

  • A user-level rule that fronts any link-heavy research with this question (works, but only for users who know to write it)
  • Mid-flight improvisation — the user notices blocked sources in the output and asks for them ("take note, I'll pull them manually")
  • A post-hoc "Blocked Sources" section in the report listing URLs for manual retrieval

All three put the burden on the user to notice the gap. The decision point belongs before the fan-out, not after.

Priority

Medium - Would be very helpful

Feature Category

Other

Use Case Example

Example scenario:

  1. User asks for product research (appliances, in this case) via the deep-research workflow
  2. Workflow fetches 28 sources; 5 are bot-blocked (Best Buy ×2, Home Depot, Whirlpool, Consumer Reports)
  3. Workflow completes with those 5 scored "unreliable" / zero claims — no flag that they were access failures rather than low-quality sources
  4. With this feature: the workflow would have asked up front, opened the 5 blocked pages in the user's browser (all pass the sanity filter — they're major US retailers), and the user could clip them for ingestion while the workflow ran
  5. Without it: the user spotted the gap, manually pulled all 5 pages, and the findings materially changed the research output — one recommendation reversed, one new top contender added

Additional Context

  • The distinction between "this source is low quality" and "this source could not be accessed" matters for research integrity — they're currently conflated in the workflow's source-quality scoring
  • The browser-open fallback pairs naturally with how users already recover from blocked pages (open → clip extension → markdown → feed back to Claude)

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗