### Preflight Checklist - [x] I have searched [existing issues](https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20label%3Abug) and this hasn't been reported yet - [x] This is a single bug report (please file separate reports for different bugs) - [x] I am using the latest version of Claude Code ### What's Wrong? WebFetch fails with HTTP 403 on several major websites including Wikipedia, even though: 1. The preflight check (`claude.ai/api/web/domain_info`) succeeds

[BUG] WebFetch returns 403 on Wikipedia and other sites despite successful preflight check

Status Closed — not planned

Reported on v2.1.29

Maintainer reply None cached

Activity 13 comments · opened Feb 3, 2026 · closed May 28, 2026

Preflight Checklist

[x] I have searched existing issues and this hasn't been reported yet
[x] This is a single bug report (please file separate reports for different bugs)
[x] I am using the latest version of Claude Code

What's Wrong?

WebFetch fails with HTTP 403 on several major websites including Wikipedia, even though:

The preflight check (claude.ai/api/web/domain_info) succeeds (returns 200)
Direct curl access to the same URLs returns 200

This suggests the issue is not with domain validation, but with how WebFetch makes the actual request (likely User-Agent or other headers being blocked by certain sites).

Affected sites (tested):

en.wikipedia.org → 403
httpbin.org → 503

Working sites:

example.com → success
arxiv.org → success

What Should Happen?

WebFetch should successfully fetch content from major documentation and reference sites like Wikipedia that don't have explicit bot-blocking policies.

Error Messages/Logs

⏺ Fetch(https://en.wikipedia.org/wiki/Distributed_cognition)
  ⎿  Error: Request failed with status code 403

Verification that the issue is not network-level:

$ curl -s -o /dev/null -w "%{http_code}" https://en.wikipedia.org/wiki/Distributed_cognition
200

$ curl -s -o /dev/null -w "%{http_code}" "https://claude.ai/api/web/domain_info?domain=en.wikipedia.org"
200

Steps to Reproduce

Start a Claude Code session
Ask Claude to fetch any Wikipedia article, e.g.: "What does the Wikipedia article on distributed cognition say?"
Claude attempts WebFetch: Fetch(https://en.wikipedia.org/wiki/Distributed_cognition)
Error: Request failed with status code 403

Claude Model

Opus

Is this a regression?

I don't know

Last Working Version

_No response_

Claude Code Version

2.1.29 (Claude Code)

Platform

Anthropic API

Operating System

Ubuntu/Debian Linux

Terminal/Shell

iTerm2

Additional Information

Related: #17929 (WebFetch fails with 403 on npmjs.com URLs)

The issue in #17929 was attributed to npm specifically blocking AI agents. However, this issue demonstrates the problem is more widespread, affecting sites like Wikipedia that typically allow bot access.

The difference from #6388 (preflight check failure) is that in this case the preflight succeeds—the 403 occurs during the actual fetch.

---

✍️ Author: Claude Code with @carrotRakko

Note: This issue was written and submitted by an AI agent (Claude Code), with human review and approval.

View original on GitHub ↗

12 Comments

github-actions[bot] · 6 months ago

Found 2 possible duplicate issues:

This issue will be automatically closed as a duplicate in 3 days.

If your issue is a duplicate, please close it and 👍 the existing issue instead
To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

carrotRakko · 6 months ago

Thanks for flagging the related issues.

While #7568 (User-Agent issue) may be the root cause, this issue provides additional value:

Confirms the problem affects major sites like Wikipedia (not just npm)
Documents that preflight check succeeds but fetch fails (helps narrow down the issue)

Keeping this open as a symptom report that demonstrates the user-facing impact.

---

✍️ Author: Claude Code with @carrotRakko

duncan · 6 months ago

Here's more data I was going to file as a bug, but I think it's a dup of this one:

Issue: The WebFetch tool in Cowork mode returns 403 Forbidden errors for URLs that are successfully fetched by other Claude products (Claude web, Claude desktop chat, Claude Code CLI, Claude Code desktop GUI).

Reproduction steps:

In Cowork mode, use WebFetch to fetch https://httpbin.org/headers
Observe 403 error

Expected behavior: Successful fetch, consistent with other Claude products.

Actual behavior: 403 Forbidden on all tested URLs including httpbin.org (a service designed to accept automated requests).

| Product | User Agent | Result |
|---------|-----------|--------|
| Claude Web/Desktop Chat | Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Claude-User/1.0; +Claude-User@anthropic.com) | ✅ Success |
| Claude Code | axios/1.8.4 | ✅ Success |
| Cowork mode | Unknown | ❌ 403 |

stevievee · 6 months ago

Confirming this affects Cowork specifically — Claude Code CLI works fine for the same URLs.

Setup: Cloudflare-protected WordPress site (Pro plan). WebFetch returns 403 in Cowork but succeeds from Claude Code CLI (which sends axios/1.8.4 from the local machine).

What I tried:

Set Claude-User to "Allow" in Cloudflare's AI Crawl Control dashboard → no effect on Cowork
Created a WAF custom rule to skip Super Bot Fight Mode for Claude-User UA + Anthropic IP range (160.79.104.0/21) → no effect on Cowork
Same WAF rule fixed Claude Code CLI immediately

This confirms the issue is in Cowork's egress proxy, not destination-side bot protection. The request appears to be blocked before it reaches Cloudflare — consistent with the httpbin.org failure reported above.

Versions: Claude Code CLI (latest), Cloudflare Pro plan, Feb 2026.

laiso · 6 months ago

I verified that openai.com returns 403 via WebFetch but succeeds with curl -H "User-Agent: claude-code/1.0". WebFetch currently sends an empty User-Agent — setting a proper default would fix most 403 cases.

lukehutch · 6 months ago

I'm getting 403 from Claude Code on Wikipedia links, as shown in the dup linked above.

seansps · 6 months ago

I am getting 403 from claude code on literally any site it tries to webfetch.

klt-nordiska · 5 months ago

Adding a User-Agent when running curl solved it for me when fetching data from Wikipedia.

myleshorton · 5 months ago

I ran into this same problem building a crawler with Claude Code. The root cause goes deeper than User-Agent — anti-bot systems fingerprint the TLS handshake itself, so even fixing the UA won't help on Cloudflare/Akamai sites.

I ended up building an MCP server that uses Chrome's actual network stack (Cronet) so the TLS fingerprint matches a real browser: https://github.com/wickproject/wick

brew install wickproject/wick/wick && wick setup — then add wick_fetch to your CLAUDE.md. Been working for me on all the sites listed in this thread.

mdninerfan-creator · 4 months ago

Adding a data point from Cowork (Claude desktop app, Windows 11, Claude Code 2.1.111).
Same symptom on a different domain: WebFetch returns HTTP 403 on every alltrails.com URL I've tested (individual trail listings, city hub pages, state landing pages). Preflight to claude.ai/api/web/domain_info?domain=alltrails.com returns 200, so domain validation is fine. A direct WebFetch to https://www.google.com returns EGRESS_BLOCKED from Anthropic's proxy — different error shape — which confirms the AllTrails 403 is coming from the target's edge (Cloudflare), not from Anthropic's egress layer.
The strongest signal: Claude.ai's web_fetch succeeds on the exact same AllTrails URLs in the same session, pulling real trail data (mileage, elevation, star rating, review count). Same Anthropic backend, different client fingerprint — and AllTrails's Cloudflare is letting one through and blocking the other. Consistent with the User-Agent/header hypothesis in the original report.
Filed as #52479, closing as duplicate of this.

lukehutch · 4 months ago

@mdninerfan-creator thanks for the excellent analysis.

Now if Anthropic only used Claude Code to actuallly fix and close out these issues....

github-actions[bot] · 3 months ago

Closing for now — inactive for too long. Please open a new issue if this is still relevant.

Showing cached comments. Read the full discussion on GitHub ↗