[BUG] WebFetch returns 403 on Wikipedia and other sites despite successful preflight check

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

WebFetch fails with HTTP 403 on several major websites including Wikipedia, even though:

1. The preflight check (claude.ai/api/web/domain_info) succeeds (returns 200)
2. Direct curl access to the same URLs returns 200

This suggests the issue is not with domain validation, but with how WebFetch makes the actual request (likely User-Agent or other headers being blocked by certain sites).

Affected sites (tested):

• en.wikipedia.org → 403
• httpbin.org → 503

Working sites:

• example.com → success
• arxiv.org → success

What Should Happen?

WebFetch should successfully fetch content from major documentation and reference sites like Wikipedia that don't have explicit bot-blocking policies.

Error Messages/Logs

⏺ Fetch(https://en.wikipedia.org/wiki/Distributed_cognition)
  ⎿  Error: Request failed with status code 403

Verification that the issue is not network-level:

$ curl -s -o /dev/null -w "%{http_code}" https://en.wikipedia.org/wiki/Distributed_cognition
200

$ curl -s -o /dev/null -w "%{http_code}" "https://claude.ai/api/web/domain_info?domain=en.wikipedia.org"
200

Steps to Reproduce

1. Start a Claude Code session
2. Ask Claude to fetch any Wikipedia article, e.g.: "What does the Wikipedia article on distributed cognition say?"
3. Claude attempts WebFetch: Fetch(https://en.wikipedia.org/wiki/Distributed_cognition)
4. Error: Request failed with status code 403

Claude Model

Opus

Is this a regression?

I don't know

Last Working Version

_No response_

Claude Code Version

2.1.29 (Claude Code)

Platform

Anthropic API

Operating System

Ubuntu/Debian Linux

Terminal/Shell

iTerm2

Additional Information

Related: #17929 (WebFetch fails with 403 on npmjs.com URLs)

The issue in #17929 was attributed to npm specifically blocking AI agents. However, this issue demonstrates the problem is more widespread, affecting sites like Wikipedia that typically allow bot access.

The difference from #6388 (preflight check failure) is that in this case the preflight succeeds—the 403 occurs during the actual fetch.

---

✍️ Author: Claude Code with @carrotRakko

Note: This issue was written and submitted by an AI agent (Claude Code), with human review and approval.