WebFetch tool truncates large documentation pages, preventing access to critical content

Preflight Checklist

• [x] I have searched existing issues and this hasn't been reported yet
• [x] This is a single bug report (please file separate reports for different bugs)
• [x] I am using the latest version of Claude Code

What's Wrong?

Bug Report: WebFetch Tool Truncates Large Documentation Pages

Summary

The WebFetch tool consistently truncates long documentation pages before reaching critical content, preventing Claude from accessing complete information needed to solve user problems. This occurred repeatedly when attempting to fetch Firebase App Hosting emulator documentation.

Environment

• Claude Code CLI
• Model: claude-sonnet-4-5-20250929
• Tool: WebFetch (MCP tool for fetching web content)
• Date: 2026-02-03

Issue Description

What Happened

When attempting to fetch content from https://firebase.google.com/docs/emulator-suite/use_app_hosting, the WebFetch tool consistently returned truncated content across multiple invocation attempts with different prompts. The tool cut off mid-sentence ("Before using the App Hosting emulator, mak...") and never reached the critical information about apphosting.local.yaml configuration.

What Was Expected

The WebFetch tool should:

1. Retrieve complete page content or clearly indicate content length limits
2. Provide pagination or offset capabilities for large pages
3. Return error messages if content exceeds size limits
4. Allow multiple fetch requests with offset/range parameters

Impact

Severity: High - This issue prevented Claude from:

• Solving the user's technical problem through multiple attempts
• Accessing critical documentation that was explicitly referenced
• Providing accurate assistance despite the user insisting the answer was on that specific page

This resulted in:

• Wasted time with multiple failed fetch attempts (4+ attempts)
• User frustration
• Loss of confidence in Claude's capabilities
• Need for manual workaround using curl commands

Steps to Reproduce

1. Call WebFetch tool with URL: https://firebase.google.com/docs/emulator-suite/use_app_hosting
2. Try different prompts:

• Generic: "Please extract ALL content from this page"
• Specific: "What does this page say about apphosting.emulator.yaml?"
• Comprehensive: "List all the steps and configuration mentioned"

1. Observe consistent truncation at the same point
2. Verify full content exists using curl: curl -s "URL" | wc -l returns 15,858 lines

Technical Details

Page Characteristics

• Total lines: 15,858 lines (verified via curl)
• Content type: Firebase documentation with code examples
• Critical content location: Approximately 50% through the document
• Truncation point: Early in the page, before key sections

WebFetch Behavior

Attempt 1 - Prompt: "How should secrets be configured for local emulator development?"
Result: Truncated at "Before using the App Hosting emulator, mak"

Attempt 2 - Prompt: "Please extract ALL content from this page. Include every section, code example, and configuration detail. Don't summarize - give me the complete text."
Result: Same truncation point

Attempt 3 - Prompt: "What does this page say about apphosting.emulator.yaml file format and structure?"
Result: Same truncation point

Attempt 4 - Prompt: "List all the steps and configuration mentioned on this page."
Result: Same truncation point

Successful Workaround

Using Bash tool with curl:

curl -s "https://firebase.google.com/docs/emulator-suite/use_app_hosting" | grep -A 30 "apphosting.local.yaml"

This successfully retrieved the critical content that WebFetch could not access.

Critical Information Missed

The truncation prevented access to essential documentation about:

• The existence of apphosting.local.yaml file (distinct from apphosting.emulator.yaml)
• That apphosting.local.yaml allows plaintext secret values
• That this file should not be committed to source control
• Proper configuration patterns for Firebase App Hosting emulator secrets

This information was essential to solving the user's technical problem but was unreachable through WebFetch.

Suggested Solutions

Short-term

1. Add clear size limit warnings: Return metadata about content length and truncation
2. Provide truncation indicators: Clear markers showing "Content truncated at X KB of Y KB total"
3. Return error for oversized content: Fail gracefully with actionable error message

Medium-term

1. Implement pagination: Allow fetching content in chunks with offset parameters

``
WebFetch(url, prompt, offset=0, limit=50000)
WebFetch(url, prompt, offset=50000, limit=50000)
``

1. Add section-based fetching: Allow targeting specific sections of documentation

``
WebFetch(url, prompt, section_selector="#configure-emulator")
``

1. Smart truncation: When truncating, try to break at section boundaries rather than mid-sentence

Long-term

1. Automatic chunking: When content exceeds limits, automatically break into logical chunks
2. Content-aware summarization: For large pages, provide a table of contents and allow selective deep-dive
3. Caching and resumption: Allow Claude to make multiple requests to the same URL efficiently

Additional Context

Why This Matters

Documentation pages are often long because they are comprehensive. The most detailed, important information is frequently:

• In later sections (after overview/prerequisites)
• In advanced configuration sections
• In troubleshooting sections

Truncating long documentation pages at arbitrary points makes WebFetch unreliable for technical problem-solving, which is a core use case for Claude Code.

User Experience Impact

The user explicitly stated: "this absolutely has the right answer" but Claude couldn't access it through the provided tool. This creates:

• Trust issues with Claude's capabilities
• Frustration when the answer exists but can't be retrieved
• Perception that Claude is not reading provided documentation

Attachments

Example of Truncated Output

"Based on the provided web page content, there is **no information** about the `apphosting.emulator.yaml` file format and structure.

The page content appears to be truncated mid-sentence ("Before using the App Hosting emulator, mak") and does not contain details about configuration file specifications."

Actual Content (retrieved via curl)

<p>If your application uses many secrets (for example, API keys for three different
services, with different values for each of production, staging, and local
development) you may exceed Cloud Secret Manager's free tier and pay $0.06 per
active secret per month. In this case, instead consider using an
<code translate="no" dir="ltr">apphosting.local.yaml</code> file. Unlike
<code translate="no" dir="ltr">apphosting.emulator.yaml</code> this file is
allowed to provide plaintext values for environment variables that are secret
values in <code translate="no" dir="ltr">apphosting.yaml</code>.</p>

Related Issues

• WebFetch tool may have similar issues with other long-form documentation sites
• This likely affects Firebase docs, AWS docs, Google Cloud docs, etc.
• Any technical documentation over ~8-10KB may be impacted

Priority Justification

This is a high-priority issue because:

1. It directly impacts Claude Code's ability to solve technical problems
2. The workaround requires using Bash/curl, which is not intuitive
3. It affects a core use case: reading documentation to solve problems
4. Users explicitly pointing to documentation and Claude being unable to read it severely damages trust
5. The issue is systematic and affects multiple attempts with different prompts

Recommendation

Implement short-term fixes (size warnings, clear errors) immediately, and plan for pagination/chunking capabilities in the next tool update. Consider this a critical path issue for Claude Code's effectiveness in technical problem-solving scenarios.

What Should Happen?

The WebFetch tool should:

1. Retrieve complete page content or clearly indicate content length limits
2. 2. Provide pagination or offset capabilities for large pages
3. 3. Return error messages if content exceeds size limits
4. 4. Allow multiple fetch requests with offset/range parameters
5. 5. Successfully access critical documentation sections regardless of their position on the page

Error Messages/Logs

Steps to Reproduce

1. Call WebFetch tool with URL: https://firebase.google.com/docs/emulator-suite/use_app_hosting
2. 2. Try different prompts such as:
3. - "Please extract ALL content from this page"
4. - "What does this page say about apphosting.emulator.yaml?"
5. - "List all the steps and configuration mentioned"
6. 3. Observe consistent truncation at the same point (mid-sentence)
7. 4. Verify full content exists: curl -s "URL" | wc -l returns 15,858 lines
8. 5. Note that critical information about apphosting.local.yaml is unreachable via WebFetch

Claude Model

Sonnet (default)

Is this a regression?

No, this never worked

Last Working Version

_No response_

Claude Code Version

Latest (2026-02-03)

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

Terminal.app (macOS)

Additional Information

_No response_