claude-api bundled skill inlines ~227K tokens on invocation — progressive disclosure defeated by bundled packaging
Note on venue: verified before filing — the skill source at anthropics/skills uses an on-demand file-read model with no inlined reference docs, while the bundled variant delivers those files pre-rendered. The regression is in the bundled-skill packaging/render step, so filing here rather than anthropics/skills. Distinct from #14882 / #14834 / #16616 / #39686, which cover startup loading and /context display — this is invocation-time payload size.
Environment: Claude Code 2.1.220 (Linux), Opus 5 (1M context), bundled
skill at bundled-skills/2.1.220/<hash>/claude-api. Repo: Go + Rust + TS
monorepo, go.mod at root, no Anthropic SDK imports anywhere (project targets
Ollama / OpenAI-compatible endpoints).
What happened: One invocation of the claude-api skill (Skill tool, args:
"Opus 5 model ids, 1M context pricing tier, effort levels, when to use opus vs
sonnet vs haiku for subagents") added +227,284 tokens to the context in a
single step — 66% of the session's total context at that point. Measured from
the session transcript's per-request usage fields: prompt total jumped
53,590 → 280,874 on the request following the skill turn. The skill content
arrived as one user turn containing a single 598,008-character block.
What the docs say should happen:
https://platform.claude.com/docs/en/agents-and-tools/agent-skills/claude-api-skill
claims: "Claude loads only the documentation relevant to your project's
language, surface (Messages API or Managed Agents), and the specific task at
hand (tool use, streaming, batches, and so on), rather than loading everything
at once."
Observed vs claimed, per axis:
- Language: WORKED — only
go/docs were included (not the other 7 languages). - Surface: FAILED — both Messages API docs AND the full Managed Agents doc set
(~14 shared/managed-agents-*.md files: core, environments, events, memory,
multiagent, outcomes, overview, scheduled-deployments, self-hosted-sandboxes,
tools, webhooks, client-patterns, api-reference, onboarding) were inlined.
The task mentioned nothing about Managed Agents.
- Task: FAILED — the full model-migration guide (largest single doc),
prompt-caching, error-codes, anthropic-cli, platform-availability,
token-counting, tool-use-concepts, agent-design all arrived regardless of the
task, which needed roughly 2K tokens of model-table + effort-ladder content.
Mechanism (verified, not hypothesized): the source SKILL.md at
anthropics/skills instructs on-demand reads of separate files and embeds no
reference docs. The bundled payload received in Claude Code 2.1.220 contained
those same files pre-rendered inline as <doc path="..."> blocks under an
"## Included Documentation" section. The inlining therefore happens in Claude
Code's bundled-skill packaging/render step, which forecloses the on-demand
disclosure the source implements and the docs page advertises. Selection
survives only at the language level (Go-only was correctly chosen).
Measurement caveat (for precision): the 598,008-char block against the
+227,284-token delta implies ~2.6 chars/token — dense but plausible for
table/code-heavy markdown; the delta window includes a small amount of other
turn content. Order of magnitude is not in question; the exact figure is
±~10%.
Impact: in a repo with no Anthropic SDK code, any prompt that merely names
a Claude model can trigger the skill and consume a quarter of a 1M window (or
exceed a 200K window outright) for a lookup-sized question. The skill's SKIP
heuristic (grep for other providers) does not protect repos that use neither
the Anthropic SDK nor a competitor's (e.g. local-model tooling).
Suggested directions (either would resolve it):
- Render only the sections matching the detected surface + a task keyword
match, keeping the rest as on-demand file reads (the model the docs already
describe); or
- Split the payload: always-inline the quick-reference tables (~2-5K), and
gate the migration guide + managed-agents corpus behind explicit
subcommands (migrate, managed-agents-onboard) that already exist.
Repro: in any Go repo without Anthropic SDK imports, invoke/claude-api <any model question> and inspect the next request'scache_creation_input_tokens / prompt-size delta in the session transcript.
Showing cached comments. Read the full discussion on GitHub ↗
3 Comments
Independent corroboration on the same version (2.1.220), macOS, with the opposite detection outcome — and it confirms your packaging thesis from the filesystem side.
You had language detection succeed (Go repo → only
go/). I hit the fallback path, where it fails, and that's the worst case: all eight language variants ship.The bundled tree has no
SKILL.mdThis is the part that most directly supports "progressive disclosure defeated by bundled packaging":
| | Source (
anthropic-agent-skillsmarketplace copy) | Bundled (bundled-skills/2.1.220/<hash>/claude-api) ||---|---|---|
|
SKILL.mdrouter | 71,590 chars | absent — 0 found || Reference tree | 779,501 chars across 64 files | 779,501 chars across 64 files |
| File count | 66 | 64 |
The router that's supposed to decide what to read isn't in the bundle. The references are there pre-rendered instead. Whatever the render step does, it isn't shipping the on-demand-read model the source describes. (Two hash directories exist under
2.1.220/;diff -rqsays they're byte-identical duplicates.)Full inventory of what's on disk
Largest single file:
shared/model-migration.mdat 174,614 chars — 22% of the entire bundle, in the unconditionalshared/set, loaded for every invocation regardless of whether the session has anything to do with migrating models.Two floors, both high
shared/+python/= 559,736 charsSo successful detection saves 219,765 chars — real, but it's a discount off an already-enormous baseline, not progressive disclosure.
shared/alone is 63% of the worst case.Chars → tokens
Worth calibrating, because the intuitive ~3.5 chars/token badly understates this content. Two independently measured reports:
cache_creation_input_tokens) = 2.71usagedelta) = 2.63≈ 2.67 chars/token. Applying it:
| Case | Chars | Est. tokens |
|---|---:|---:|
| Language detected (Python) | 559,736 | ~210k |
| Language not detected | 779,501 | ~292k |
A second user reproduced the fallback path in a clean session and measured context going 47k → 377k (+330k tokens) on a single by-name invocation — above the static estimate, since it also carries the tool_use block, thinking, and reply. The two figures agree.
This is a regression against a report that was closed as a duplicate
#50079 filed this exact fallback in April on 2.1.112: same repro (directory with no auto-detectable source files), same quoted string in the expansion —
— followed by all the docs anyway, so the "ask the user" step can only narrow what gets quoted, never unload what's already cached. It measured 133,510 tokens, twice, agreeing within 7 tokens across sessions.
It was auto-closed as a duplicate of #14882 after 3 days and auto-locked 7 days later. Nobody triaged it.
The same bug is now ~292k tokens. It more than doubled while sitting closed.
It exceeds a 200k context window outright
Not an arithmetic point — it's documented twice:
Prompt is too longon the next call. "One tool call from a fresh session to a dead session."At ~292k tokens the fallback cannot fit a 200k window under any circumstances, so on those models the failure isn't degraded quality or cost — it's an unrecoverable session, triggered by a skill the user may never have asked for.
---
Corroborating measurements by Victor, who independently ran the clean-session experiment (47k → 377k) and identified the empty-working-tree fallback as the trigger. Happy to add their GitHub handle here on request.
Related: #14882 (canonical), #79457 (cost impact, triaged), #80190 (
shared/inlines unconditionally), #81059 (trigger breadth), #50079 (closed/locked, same fallback at half the size), anthropics/skills#1487 and #1486.I've also observed this. Further, if you try to invoke the skill from a weaker model with 200k context (e.g. haiku 4.5) it immediately triggers compaction.
<img width="862" height="190" alt="Image" src="https://github.com/user-attachments/assets/803f049a-f99f-42dd-b737-aa804c5214a9" />
Independent third corroboration — Claude Code 2.1.220, macOS. Measured the bundled tree at
bundled-skills/2.1.220/<hash>/claude-apidirectly (byte counts viawc -c), no target file in the working directory, language fell back to TypeScript:Every figure matches the inventory already posted here exactly (494,077 / 174,614 / 54,833), confirming the bundle is deterministic across machines/sessions and the root cause is packaging, not per-session variance.
I filed this independently as #83532 before finding this thread — closing mine as a duplicate of this one, since this issue already has the fuller root-cause writeup (source-vs-bundle diff, regression history via #50079, chars/token calibration).