[BUG] Skill auto-activation appears to use token/keyword overlap, not semantic matching

Summary

Based on third-party empirical analysis, skill auto-activation in Claude Code appears to rely on token and keyword overlap between the user's prompt and the skill's description field, rather than semantic matching. This contradicts the common mental model (and how many skill authors write descriptions). Clarity on the intended behavior would help authors write descriptions that actually trigger.

Evidence

Scott Spence ran sandboxed evals across skill descriptions and concluded: "Claude isn't doing semantic matching at the activation layer. It's doing something closer to keyword matching."

Source: https://scottspence.com/posts/measuring-claude-code-skill-activation-with-sandboxed-evals

Follow-up post documenting additional reproduction: https://scottspence.com/posts/claude-code-skills-dont-auto-activate

Separate analysis by fsck.com documents a related constraint. Skill descriptions share a budget of approximately "15,000 characters (or around 4000 tokens)". Beyond that, skills are not surfaced for activation.

Source: https://blog.fsck.com/2025/12/17/claude-code-skills-not-triggering/

Expected behavior

Given Claude's semantic capabilities, users expect skill descriptions to be matched against user intent semantically. A description like "Formats and validates JSON files" should trigger on prompts like "clean up this data file" when the file is JSON.

Actual behavior

Activation appears to require surface-level token overlap. Descriptions written semantically (explaining purpose in natural language) do not trigger reliably. Descriptions stuffed with literal keywords from anticipated user prompts trigger more reliably. This is the opposite of how skill authoring guidance is written.

Why existing issues do not cover this

• #32184, #34648, #36570 are scoped to headless mode (claude -p) specifically. This report is about interactive mode activation mechanism.
• #39390 reports explicit /skill-name invocation being overridden by keyword matching on arguments. That is a symptom of the same underlying mechanism, but framed as "explicit invocation should win" rather than "document or fix the activation mechanism."
• #43259 and #46383 propose per-skill fixes (narrower descriptions, auto_invoke flag) without documenting the activation layer itself.
• None of the above reference the empirical sandboxed eval data or the 15,000 character budget constraint.

Request

1. Confirm or correct the public understanding that activation uses token and keyword overlap rather than semantic matching.
2. Document the character and token budget constraint for the skills block.
3. Provide official skill authoring guidance consistent with the actual mechanism.

Suggested labels: bug, area:skills, documentation