[FEATURE] Skill invocation tracking and usage analytics

Status Fixed / completed
Maintainer reply None cached
Activity 7 comments · opened Mar 17, 2026 · closed Aug 17, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

Organizations adopting Claude Code at scale are hitting a skill management problem: there is no way to know which skills are actually being used.

Our org went from 67 to 183 skills in under a month. Multiple people have flagged concerns about bloat, redundancy, and context window pressure (#10238, #33285), but every proposed solution — hierarchy, owners, cleanup policies — requires one missing primitive: invocation data.

Today, session metadata tracks "Skill": 3 (aggregate tool count) but not which skill was invoked. Without per-skill invocation tracking, we cannot:

  • Identify unused skills for deprecation
  • Measure adoption of newly created skills
  • Detect duplicate skills (two skills solving the same problem, one unused)
  • Justify context window budget for skills that compete for the ~2% token limit
  • Give skill authors feedback on whether their skill is being used

This was previously requested in #20970, which was auto-closed for inactivity without any response.

Proposed Solution

Level 1: Local primitive — Log skill name on invocation

When the Skill tool is called, include the skill name in session telemetry. Minimal addition to the existing session-meta schema:

{
  "tool_counts": {
    "Skill": 3,
    "Skill:create-pr": 2,
    "Skill:code-review": 1
  }
}

This is backward-compatible — existing "Skill": N stays, namespaced keys are additive.

Level 2: Local usage log

Append skill invocations to ~/.claude/analytics/skill-usage.jsonl:

{"timestamp": "2026-03-17T10:30:00Z", "skill": "create-pr", "trigger": "slash_command", "project": "/path/to/repo"}
Level 3: CLI query command
claude skills --stats

# Skill Usage (last 30 days)
# ===========================
# create-pr        45 invocations
# code-review      32 invocations  
# triage            8 invocations
# my-unused-skill   0 invocations  ← candidate for removal
Level 4: Org-wide analytics via external observability platforms

Levels 1–3 solve the individual developer case but do not answer the org-level question: "Which of our 183 skills should we deprecate?" That requires aggregating usage data across all developers.

Proposal: Expose skill invocation events to external analytics/observability platforms.

Claude Code should support emitting skill invocation events to configurable external sinks so organizations can aggregate, query, and dashboard skill usage across their entire developer base. This could work via:

Option A: Webhook / HTTP sink (simplest)
Configure an endpoint in settings that receives skill invocation events:

{
  "analytics": {
    "skill_events_webhook": "https://internal-collector.example.com/claude/skill-events"
  }
}

Each invocation emits a lightweight payload:

{
  "event": "skill_invocation",
  "skill": "create-pr",
  "trigger": "slash_command",
  "timestamp": "2026-03-17T10:30:00Z",
  "session_id": "abc123",
  "user_hash": "sha256(username)",
  "project_hash": "sha256(repo_path)"
}

Organizations can then route these to their observability stack of choice — Snowflake for historical analytics and trend queries, Chronosphere/Datadog/Grafana for real-time dashboards and alerting on skill adoption, or any data warehouse.

Option B: OpenTelemetry (OTEL) integration
Emit skill invocations as OTEL spans or events, allowing orgs to plug into any OTEL-compatible backend (Chronosphere, Datadog, Splunk, BigQuery, etc.):

{
  "analytics": {
    "otel_endpoint": "https://otel-collector.example.com:4317",
    "otel_service_name": "claude-code"
  }
}

This is the most flexible approach — OTEL is an industry standard and most enterprise observability platforms already support it.

Option C: Hook-based (works today, fragile)
Organizations can approximate this today using PreToolUse hooks that fire on Skill tool calls and log to an external system. But this is fragile — hooks don't receive the skill name as a parameter, so parsing is unreliable. Native support in Options A or B would be far more robust.

What org-level analytics enables:

| Query | Value |
|-------|-------|
| Which skills have zero invocations in 90 days? | Deprecation candidates |
| Which skills are used by only 1 person? | Bus factor risk or personal-use skills that shouldn't be in shared config |
| Which skills have high invocation but low success? | Quality improvement targets |
| What's the adoption curve for a newly shipped skill? | Measures rollout effectiveness |
| Which teams use which skills? | Informs ownership and maintenance responsibility |
| Skill invocations per week trending up or down? | Measures overall platform adoption |

Why This Matters at Scale

| Metric | Our org |
|--------|---------|
| Total skills | 183 |
| Growth rate | 67 → 183 in 4 weeks |
| Context window usage | 1.3% and climbing |
| Skills with evals | 23 / 183 (13%) |
| Skills with known usage data | 0 / 183 (0%) |

Without invocation tracking, the only cleanup signal is git history (last modified date), which conflates "stable and widely used" with "abandoned." We need usage data to make informed decisions — both locally and across the organization.

Priority

High - Critical for large-scale adoption

Feature Category

CLI commands and flags

Additional Context

  • Refs: #20970 (closed as stale, same request), #10238 (skill hierarchy), #33285 (skill discovery/dedup)
  • Privacy: Local logging (Levels 1–3) stays local with opt-out. Org-level analytics (Level 4) is explicitly opt-in and configured by the organization. User identifiers are hashed by default. No data sent to Anthropic.
  • Implementation scope: Level 1 (namespaced skill names in session-meta) is a small change that unblocks everything else. Levels 2–4 can be built incrementally.

View original on GitHub ↗

6 Comments

github-actions[bot] · 5 months ago

Found 1 possible duplicate issue:

  1. https://github.com/anthropics/claude-code/issues/20970

This issue will be automatically closed as a duplicate in 3 days.

  • If your issue is a duplicate, please close it and 👍 the existing issue instead
  • To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

harrism04 · 5 months ago

We have this requirement as well

grob-bot · 5 months ago

+1 — specifically for Level 4, Option B (OTEL) with per-API-call skill attribution.

Use case: We run a self-hosted cost dashboard that ingests claude_code.api_request OTEL logs. Today these include model, input_tokens, output_tokens, cost_usd, session_id, and process.cwd — but no execution context. We can break down spend by model and project, but not by skill, agent depth, or tool.

What we'd want on each claude_code.api_request log record:

| Attribute | Value | Why |
|-----------|-------|-----|
| skill_name | "commit", "review", "" | Per-skill cost attribution |
| agent_depth | 0, 1, 2 | Distinguish main vs subagent spend |
| tool_name | "Bash", "Edit", "Agent" | Know which tool triggered the API call |
| conversation_turn | 5 | Spot runaway loops |

The first three would let us answer "which skills cost the most?" and "are subagents worth their token spend?" — questions that matter a lot when managing AI-assisted development costs across a portfolio of projects.

Level 1 (local skill counts) is useful but doesn't solve cost attribution — we need the skill context attached to each API request event, not just aggregated invocation counts.

backstabslash · 4 months ago

The raw JSONL session logs already contain per-skill invocation data - the Skill tool calls include the skill name in their input, and Agent calls include the subagent_type. So Level 1 data effectively exists today, it's just not surfaced.

I built a CLI tool that parses these logs and provides the Level 3 experience: goccc -tools. It covers tool, skill, and agent breakdowns with error rates, duration tracking, and unused skill detection.

That said, this doesn't address Level 4 (org-wide analytics) at all, which is where the real gap is for teams at scale.

neeltom92 · 2 months ago

Built something for Skills analytics & usage using OTEL/Grafana : https://github.com/neeltom92/claude-code-observability

<img width="1192" height="831" alt="Image" src="https://github.com/user-attachments/assets/13905ef5-b4a9-4d48-85f5-fd2bd23de5f9" />

anandsaini18 · 1 month ago

Built a small open-source CLI that covers part of this locally while a native solution ships: deadskills (npx deadskills, npm).

It parses Claude Code session transcripts in ~/.claude/projects (JSONL), extracts per-skill invocations, and reports:

  • Invocation counts per skill (from your history, not aggregate Skill: N)
  • Dead skills: installed, never invoked (deprecation candidates)
  • Zombie skills: invoked before, silent 90+ days
  • Context tax: estimated tokens every installed skill adds to every prompt

This maps roughly to Levels 1–3 in the proposal, but as offline transcript analysis rather than new telemetry in session-meta. No config, no login. Data never leaves your machine.

Also supports Codex (~/.codex) for cross-agent comparison.

Caveats (being upfront):

  • Invocation detection is transcript-based; if log shapes change, counts can drift. deadskills doctor surfaces parse health.
  • Token numbers are estimates (~chars/4), always labeled with ~.
  • No org-wide aggregation (Level 4). That likely needs native hooks or OTEL anyway.

Example:

── claude-code ─────────────────────────────────────────────
107 sessions · 34,296 turns analyzed
Context tax: ~587 tokens added to every prompt by 6 installed skills

  swiftui-expert-skill           ████████████████████████    9×   ~3.5M tok
  ui-ux-pro-max                  ████████████████░░░░░░░░    6×     ~8M tok

Dead skills (1): installed, never invoked
  xcode-project-setup            personal · costs ~49 tok/prompt for nothing

If this is useful as a stopgap or helps inform the native design, feedback welcome. Happy to align on whatever schema you pick for namespaced skill counts in session-meta.

Showing cached comments. Read the full discussion on GitHub ↗