[FEATURE] - [Telemetry] Opt-in for verbatim plugin/skill labels on token.usage and cost.usage metrics
Preflight Checklist
- [x] I have searched existing requests and this feature hasn't been requested yet
- [x] This is a single feature request (not multiple features)
Problem Statement
We operate an internal Claude Code marketplace (~10 plugins, growing) and run a local OpenTelemetry → Prometheus → Grafana stack so plugin authors can see token usage, cache-hit ratio, and cost-per-invocation broken down by which plugin consumed the tokens.
That breakdown is the central question the framework exists to answer ("which of my plugins is expensive?"), and right now it's impossible to answer.
When a skill from a non-Anthropic plugin is invoked, Claude Code emits claude_code.token.usage and claude_code.cost.usage metrics with plugin_name="third-party" and skill_name="third-party" - regardless of OTEL_LOG_TOOL_DETAILS=1 or any other knob we could find in the docs. Every plugin in our marketplace lumps into a single "third-party" bucket on those metrics.
The redaction is also asymmetric:
claude_code.skill_activatedevents emit verbatim plugin/skill names.agent_nameontoken.usageis verbatim (e.g."Explore"when a subagent fires inside a third-party plugin's context).- Only
plugin_nameandskill_nameon token/cost metrics are redacted.
So Claude Code clearly has the verbatim names at the moment those metrics are emitted - they just get scrubbed before export
Proposed Solution
Either of the following would solve our problem; they're listed in order of our preference.
Option A - opt-in env var. Add OTEL_LOG_PLUGIN_DETAILS=1, paralleling the existing OTEL_LOG_TOOL_DETAILS=1 and OTEL_LOG_USER_PROMPTS=1 opt-ins. When set, emit verbatim plugin_name and skill_name on claude_code.token.usage, claude_code.cost.usage, and claude_code.code_edit_tool.decision.
Default stays redacted - the operator explicitly chooses to surface this to their own collector.
Interaction:
``bash``
export OTEL_LOG_PLUGIN_DETAILS=1
claude
# now: claude_code_token_usage_tokens_total{plugin_name="jira-to-plan", skill_name="ticket-fetcher", ...}
Option B - trust allowlist. Support trustedPluginSources: [...] in settings or a CLAUDE_TRUSTED_PLUGIN_SOURCES env var listing marketplace URLs / plugin namespaces whose names should not be redacted.
Plugins matching the list emit verbatim labels; everything else stays redacted.
Narrower privacy surface than the global env var, and it mirrors the trust model Claude Code already uses for plugin auto-install gating.
Alternative Solutions
We tried each of these before filing this issue.
- OTEL_LOG_TOOL_DETAILS=1 - only affects tool input logging, not metric labels. No effect on the third-party redaction.
- Per-plugin shell wrappers that prepend plugin.name=X to OTEL_RESOURCE_ATTRIBUTES before invoking claude - works for one-shot CI runs, but doesn't help interactive sessions where users invoke multiple plugins.
- session_id correlation -pure session-id joins fail for multi-plugin sessions (which plugin owns the tokens when two plugins fire in one session?).
- A PreToolUse hook on the Skill tool that emits its own attribution event -this is what we ended up shipping as a workaround. It POSTs an OTLP log to the local collector with verbatim plugin.name / skill.name
/ session.id on every Skill invocation, and the dashboard does a PromQL join with token_usage time-bucketed by session_id.
- This works, but:
- It duplicates information Claude Code already has internally.
- Hooks only see the tool boundary, so attribution is approximate - we can't bind tokens consumed during a multi-turn skill execution back to that skill with certainty.
- The time-bucketed PromQL join is fragile at activation boundaries (back-to-back plugin invocations within one batch interval get misattributed).
- Every new marketplace operator who wants per-plugin cost has to install this hook plugin, when the data already exists upstream.
Priority
Medium - Would be very helpful
Feature Category
Other
Use Case Example
Scenario: a marketplace operator (us) running a Tuesday cost review.
- Author of an internal jira-to-plan plugin asks: "did my refactor last week reduce token usage?"
- Operator opens the marketplace dashboard, filters last 7 days, sorts by sum(rate(claude_code_cost_usage_USD_total[7d])) by (plugin_name).
- With current redaction: dashboard shows two rows - plugin_name="third-party" with $42 spent across the org, and a smattering of Anthropic builtin subagents. The author cannot tell whether jira-to-plan got
cheaper, more expensive, or unchanged. The dashboard is unanswerable for its primary use case.
- With OTEL_LOG_PLUGIN_DETAILS=1 (Option A): dashboard shows one row per Driivz plugin. Author sees jira-to-plan dropped from $8.20 → $5.10 week-over-week, confirms the refactor worked, ships it more widely.
Same scenario applies to: finding expensive skills inside cheap plugins, comparing model selection across plugins (Opus-heavy vs. Haiku-heavy), and showing a plugin author "your cache-read ratio is 30% - fix your
prompt caching."
Additional Context
_No response_
Showing cached comments. Read the full discussion on GitHub ↗
6 Comments
+1 on this being opt-in rather than all-or-nothing. Plugin/skill labels are often the only human-readable clue that a cost spike came from a specific workflow, but raw labels can also leak internal names.
One compromise that has worked well in governance tooling is to emit both a stable opaque id and an optional display label:
That lets teams do reliable cost attribution without forcing everyone to expose internal marketplace naming in telemetry by default.
Do we have any timeline for this feature?
An opt-in privacy mode could support both useful labels and safer defaults.
Emit stable hashed skill/plugin labels by default, with an explicit allowlist for verbatim names. Include label scheme/version in the metric attributes so dashboards can aggregate per-skill cost without leaking internal plugin names unless the operator opted in.
---
_Generated with ax._
Closing for now — inactive for too long. Please open a new issue if this is still relevant.
This would be great to have natively. In the meantime, I've been doing a rough version of this offline by
parsing the session JSONL transcripts.
The attribution I get from 82 sessions:
Per-tool granularity from the transcripts is limited since the role is just "tool" without labeling which tool
produced it, but even the aggregate split is eye-opening — it makes clear that tool output management is the
single biggest lever for cost control.
If you want to run this on your own sessions: https://github.com/yingxiangge/agent-cost (read-only Python CLI,
parses local transcripts).
This would be great to have natively. In the meantime, I've been doing a rough version of this offline by parsing the session JSONL transcripts.
The attribution I get from 82 sessions:
Per-tool granularity from the transcripts is limited since the role is just "tool" without labeling which tool produced it, but even the aggregate split is eye-opening — it makes clear that tool output management is the single biggest lever for cost control.
If you want to run this on your own sessions: https://github.com/yingxiangge/agent-cost (read-only Python CLI, parses local transcripts).