/doctor extension audit: count-bounded window, unverified truncation estimate, and disuse verdicts that ignore install recency
/doctor's extension-usage audit produced misleading verdicts on a
heavily-extended setup (70 user skills, 28 enabled plugins). Three separable
defects, observed on one real run (2026-07-30):
1. The transcript window is bounded by count, so its time span depends on usage density
The audit scans the ~50 most-recently-modified transcripts. On a machine running
many sessions per day, 50 transcripts resolved to 2 calendar days across 2
projects — and judging 98 extensions for disuse against a 2-day window makes
everything look unused. Re-running across all 354 available transcripts (35
projects, 31 days) materially changed the verdicts: several "unused" extensions
had real usage, and the genuinely-unused set became defensible.
Ask: bound the window by time, not item count — widen past N files until
the window spans ~14–30 days or transcripts are exhausted; report the resolved
window in BOTH dimensions (sessions and calendar days); and refuse to assign
disuse verdicts when the window spans under ~7 days.
2. Skill-listing truncation is estimated from disk when it is directly observable in context
The audit estimates listing cost as chars/4 against the ~1% context budget. But
the agent running the check HAS the rendered listing in its own context, where
truncation is directly observable: over-budget entries appear as bare names with
no description. Cross-checking three bare-listed skills against their SKILL.md
frontmatter (descriptions of 315/251/192 chars on disk, absent from the listing)
converted a soft token-cost estimate into a confirmed routing defect — those
skills cannot be routed to by description match.
Ask: add a verification step — inspect the live rendered listing for
entries lacking descriptions, confirm against frontmatter on disk, and report
confirmed truncation as a routing/correctness finding naming affected skills,
not merely a token-cost figure.
3. Zero usage counters conflate "not wanted" with "not yet reached"
Zero lifetime invocations + zero window hits currently earns a "remove"
recommendation. On this run that produced a recommendation to remove 58 skills
and 14 plugins — and the user rejected it: the four largest zero-use bundles
(94 skills, ~60% of the listing cost) had been deliberately installed within the
preceding 9 days and simply not reached yet. The counter cannot distinguish
"installed, evaluated, abandoned" from "installed, not yet used", and the
verdict rules never consult install recency, though the signal exists
(pluginUsage.lastUsedAt is seeded at install/enable; skill directory mtimes
are on disk).
Ask: weight disuse verdicts by install/enable recency — anything installed
within roughly the scan window gets a question, not a "remove" recommendation.
And decouple the two problems the check fuses: when the listing budget is the
binding constraint, the remedy is rotating bundles in/out by domain, not
permanent removal — offer rotation as a first-class option.
---
*General principle across all three: a scan whose conclusions depend on elapsed
time must bound its window by time; a metric that cannot distinguish "rejected"
from "not yet reached" needs a recency signal; and when the agent is itself the
system under inspection, direct observation of live state beats reconstructing
it from disk.*