/insights samples only 3 sessions from 2,912, produces misleading analysis
Bug Report: /insights samples too few sessions, causing misleading analysis
Summary
The /insights command samples only 3 sessions regardless of total session count, then extrapolates those 3 sessions to make sweeping generalizations about usage patterns. This produces misleading analysis that doesn't represent actual usage.
Environment
- Claude Code version: (latest as of Feb 3, 2026)
- Sessions in date range: 2,912
- Sessions actually analyzed: 3 (0.1%)
Steps to Reproduce
- Use Claude Code extensively over a month (2,912 sessions in my case)
- Run
/insights - Check the facets directory:
ls ~/.claude/usage-data/facets/ - Note only 3 JSON files exist - one per sampled session
Expected Behavior
The insights command should:
- Sample a representative number of sessions (50-100 minimum)
- Weight sampling evenly across the date range
- Be transparent about sample size in the output
- Not extrapolate small samples into confident generalizations
Actual Behavior
- Only 3 sessions sampled from 2,912 (0.1%)
- All 3 happened to be recent sessions
- Report confidently states "88 sessions" of debugging work - this number appears fabricated/extrapolated
- Characterizes user as "system forensics partner" based on ONE debugging session
- Misses entire categories of actual usage (strategic intelligence, knowledge work, automation)
Impact
The generated report is actively misleading:
Report claimed:
- "Power user treating Claude Code as system investigation partner"
- "Heavy macOS log diving and security analysis"
- "88 sessions of troubleshooting/debugging"
Actual usage (from manual sampling across date range):
- Chief of Staff personal assistant system development
- Strategic intelligence and executive support
- Tool integration and system configuration
- Knowledge work and document generation
- Notion/Memory MCP integration for strategic workflows
The report would give someone a completely wrong impression of how I use Claude Code.
Evidence
Facets directory contains exactly 3 files:
136816fa-d68e-491d-bce4-751d000976eb.json # Twitter archiving
90e200c8-6259-47eb-85d3-3605049fcba0.json # Keychain investigation (the one that skewed everything)
cb282d4c-2589-4412-8a68-d385cb4444b7.json # Parallel agent demo
Manual sampling of 6 sessions across Jan 14 - Feb 2 shows completely different usage patterns focused on strategic intelligence and automation work.
Suggested Fixes
- Increase sample size: Sample at least 50-100 sessions, or scale with total count
- Stratified sampling: Ensure samples are distributed across the full date range
- Transparency: Report sample size and confidence level in output
- Avoid extrapolation: Don't multiply a 3-session sample into "88 sessions" claims
Additional Context
The "88 sessions" number appears in multiple places in the report as if it's real data:
- "88 sessions marked as 'wrong_approach' friction"
- "88 of 90 analyzed sessions ended as 'partially_achieved'"
- "88 sessions showed 'wrong_approach' friction"
These numbers seem to be extrapolated from the 3 actual samples, which is statistically invalid and creates false precision.
This issue has 5 comments on GitHub. Read the full discussion on GitHub ↗