/insights samples only 3 sessions from 2,912, produces misleading analysis

Resolved 💬 5 comments Opened Feb 4, 2026 by elrolio Closed Mar 5, 2026

Bug Report: /insights samples too few sessions, causing misleading analysis

Summary

The /insights command samples only 3 sessions regardless of total session count, then extrapolates those 3 sessions to make sweeping generalizations about usage patterns. This produces misleading analysis that doesn't represent actual usage.

Environment

  • Claude Code version: (latest as of Feb 3, 2026)
  • Sessions in date range: 2,912
  • Sessions actually analyzed: 3 (0.1%)

Steps to Reproduce

  1. Use Claude Code extensively over a month (2,912 sessions in my case)
  2. Run /insights
  3. Check the facets directory: ls ~/.claude/usage-data/facets/
  4. Note only 3 JSON files exist - one per sampled session

Expected Behavior

The insights command should:

  • Sample a representative number of sessions (50-100 minimum)
  • Weight sampling evenly across the date range
  • Be transparent about sample size in the output
  • Not extrapolate small samples into confident generalizations

Actual Behavior

  • Only 3 sessions sampled from 2,912 (0.1%)
  • All 3 happened to be recent sessions
  • Report confidently states "88 sessions" of debugging work - this number appears fabricated/extrapolated
  • Characterizes user as "system forensics partner" based on ONE debugging session
  • Misses entire categories of actual usage (strategic intelligence, knowledge work, automation)

Impact

The generated report is actively misleading:

Report claimed:

  • "Power user treating Claude Code as system investigation partner"
  • "Heavy macOS log diving and security analysis"
  • "88 sessions of troubleshooting/debugging"

Actual usage (from manual sampling across date range):

  • Chief of Staff personal assistant system development
  • Strategic intelligence and executive support
  • Tool integration and system configuration
  • Knowledge work and document generation
  • Notion/Memory MCP integration for strategic workflows

The report would give someone a completely wrong impression of how I use Claude Code.

Evidence

Facets directory contains exactly 3 files:

136816fa-d68e-491d-bce4-751d000976eb.json  # Twitter archiving
90e200c8-6259-47eb-85d3-3605049fcba0.json  # Keychain investigation (the one that skewed everything)
cb282d4c-2589-4412-8a68-d385cb4444b7.json  # Parallel agent demo

Manual sampling of 6 sessions across Jan 14 - Feb 2 shows completely different usage patterns focused on strategic intelligence and automation work.

Suggested Fixes

  1. Increase sample size: Sample at least 50-100 sessions, or scale with total count
  2. Stratified sampling: Ensure samples are distributed across the full date range
  3. Transparency: Report sample size and confidence level in output
  4. Avoid extrapolation: Don't multiply a 3-session sample into "88 sessions" claims

Additional Context

The "88 sessions" number appears in multiple places in the report as if it's real data:

  • "88 sessions marked as 'wrong_approach' friction"
  • "88 of 90 analyzed sessions ended as 'partially_achieved'"
  • "88 sessions showed 'wrong_approach' friction"

These numbers seem to be extrapolated from the 3 actual samples, which is statistically invalid and creates false precision.

View original on GitHub ↗

This issue has 5 comments on GitHub. Read the full discussion on GitHub ↗