Insights report analyzes too few sessions and over-extrapolates from thin data

Resolved 💬 3 comments Opened Feb 5, 2026 by antondelafuente Closed Feb 8, 2026

Problem

The /insights report analyzes a tiny fraction of sessions and then over-extrapolates single incidents into recurring themes across the entire report.

Details

Under-sampling: With 871 sessions available, the report only generated 6 facet files (in ~/.claude/usage-data/facets/). The hardcoded limit is 50 sessions, but it didn't even reach that — only ~6 were analyzed, covering 5 "project areas." This makes the report feel useless for power users with significant history.

Over-extrapolation from single incidents: The report found one session where Claude suggested using a REPL instead of a file-based approach. This single incident was then repeated as a theme in nearly every section of the report:

  • Interaction style narrative
  • Friction analysis examples
  • CLAUDE.md suggestions ("never suggest REPL")
  • Usage patterns ("start sessions with explicit constraints" referencing REPL)
  • At-a-glance (what's working, what's hindering, quick wins)
  • On the horizon

The user didn't even remember this incident or know what REPL meant, yet it was presented as a defining workflow preference. A single data point shouldn't be amplified into the report's central narrative.

Suggestions

  1. Analyze more sessions — the 50-session cap is already low for power users; 6 out of 871 is not useful. Consider making this configurable or scaling with session count.
  2. Don't repeat the same anecdote across every section — if there's only one example of a friction pattern, present it once, not as a recurring theme throughout the report.
  3. Be transparent about sample size — the report should clearly state how many sessions were actually analyzed vs. total, so users understand the confidence level.

Environment

  • Claude Code version: 2.1.31
  • 871 sessions over ~1 month
  • 6 facet files generated

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗