/insights: prescriptions need repo-grounding; one trust-eroding security false-positive
Resolved 💬 1 comment Opened May 29, 2026 by commonontology Closed Jul 2, 2026
/insights— prescriptions need grounding; one trust-eroding false-positive pattern The behavioural/narrative half of/insightsis genuinely good — a fair mirror of how I work. The prescriptive half (suggestions, "features to try," "on the horizon," fun-ending) is unreliable in a consistent, structural way: it's generated from session transcripts in isolation, with no access to my repo's current state and no confidence calibration. Instances from one report: 1. Recommended building skills I already have (/handoff,/review, a verdict taxonomy, a validation loop). It can't see.claude/skills/, so it reinvents existing infrastructure. 2. A security false-positive, dramatized as fact. It identified the stringCC_APPLY_LIVE_AUTHORISEDas a "leaked secret auth token" and built a headline + a CLAUDE.md rule + a pre-commit hook + a roadmap item on it. It's actually a documented safety-tripwire flag name, correctly present in 7+ files; the proposed grep hook would block every legitimate commit touching them. Telling a user they leaked a secret when they didn't erodes trust fast — this should never surface as an established-fact "fun ending" without a verification pass. 3. Confabulated a specific figure ("500-token output max") that doesn't exist. 4. Inverted a causal chain (called my transcription fallback "Whisper→yt-dlp"; it's the reverse). 5. Proposed bash/grep hooks on a Windows/PowerShell environment obvious from the transcripts. Fixes: (a) ground prescriptions against the repo's control surfaces before suggesting users build what exists; (b) attach confidence + an explicit "verify against your setup" marker to every recommendation; (c) gate security/leak claims behind a semantic check, and never dramatize an unverified one.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗