Recurring pattern: evidence-substitution under concurrency + reactive-not-proactive gap handling in long-running sessions

Status Open
Maintainer reply None cached
Activity 0 comments · opened Sep 1, 2026

Summary

Recurring failure pattern across a long-running personal-assistant use case (a digital library management project spanning 40+ sessions, with a user-maintained "Never Event" ledger tracking incidents since the project began): Claude repeatedly treats adjacent or proxy evidence as if it were direct verification of the specific claim being made, and separately, defaults to reactive rather than proactive problem-solving even when it has already independently discovered the problem. Both patterns recur across many sessions despite the user writing increasingly explicit standing rules to prevent them.

Pattern 1: Evidence substitution under concurrency / partial-completion conditions

Concrete instance from today's session: a batch file-reorganization script attempted a database write via a serialized single-writer queue, got an abort response ("a Scribe is already running" — a different writer instance already held the lock), and its rollback logic treated the abort as proof the write had not happened, physically reverting 7 file moves. In fact the other, already-running writer instance drained the same queue moments later and did apply the writes — leaving the database correctly pointing at the new (intended) file locations while the actual files sat at the old locations. The mismatch was only caught because a user-configured standing rule required checking actual physical file state against the database state directly, rather than trusting the script's own "rolled back successfully" report.

This is the same class of failure the user's own incident ledger documents recurring at least three times previously in this project under different names:

  • An exile action was reported as successful because the API returned {ok:true}, when the underlying database write had actually been silently rejected by a constraint.
  • A dry-run classification pass was marked complete in a progress tracker without ever confirming the files it referenced had actually been moved.
  • A "verified surviving copy" check for a deletion candidate accepted any 64-character string as proof, without resolving it against a real database row.

In every case (including today's), the evidence cited was real and honestly reported — it just proved something near the claim, not the claim itself. The user formalized this as a standing rule after the third occurrence; today's session is the fourth distinct concrete instance of the same underlying pattern, now specifically in a concurrency/race-condition context rather than the earlier data-completeness contexts.

Pattern 2: Reactive rather than proactive problem-solving on self-discovered gaps

Across the same session, Claude's own code audits (not user prompts) surfaced three concrete, real gaps in the user's system: a RAG pipeline with no retrieval-relevance grading or hallucination checking, a metadata-extraction stage that was documented as implemented but had zero actual code behind it, and two locked design decisions that directly contradicted each other. In every case, Claude reported the finding as a bare fact and stopped there. The user then had to separately locate and paste ten different candidate open-source tools, one at a time, before Claude did any comparative research into solutions — despite an existing standing project rule explicitly requiring proactive best-practice research before/alongside surfacing any gap, not after a user supplies a candidate.

When the user pointed this out directly, Claude's assessment of its own behavior (independently, before being told what to conclude) matched the user's assessment exactly: the discovery work should have happened when the gap was found, not after the user went and found solutions independently.

Why this seems worth reporting

Both patterns look like instances of the same shape of problem: a plausible, well-formed response is generated and treated as sufficient, without an explicit check for whether it actually closes the specific gap or answers the specific question, and neither reasoning-model quality nor an increasingly detailed set of user-written standing rules has closed the gap — it just shifted where the next instance shows up. This suggests the fix isn't more rules for the assistant to remember, but some kind of structural verification step for exactly this shape of claim ("this specific thing is true / this specific thing is proactively covered") that doesn't rely on the model choosing to run it.

Context

  • Environment: Claude Code (Sonnet 5), long-running project session, Ultracode/heavy subagent-workflow usage enabled.
  • The user maintains a detailed dated incident ledger for this project going back months, which is where the three prior Pattern-1 instances are documented with fuller detail than summarized here.
  • Happy to provide more specific logs/timestamps if useful; redacted personal-library specifics above since the underlying pattern is general.

View original on GitHub ↗