auto-memory entries activate reactively after pushback, not proactively before claims
Summary
Auto-memory feedback entries that explicitly say "verify before claiming X" only fire after the user corrects me, not before the original claim. The result: safeguards function as apology generators rather than gatekeepers.
Behavior observed
In a single session troubleshooting a MikroTik VLAN setup, I had three relevant memory entries loaded:
feedback_validate_not_assume— verify with data, don't speculatefeedback_no_product_specs_from_recall— WebSearch hardware/product capabilities firstfeedback_verify_safety_mechanism— read platform docs before designing safety nets
Despite all three being in context, I made multiple confident-but-wrong recommendations from training-data recall. Examples from one session:
- Suggested "switch 2.4GHz radio to 802.11ax for IoT compatibility" — vendors (ASUS, others) actually recommend the opposite: disable ax on 2.4GHz for ESP8266/Tuya compatibility. ESP-based IoT devices choke on RTS/CTS and WMM that 802.11ax requires.
- Suggested "set channel to auto" — useless when WiFi-side scan was already clean (2 distant neighbors) and the actual noise was non-WiFi (Bluetooth, ZigBee, USB3) which auto-channel logic cannot see.
- Earlier, designed a wall-clock-anchored auto-rollback on a MikroTik hAP ax³ — a device with no RTC battery that loses wall-clock on every cold boot. Documented in MikroTik's own product specs. Would have been caught by reading the docs.
Each time, only after the user pushed back did I run a WebSearch / read docs that immediately surfaced the correct answer.
The structural gap
Memory loads passively into the model's context, but the response-formation policy does not treat factual / platform-spec / safety claims as a trigger to re-read and apply the relevant memory entries before emitting the claim. So:
- The rule ("validate first") is correctly recorded.
- The activation point is wrong (post-correction, not pre-claim).
- Users effectively train the same correction repeatedly — the entry exists, the lesson is documented, and yet the next analogous claim still ships unvalidated.
Why this matters
The auto-memory system is positioned as "the model learns and adapts to the user." In practice, for the most important class of corrections — "don't speculate, verify" type entries — the learning is asymmetric: corrections accumulate in memory but do not change first-response behavior. Trust degrades because users see the same failure with the same apology cycle.
Cost-of-failure example: the same model is routinely used in parallel windows for high-stakes work — production code review, trading systems, security configurations, infrastructure migrations. A pattern of "confident-recall first, validate only after pushback" that's merely annoying on an RF question is catastrophic on financial code or a database migration. The asymmetry — that documented "verify first" lessons don't fire pre-emptively — degrades trust across all uses, not just the one where the failure was observed.
Suggested direction
Not prescriptive — options worth exploring:
- At response-formation time, scan loaded feedback entries for ones whose
descriptionmatches the type of claim about to be made (factual / hardware / platform / safety) and force a verification step before emitting. - A class of memory entries marked as "pre-claim gate" rather than "general guidance" — ones that block confident emission until the verification action specified in the memory has run in the current turn.
- Telemetry on "memory entry surfaced after correction vs. before claim" so this gap is measurable.
Repro
Any session where the user has a feedback_* memory entry of the form "don't recall, verify X via Y first" and then asks a question that would invite an X-type recall. Empirically: model still recalls; user still corrects; memory entry fires only in the apology.
Environment
- Claude Code CLI, model
claude-opus-4-7 - Auto-memory directory under
~/.claude/projects/.../memory/withMEMORY.mdindex - macOS Darwin 24.6.0
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗