Auto-memory at heavy scale: MEMORY.md size budget forces working sessions into constant index-curation (55-byte trims mid-task)
Auto-memory at heavy scale: the always-loaded MEMORY.md budget turns every memory write into an index-curation tax on working sessions
Environment
Claude Code (macOS), single project used daily for ~1 year with auto-memory enabled.
Scale data point
- 1,200+ memory fact files (~26 MB) in one project's
memory/directory MEMORY.mdindex pinned at its ~17 KB auto-load budget for weeks- ~8–10 new memories written per day at peak (≈2 KB/day of index growth)
The problem
The memory instructions tell the model to add a one-line pointer to MEMORY.md after every memory write, and MEMORY.md has an effective size budget because it auto-loads every session. At heavy usage those two rules collide: the index reaches the budget and stays pinned there, so every subsequent memory write forces the working session to stop mid-task and hand-trim the index.
Observed verbatim in a session transcript today, mid-task:
"MEMORY.md went 55 bytes over cap — trimming my own line."
That session was doing product work; it detoured into index prose-surgery over 55 bytes. Across sessions this compounds — we counted nine hand-compactions in sixteen days before building workarounds. The curation tax lands on the most expensive resource (an in-flight session with the user's real task in context) instead of on a batch process.
Feature requests (any subset would help)
- Native index lifecycle. Harness-side eviction/aging for
MEMORY.mdentries (TTL for dated entries, archive-not-delete), so the model isn't the garbage collector. - Hysteresis, not a hard ceiling. Treat the budget as a high-water mark with tolerance; when compaction happens, compact to a low-water mark so headroom is restored in one pass instead of 55-byte trims every session.
- Move compaction off the critical path. Let index maintenance run out-of-session (background/scheduled), never as an interruption to a working session.
- Usage telemetry for memories. Expose which memory files were actually recalled/read per session, so eviction can be evidence-based (hit-count ranking) rather than judgment calls.
- Relevance-scoped index loading. If the harness can load only the index sections relevant to the session's early context, the always-loaded surface shrinks and the byte budget stops being the binding constraint.
Workaround we built (kept harness-native on purpose)
A PostToolUse hook + daily launchd "custodian" that (a) makes working sessions append-only — the hook never asks a session to trim; (b) ages dated index lines out after 21 days to an archive file; (c) watermark-compacts 16.5 KB → 13.5 KB by moving whole lines to the archive. We deliberately kept the stock format (one fact file + frontmatter, MEMORY.md as index) so future harness-side memory improvements land cleanly — which is also why we'd prefer this class of maintenance to move into the product.
3 Comments
Third site with the same failure, and I want to add a measurement that I think changes what the
fix should be — because the obvious remedy ("curate better", "prune the store") is aimed at the
wrong half.
Scale here
MEMORY.mdindexes: median 174 words, largest 2,131Smaller than your 1,200 files, and the tax lands identically, which is the point — this isn't a
1,200-file problem, it starts much earlier.
The measurement: the store is not what costs you
I instrumented session startup across 24 projects (20 of which have a
MEMORY.md) to find outwhat actually drives startup context.
MEMORY.mdwas the single strongest predictor —r = +0.76, against a significance floor of about 0.40 at that n, and ahead of
CLAUDE.md's+0.73. Cost is linear in index size at roughly 108 tokens fixed + 0.44 tokens/byte — that
slope is fitted from two ablation points and then validated across all 24, so treat it as a good
estimate rather than a precise constant. The probe itself is deterministic: three repeat runs on
each of two projects gave a spread of 0 tokens.
The part worth the thread's attention: at session start, only the always-on index costs
anything. The sibling fact files weren't distinguishable from zero, which follows from the
documented behaviour — the index auto-loads, the fact files are read on demand. So your 26 MB is,
at startup, approximately free, and every session that stopped to trim 55 bytes off the index was
paying a real tax to protect a budget the 26 MB never threatened.
That reframes the request. "Raise the cap" and "prune the store" are both answers to a size
problem. The actual problem is that one always-on file is doing two jobs: it is the routing
table and it is the thing under budget pressure, so every write to the store has to negotiate
with the budget. Decouple those and the mid-task curation disappears without changing any limit.
What worked here
Two rules, both cheap:
Explicitly no cap on the fact files, because they've been measured at ~zero.
from. 6.7% of my files are over 800 words and they are almost entirely transcripts of single
sessions — my largest is literally named
project_backlog_execute_session_20260627.md. Everyone of those bought an index line that will never be true again. The rule that fixed it: if
what you're about to save is a narrative of what happened this session, it belongs in the
repo, not in memory.
The second rule did more than the first. The index stopped approaching the cap once run logs
stopped becoming facts, and at that point the 200-line limit was no longer binding.
On the product side
The behaviour that produces your "55-byte trim mid-task" is the instruction to append a pointer
to
MEMORY.mdon every write. That coupling is the whole bug, and it's fixable without touchingthe limit: if the index could be regenerated from the fact files' front-matter rather than
hand-appended, no working session would ever need to edit it. That's the shape I'd argue for
ahead of a configurable cap (#79217) — a bigger index still loads in full, every session, forever.
One thing I'd also want, orthogonal to size: a session cannot currently tell whether its index
loaded whole, truncated, or not at all (#82056). A pinned-at-cap index is exactly the condition
where that matters, because truncation and "no such rule exists" are indistinguishable from
inside the session.
1,200 files and 26 MB is the point where the memory becomes a second codebase nobody owns. The curation tax you name is what killed every version of this I tried on disk: writing is cheap, keeping the index true is not, and the index is what decides whether anything ever gets read back.
What removed the tax for us was dropping the index entirely. Entries are atomic and self-describing, one closed thing each, with an author and a date, and retrieval is a query rather than a path somebody has to maintain. There is nothing to re-organize as it grows, which is the only property that survives 1,200 entries.
Your ask #4 (usage telemetry driving eviction) and "curation off the critical path" are the two design decisions that fixed this for me: curation only runs at milestones (tag / merge / wrap-up), never mid-task, and every retained line carries a hit counter that protects it from displacement. It's deliberately a ritual, not a hook. https://github.com/hemangnagar/retro if you want to steal the eviction rule.