[Model behavior] After context compaction, mandated memory/protocol files are not reloaded before the next task

We ran into this exact failure building a local-first memory layer for coding agents. After a compaction, the agent would resume from the summary and silently skip the protocol/memory files it was supposed to re-read first — same root cause as #101577.

Two design choices that fixed it for us: (1) memory lives in an explicit, queryable store, not only inside the prompt — on resume we re-hydrate from that store instead of trusting the compaction summary; (2) 're-read memory before the next task' is a hard, logged gate, not a convention the model is expected to remember — if skipped, the run refuses to start. (Disclosure: I'm one of the builders.)

Two halves to this. The reload is unenforced, and the manual recovery path collides with the reader's own caps, so even a user who notices can end up with nothing. Both are testable now.

After a compaction the resume turn carries the summary, the instruction files re-injected from disk, and the index. The topic files are cold, and nothing in that path requires reading them, which is exactly the gap you are seeing. Then "recall everything" as one file is unreadable: a scratch file past 256 KB refuses to open, and a dense file under that comes back as a partial view. Reproduced on 2.1.292 with a 580.7KB concatenation of an 81-file memory dir:

File content (580.7KB) exceeds maximum allowed size (256KB). Use offset and limit parameters to read specific portions of the file, or search for specific content instead of reading the whole file.

and under the byte cap, over the token estimate:

showing lines 1-242 of 340 total (29767 tokens, cap 25000)

So the fix has two parts: a way to measure whether the reads happened, and a way to make "read it all" a sequence of reads that fit.

Part 1: memory-receipts.py. Scans the session transcript, finds the last compact_boundary record, and lists which files linked from MEMORY.md have a Read after it. Main-context reads count, sidechain reads do not, and the index itself is skipped because it comes back on its own.

#!/usr/bin/env python3
"""memory-receipts: which indexed memory files were actually Read after the last compaction?

Scans a Claude Code session transcript for the last compact boundary and lists
which files linked from MEMORY.md have a Read tool call after that point.

Usage:
  memory-receipts.py [--dir MEMORY_DIR] [--transcript FILE] [--all]

  --dir         dir holding MEMORY.md + topic files (default: .claude/memory, then ~/.claude/memory)
  --transcript  session .jsonl (default: newest in ~/.claude/projects/<slug-of-cwd>)
  --all         check every transcript in the project dir instead of just the newest

Exit codes: 0 = all indexed files read after the boundary; 1 = missing files;
            2 = inputs problem; 3 = no compact boundary in transcript.
"""
import json, os, re, sys, glob

def slug(path):
    return re.sub(r'[^A-Za-z0-9]', '-', path)

def resolve_dir(arg):
    for cand in ([arg] if arg else ['.claude/memory', os.path.expanduser('~/.claude/memory')]):
        if cand and os.path.isfile(os.path.join(cand, 'MEMORY.md')):
            return os.path.abspath(cand)
    return None

def index_files(mem_dir):
    """files referenced from the index: markdown links, backticked paths, bare *.md tokens"""
    text = open(os.path.join(mem_dir, 'MEMORY.md'), encoding='utf-8', errors='replace').read()
    found = set()
    for m in re.finditer(r'\]\(\s*([^)\s]+\.md)\s*\)', text):
        found.add(m.group(1))
    for m in re.finditer(r'`([^`\s]+\.md)`', text):
        found.add(m.group(1))
    for m in re.finditer(r'(?m)(?<![\w./-])([A-Za-z0-9_][\w./-]*\.md)', text):
        found.add(m.group(1))
    out = {}
    for f in sorted(found):
        if os.path.basename(f) == 'MEMORY.md':
            continue  # the index itself is auto-injected; receipts cover the files it points to
        if os.path.isabs(f):
            p = f
        else:
            p = os.path.normpath(os.path.join(mem_dir, f))
        if os.path.isfile(p):
            out[os.path.realpath(p)] = f
    return out

def normalize_read_path(raw, base):
    if not raw:
        return None
    raw = os.path.expanduser(raw)
    if not os.path.isabs(raw):
        raw = os.path.join(base or '.', raw)
    real = os.path.realpath(raw)
    return real

def scan_transcript(tpath, expected):
    """returns (boundary_idx, boundary_ts, reads_after) where reads_after = list of realpaths"""
    last_boundary = None
    reads = []  # (line_no, raw_path, base_dir)
    n = 0
    with open(tpath, encoding='utf-8', errors='replace') as fh:
        for line in fh:
            n += 1
            if 'compact_boundary' in line:
                try:
                    rec = json.loads(line)
                except Exception:
                    continue
                if rec.get('subtype') == 'compact_boundary':
                    last_boundary = (n, rec.get('timestamp'))
                    reads = []  # reset: only reads after the LAST boundary count
                    continue
            if '"tool_use"' not in line or '"Read"' not in line:
                continue
            try:
                rec = json.loads(line)
            except Exception:
                continue
            if rec.get('type') != 'assistant' or rec.get('isSidechain'):
                continue
            msg = rec.get('message') or {}
            for block in (msg.get('content') or []):
                if isinstance(block, dict) and block.get('type') == 'tool_use' and block.get('name') == 'Read':
                    inp = block.get('input') or {}
                    reads.append((n, inp.get('file_path'), rec.get('cwd')))
    reads_after = []
    for (ln, raw, base) in reads:
        rp = normalize_read_path(raw, base)
        if rp:
            reads_after.append(rp)
    return last_boundary, reads_after

def main():
    args = sys.argv[1:]
    mem_arg = t_arg = None
    all_t = False
    while args:
        a = args.pop(0)
        if a == '--dir': mem_arg = args.pop(0)
        elif a == '--transcript': t_arg = args.pop(0)
        elif a == '--all': all_t = True
        else: print(f'unknown arg: {a}', file=sys.stderr); return 2

    mem_dir = resolve_dir(mem_arg)
    if not mem_dir:
        print('no MEMORY.md found; pass --dir', file=sys.stderr); return 2

    expected = index_files(mem_dir)
    if not expected:
        print(f'index has no resolvable .md links: {mem_dir}/MEMORY.md', file=sys.stderr); return 2

    if t_arg:
        transcripts = [t_arg]
    else:
        pdir = os.path.join(os.path.expanduser('~/.claude/projects'), slug(os.getcwd()))
        transcripts = sorted(glob.glob(os.path.join(pdir, '*.jsonl')), key=os.path.getmtime, reverse=True)
        if not all_t:
            transcripts = transcripts[:1]
    if not transcripts:
        print('no transcript found; pass --transcript', file=sys.stderr); return 2

    worst = 0
    for tpath in transcripts:
        boundary, reads_after = scan_transcript(tpath, expected)
        print(f'transcript: {tpath}')
        if not boundary:
            print('  no compact boundary in this transcript (nothing was compacted; the post-compact question does not apply)')
            worst = max(worst, 3); continue
        b_line, b_ts = boundary
        read_set = set(reads_after)
        hit = [exp for (rp, exp) in sorted(expected.items(), key=lambda kv: kv[1]) if rp in read_set]
        miss = [exp for (rp, exp) in sorted(expected.items(), key=lambda kv: kv[1]) if rp not in read_set]
        total = len(expected)
        print(f'  last compact boundary: line {b_line} at {b_ts}')
        print(f'  indexed files: {total}   read after boundary: {len(hit)}   missing: {len(miss)}')
        if hit:
            print('  READ after boundary (sidechain/subagent reads do not count):')
            for f in hit:
                print(f'    - {f}')
        if miss:
            worst = max(worst, 1)
            print('  MISSING (no Read after the boundary):')
            for f in miss:
                print(f'    - {f}')
        else:
            print('  all indexed files have a post-compact Read')
    return worst

if __name__ == '__main__':
    sys.exit(main())

Run against the current session's transcript, or a specific one, or the whole project dir:

python3 memory-receipts.py --dir ~/.claude/memory
python3 memory-receipts.py --dir ~/.claude/memory --transcript ~/.claude/projects/<slug>/<session-id>.jsonl
python3 memory-receipts.py --dir ~/.claude/memory --all

Trimmed output from a session where nothing protocol-shaped was read after its boundary:

transcript: /Users/.../c759ccc9-9029-4a40-bf96-de5b85bb2ef1.jsonl
  last compact boundary: line 32 at 2026-10-03T21:30:23.909Z
  indexed files: 2   read after boundary: 0   missing: 2
  MISSING (no Read after the boundary):
    - docs/process.md
    - docs/project-definition.md

And from a run where part of the set was read:

  indexed files: 81   read after boundary: 2   missing: 79
  READ after boundary (sidechain/subagent reads do not count):
    - topic-02.md
    - topic-03.md

That count is the number your rule is actually about, per segment, and it is the state a gate can key off when the reload stops being a convention.

Part 2: recall-bundles.py. Packs the memory dir into ordered bundles that each read whole, index order first, oversized files split with part markers, plus a manifest to read before the bundles. Deterministic, so bundle numbers stay stable across rebuilds.

#!/usr/bin/env python3
"""recall-bundles: pack a memory dir into read-sized bundles so "recall everything"
never produces one giant concatenation that fails the reader's size caps.

Claude Code's Read tool rejects files over 256 KB outright and paginates files
over its ~25k-token estimate; a naive `cat memory/*.md > scratch.md` collides
with both once a memory dir grows past a few hundred KB. This builds ordered
bundles sized under both caps, plus a manifest to read first.

Usage:
  recall-bundles.py [--dir MEMORY_DIR] [--out OUT_DIR] [--max-kb 64] [--max-lines 1500]

Notes:
  - file order follows MEMORY.md (index order first, then any extra *.md alphabetically)
  - a file larger than the cap is split across bundles with (part x/y) markers
  - deterministic: same input -> same bundle boundaries
  - defaults come from measuring the Read tool on 2.1.292: one file past 256 KB refuses
    to open; dense files past the ~25k-token estimate come back as a partial view.
    Lower --max-kb for unusually dense files (minified, tabular, single long lines).
"""
import os, re, sys, glob, hashlib

def index_order(mem_dir, files):
    try:
        text = open(os.path.join(mem_dir, 'MEMORY.md'), encoding='utf-8', errors='replace').read()
    except OSError:
        return sorted(files), {}
    order, desc = [], {}
    for m in re.finditer(r'\]\(\s*([^)\s]+\.md)\s*\)([^\n]*)', text):
        f = m.group(1)
        if f not in order and os.path.isfile(os.path.join(mem_dir, f)):
            order.append(f)
            tail = m.group(2).strip(' -:\u2014')
            if tail:
                desc[f] = tail[:120]
    for f in sorted(files):
        if f not in order:
            order.append(f)
    return order, desc

def split_file(text, cap_b, cap_l):
    """greedy split of a file into parts under caps; returns list of part texts"""
    parts, cur, cur_b, cur_l = [], [], 0, 0
    for ln in text.splitlines():
        b = len(ln.encode('utf-8')) + 1
        if cur and (cur_b + b > cap_b or cur_l + 1 > cap_l):
            parts.append('\n'.join(cur))
            cur, cur_b, cur_l = [], 0, 0
        cur.append(ln); cur_b += b; cur_l += 1
    if cur or not parts:
        parts.append('\n'.join(cur))
    return parts

def main():
    args = sys.argv[1:]
    mem_dir = '.claude/memory'
    out_dir = None
    max_kb = 64
    max_lines = 1500
    while args:
        a = args.pop(0)
        if a == '--dir': mem_dir = args.pop(0)
        elif a == '--out': out_dir = args.pop(0)
        elif a == '--max-kb': max_kb = int(args.pop(0))
        elif a == '--max-lines': max_lines = int(args.pop(0))
        else: print(f'unknown arg: {a}', file=sys.stderr); return 2
    mem_dir = os.path.abspath(mem_dir)
    if not os.path.isdir(mem_dir):
        print(f'no such dir: {mem_dir}', file=sys.stderr); return 2
    out_dir = os.path.abspath(out_dir or os.path.join(mem_dir, 'bundles'))

    all_md = [os.path.relpath(p, mem_dir) for p in glob.glob(os.path.join(mem_dir, '**', '*.md'), recursive=True)]
    skip_out = ''
    if out_dir.startswith(mem_dir + os.sep):
        skip_out = os.path.relpath(out_dir, mem_dir).replace(os.sep, '/') + '/'
    files = [f for f in all_md if not f.startswith('bundles/') and not f.startswith(skip_out) and f != 'MEMORY.md']
    order, desc = index_order(mem_dir, files)

    max_bytes = max_kb * 1024
    cap_b = int(max_bytes * 0.90)   # per part
    cap_l = int(max_lines * 0.90)

    chunks = []  # (file, part_idx, total_parts, chunk_text)
    for f in order:
        text = open(os.path.join(mem_dir, f), encoding='utf-8', errors='replace').read()
        parts = split_file(text, cap_b, cap_l)
        for pi, ptxt in enumerate(parts, 1):
            head = f'### FILE: {f}' + (f' (part {pi}/{len(parts)})' if len(parts) > 1 else '')
            chunks.append((f, head + '\n' + ptxt + '\n\n'))

    bundles, cur, cur_b, cur_l = [], [], 0, 0
    for f, chunk in chunks:
        cb, cl = len(chunk.encode('utf-8')), chunk.count('\n')
        if cur and (cur_b + cb > max_bytes or cur_l + cl > max_lines):
            bundles.append(cur); cur, cur_b, cur_l = [], 0, 0
        cur.append(chunk); cur_b += cb; cur_l += cl
    if cur:
        bundles.append(cur)

    os.makedirs(out_dir, exist_ok=True)
    for old in glob.glob(os.path.join(out_dir, 'bundle-*.md')):
        os.remove(old)

    manifest = ['# Memory bundles', '',
                'Read this manifest, then read bundle-01 onward in order.',
                'Each bundle is sized under the reader caps; do not concatenate them back into one file.', '',
                f'files: {len(order)} | bundles: {len(bundles)} | cap: {max_kb} KB / {max_lines} lines per bundle',
                '', '## Contents']
    for i, b in enumerate(bundles, 1):
        content = ''.join(b)
        fns = [l[len('### FILE: '):] for l in content.splitlines() if l.startswith('### FILE: ')]
        kb = (len(content.encode('utf-8')) + 1023) // 1024
        manifest.append(f'- {i:02d} bundle-{i:02d}.md  ({kb} KB, {content.count(chr(10))} lines)  files: {", ".join(fns)}')
    manifest += ['', '## Files'] + [f'- {f}' + (f'  -- {desc[f]}' if f in desc else '') for f in order]
    open(os.path.join(out_dir, 'manifest.md'), 'w', encoding='utf-8').write('\n'.join(manifest) + '\n')

    for i, b in enumerate(bundles, 1):
        open(os.path.join(out_dir, f'bundle-{i:02d}.md'), 'w', encoding='utf-8').write(''.join(b))

    print(f'wrote {len(bundles)} bundles + manifest to {out_dir}')
    for i, b in enumerate(bundles, 1):
        content = ''.join(b)
        print(f'  bundle-{i:02d}.md  bytes={len(content.encode("utf-8"))} lines={content.count(chr(10))}')
    return 0

if __name__ == '__main__':
    sys.exit(main())

python3 recall-bundles.py --dir ~/.claude/memory

On the test dir (81 topic files, 594,628 bytes) it writes 10 bundles plus the manifest, largest bundle 65,184 bytes. I read all ten through the Read tool on 2.1.292 and every one came back whole, no pagination note. The same reader paginates 92-96 KB bundles, which is why the cap defaults to 64 KB / 1,500 lines; lower it if your files are denser than prose.

Loop after a compaction: run receipts, and if the missing list is non-empty, read the manifest then the bundles in order. The summary stays what it is, the files come back through reads that fit, and the count tells you when they did not.

The question is why people who aren’t affiliated with Anthropic are proposing solutions in my issues when these are problems that Anthropic itself should be addressing....

@John-Rood @qianqiuwanzi, I appreciate the input, but these issues need to be addressed by Anthropic, not by unaffiliated individuals. I’ll continue reporting them, as I already have 100+ open issues awaiting attention from the people responsible for the product