[BUG] Text block streamed between two thinking blocks is never written to the session JSONL (silent loss of user-facing replies; 3,738 across two machines since v2.1.170)
Summary
In Claude Code, when a single streamed assistant response has the content-block
shape [thinking][text][thinking][tool_use…] (interleaved thinking around a
mid-turn text block, followed by tool use), the text block is never written
to the session transcript JSONL. The sibling thinking records persist, the
tool_use records persist and the tools execute — but the user-facing prose is
silently absent from the record. It therefore never appears on any surface
rendered from the transcript: remote-control clients (desktop app bridge),
resumed sessions, --print/transcript exports, or any transcript-based audit.
The text exists only in the live model context until the session ends, then is
gone permanently.
Environment
- Claude Code 2.1.225 (signature also present in transcripts written by
2.1.170, 2.1.202, 2.1.207, 2.1.210, 2.1.211, 2.1.212, 2.1.215, 2.1.218,
2.1.220, 2.1.222, 2.1.224 — see Scope)
- Windows 11 Pro 10.0.26200 (PowerShell/wezterm host; also observed via
remote-control bridge to desktop apps on Windows and macOS)
- Model: claude-fable-5 with interleaved thinking (the response shape that
triggers the bug requires thinking blocks around a text block)
Reproduction / detection signature
The transcript writes one JSONL record per content block of a streamed
response; records of one response share requestId and message.id.
- Kept responses persist as
[thinking][text] → tool_use…or
[thinking][text] (end_turn): one thinking record, text intact.
- Eaten responses persist as **two adjacent
thinkingrecords sharing one
message.id**, followed by the tool_use record(s). The text block that was
streamed between the two thinking blocks has no record at all.
Confirmed live twice in one session (2026-08-09): the assistant streamed a
substantive mid-turn answer, then a thinking block, then tool calls; the user
could not find the text on any surface; the JSONL shows[thinking][thinking][tool_use] for that message.id
(examples: message.id …oxt4w3S9 / requestId …T4rosA1g, and …YvLfnhoS /
…pBkUD74E). The assistant subsequently quoted both lost texts verbatim from
live context, confirming they were generated and streamed.
Detection rule for scanning any transcript: within consecutive records, flag a
response where two adjacent assistant records share message.id and both havecontent[0].type == "thinking" (records of passive types — queue-operation,
bridge-session, attachment, etc. — may interleave without breaking adjacency).
Scope measured on one user's machine
Scanning all 1,666 session JSONLs under ~/.claude/projects/ on one Windows
machine (2026-08-09). Method: group assistant records by message.id
(sidechain/subagent records counted separately); flag a response whose group
has >= 2 thinking-bearing records and ZERO text-bearing records, checking
text presence across every block of every record (3 multi-block assistant
records exist fleet-wide; the check covers them).
Windows machine (1,666 transcripts):
- 3,017 flagged responses in main (user-facing) context across **24
transcripts, plus 87 more inside subagent sidechains**
- Earliest: 2026-06-09 (CC 2.1.170); continuous through 2.1.225
- Monthly (main context): 2026-06: 11 · 2026-07: 1,973 · 2026-08: 1,033
macOS machine (same user, scanned 2026-08-09):
- 721 flagged responses in main context across 13 transcripts, plus
81 in subagent sidechains
- Earliest: 2026-07-05; versions 2.1.201, 2.1.202, 2.1.205, 2.1.206,
2.1.218, 2.1.220, 2.1.222 — four versions not represented in the Windows
data, so the two machines jointly cover a near-continuous version range.
Fleet total: 3,738 user-facing eaten responses since 2026-06-09.
Cross-validation (both machines): zero flagged responses also persisted
a text block for the same message.id — no false-positive shape was
observed; the API does not emit back-to-back thinking blocks in one
response.
Each flagged response is (at least) one streamed assistant text block absent
from the record.
Impact
- Users reading via remote-control (desktop app bridge) or resumed sessions
silently miss mid-turn assistant prose; the assistant believes it delivered
the content (it did stream it) — repeated trust damage on both sides.
- Transcript-based tooling (audits, sweeps, TTS hooks reading the transcript,
compaction summaries built from the file) operates on a record that is
missing content; conclusions drawn from it under-count what was said.
- The loss is unrecoverable once the live context is gone.
Local mitigation we deployed (workaround)
One script (guard_eaten_text.py, inlined below; stdlib-only, Windows + macOS)
registered as two hooks:
- PostToolUse arm (
--post): after each tool call, incrementally scans
the records appended to the transcript since its last fire (byte-offset
state per session; per-message.id counts carry across scan windows
because transcript_path is documented to lag the conversation). On a
new eaten-response signature it exits 2, which feeds a short stderr note
to the model mid-turn naming the affected message.id(s). The model then
weaves the lost content into the turn's final text message — which is
a shape the bug does not affect — and stamps a [recovered: N] footer.
Recovery works because the lost text still exists in live model context.
- Stop arm: mechanically verifies the final message carries a covering
[recovered: N] footer for every id surfaced mid-turn (blocks the stop
once if not — bounded by a terminating consecutive-block counter), and
silently collects signatures the post arm could not see (writer lag; an
eaten text in the turn's final response, which has no tool call after it
to hook) for delivery into the NEXT turn's weave. Flagged responses are
never re-flagged; subagent sidechain records are skipped; on first fire
only the transcript's final turn is eligible, so unrecoverable history is
never demanded; fail-open on any error; block/surface/verify events are
telemetried for false-positive review.
The hook proved itself while being built: armed hot mid-session, the
original Stop-only version recovered ~20 eaten messages across one evening,
and the PostToolUse arm caught an eaten text from its own build turn on its
first live fire.
This converts "silently gone forever" into "delivered in the same turn's
final message," but it is a workaround: the transcript remains missing the
original block, and only sessions with the hook installed are protected.
What this mitigation does NOT do: no chain-of-thought access
Explicitly, because the mechanism sits near thinking blocks: nothing here
reads, extracts, restates, or distills thinking content.
- The detector inspects only each record's block type labels and
message metadata: the load-bearing line is
kinds = {b.get("type") for b in blocks} — it looks at
content[].type ("thinking"/"text"/"tool_use"), message.id, and
the timestamp. No block body of any kind is ever read, stored, or
transmitted by the scanner or the hook.
- What is recovered is exclusively the assistant's **user-facing text
block — prose that was already streamed to the user's own screen in
that same session and that the transcript was supposed to persist. The
hook asks the assistant to restate its own already-delivered reply**,
nothing else; the instruction text (attached source) never references
thinking content.
- The thinking records themselves are written to the user's local
transcript by Claude Code itself, verbatim, as part of normal operation;
this tooling adds zero access beyond counting how many records of each
type exist.
The signature ("two thinking records, zero text records for onemessage.id") is a presence/absence test used to detect a missing
user-facing block — the thinking records are only the shadow that reveals
the text block was dropped.
Attachments (inlined)
eaten_text_scan.py— the fleet survey tool that produced the Scope
numbers. Stdlib-only; run python3 eaten_text_scan.py (or py on
Windows); scans every *.jsonl under ~/.claude/projects/ and reports
flagged responses per transcript, main-context vs sidechain, by month
and CC version. Any affected user can reproduce their own loss count
with it.
guard_eaten_text.py— the two-arm mitigation hook described above.
Stdlib-only, self-contained (no wrapper required). Wiring in
~/.claude/settings.json:
``json``
"hooks": {
"PostToolUse": [
{ "matcher": ".*", "hooks": [ { "type": "command",
"command": "python3 ~/.claude/hooks/guard_eaten_text.py --post",
"timeout": 10 } ] }
],
"Stop": [
{ "hooks": [ { "type": "command",
"command": "python3 ~/.claude/hooks/guard_eaten_text.py",
"timeout": 10 } ] }
]
}
For anyone rebuilding from scratch instead: group assistant records by
message.id; flag groups with >= 2 thinking-bearing records and zero
text-bearing records (checking every block of every record, skipping
isSidechain); surface new flags to the model mid-turn via a
PostToolUse exit-2 stderr note so it can carry the content into its
final message; verify at Stop that the final message acknowledges the
surfaced ids; bound any blocking with a terminating counter and fail
open on every error.
Ask
Persist every streamed content block to the session JSONL regardless of
position — specifically the text block(s) of responses shaped[thinking][text][thinking][tool_use…]. If helpful we can provide
record-level line numbers from affected transcripts.
<details>
<summary><b>eaten_text_scan.py</b> — fleet survey tool (measure your own losses)</summary>
"""Fleet survey of the Claude Code eaten-text persistence bug.
The bug (identified 2026-08-09, PHAROTH; see
~/.claude/scratch/eaten-text-bug-report.md and the guard_eaten_text.py Stop
hook): a streamed response shaped [thinking][text][thinking][tool_use...]
never persists its TEXT block to the session JSONL.
Signature: group assistant records by message.id; a response persisting >= 2
thinking-bearing records and ZERO text-bearing records is a loss. Text
presence is checked across ALL blocks of each record (multi-block records
exist). Sidechain (subagent) hits are counted separately — they are real
losses but not user-facing.
Run on any machine: python3 ~/.claude/tools/eaten_text_scan.py
Re-run after every Claude Code update to see whether upstream fixed it.
"""
import json
from collections import defaultdict
from pathlib import Path
ROOT = Path.home() / ".claude" / "projects"
per_month = defaultdict(int)
main_hits = 0
side_hits = 0
multiblock = 0
sessions = set()
earliest = None
version_hits = defaultdict(int)
top = defaultdict(int)
for jl in ROOT.rglob("*.jsonl"):
groups = {} # (sidechain, mid) -> [n_think, n_text, ts, version]
try:
f = open(jl, encoding="utf-8")
except OSError:
continue
with f:
for line in f:
try:
r = json.loads(line)
except json.JSONDecodeError:
continue
if r.get("type") != "assistant":
continue
msg = r.get("message") or {}
mid = msg.get("id")
blocks = msg.get("content")
if not mid or not isinstance(blocks, list):
continue
real = [b for b in blocks if isinstance(b, dict)]
if len(real) > 1:
multiblock += 1
kinds = {b.get("type") for b in real}
key = (bool(r.get("isSidechain")), mid)
g = groups.setdefault(
key, [0, 0, r.get("timestamp", ""), r.get("version", "")])
if "thinking" in kinds:
g[0] += 1
if "text" in kinds:
g[1] += 1
for (side, mid), (t, x, ts, ver) in groups.items():
if t >= 2 and x == 0:
if side:
side_hits += 1
continue
main_hits += 1
sessions.add(jl.name)
per_month[ts[:7]] += 1
top[str(jl.relative_to(ROOT))[:70]] += 1
if ver:
version_hits[ver] += 1
if ts and (earliest is None or ts < earliest):
earliest = ts
print(f"MAIN-context eaten responses: {main_hits} in {len(sessions)} transcripts")
print(f"SIDECHAIN (subagent) eaten responses: {side_hits}")
print(f"multi-block assistant records seen: {multiblock}")
print(f"earliest main hit: {earliest}")
print("\nmain hits by month:")
for m in sorted(per_month):
print(f" {m}: {per_month[m]}")
print("\nmain hits by CC version:")
for v in sorted(version_hits):
print(f" {v}: {version_hits[v]}")
print("\ntop affected transcripts:")
for name, n in sorted(top.items(), key=lambda kv: -kv[1])[:10]:
print(f" {n:5d} {name}")</details>
<details>
<summary><b>guard_eaten_text.py</b> — the two-arm mitigation hook</summary>
"""Two-arm guard against assistant text blocks the CLI never persisted.
WHY THIS EXISTS
---------------
Claude Code (observed on 2.1.225; fleet survey found the signature back to
2.1.170 / 2026-06-09 — method and numbers in
~/.claude/scratch/eaten-text-bug-report.md) has a persistence bug: when a
streamed response interleaves thinking around text —
[thinking][text][thinking][tool_use...] — the sandwiched TEXT block is never
written to the session JSONL. The sibling thinking records persist, the tool
calls persist and execute, but the prose the user was meant to read is gone
from the record: invisible to remote surfaces, to resumed sessions, and to
any transcript-based audit.
TWO ARMS (Craig-ruled 2026-08-09: weave, don't block-and-restate)
-----------------------------------------------------------------
The same script serves two hook events, selected by the `--post` flag.
POST ARM (PostToolUse, `--post`): runs the incremental scan after each tool
call. On a new signature it exits 2 with a terse note naming the affected
ids — PostToolUse exit-2 cannot block (the tool already ran); its stderr is
fed to Claude mid-turn (docs-verified AND probe-verified 2026-08-09; renders
on the CC app as a collapsed "Hook re-prompted Claude" line). Claude then
WEAVES the lost substance into the turn's final message where it naturally
belongs and stamps the footer `[recovered: N]`. Surfaced ids are recorded in
state for the stop arm to verify.
STOP ARM (Stop, no flag): two jobs.
1. MARKER TRIPWIRE — for ids the post arm surfaced this turn, mechanically
verify the final message (`last_assistant_message`) carries a
`[recovered: N]` footer with N covering them; block once if not. This is
plumbing against the weave being forgotten, not trust in it. The footer
may sit inline at the end of the last paragraph (Craig-ruled: no
newline required; the check is tail-scoped, not line-scoped).
2. STRAGGLER COLLECTION — signatures the post arm could not see (a
transcript writer lagging past the last tool call, or an eaten text in
the turn's FINAL response, which has no tool call after it to hook) are
collected SILENTLY into `carryover`, along with surfaced ids from an
aborted prior turn. The next post-arm fire delivers them for weaving
into THAT turn's final message, where they enter the normal
footer-verified flow. No post-turn recovery block exists any more
(Craig-ruled 2026-08-10 after watching all three paths fire live: the
weave replaces the recovery-recap genre; a one-turn delivery delay
costs less than an extra blocked turn). Accepted residual: ids
collected on a session's very last turn are never delivered — they
remain in state and telemetry (`carryover` events), where the fleet
scanner still counts the loss.
PROMPT PARTITION (load-bearing assumption, stated)
--------------------------------------------------
`prompt_id` is a documented common stdin field on all hook events and was
present in the post-arm probe's captured payload (2026-08-09). The stale
split assumes post and stop fires of one turn carry the SAME value; that is
docs-implied, not yet live-verified — the first armed session verifies it
via blocks.jsonl (a `surface` and a later `marker-ok` sharing a prompt
value). Degradation is graceful in both failure directions: an empty
prompt_id keeps every surfaced entry current (fail-toward-verification),
and a value mismatch routes entries to carryover delivery (fail-toward-
re-weave) — neither direction drops an id.
The footer count is the user's thinness signal: `[recovered: 3]` over a thin
summary invites "show me more" while the texts are still in live context.
If the mid-turn re-prompt line proves noisy on wezterm, the designated
escape is switching the post arm to exit-0 JSON
`hookSpecificOutput.additionalContext` (docs-verified available) — one
function changes, no architecture moves.
DETECTION
---------
The transcript writes records per content block; records of one streamed
response share `message.id`. A response that persisted two or more
thinking-bearing records and ZERO text-bearing records is the bug's residue
(survey: zero flagged responses fleet-wide also persisted text; the API does
not emit consecutive thinking blocks with nothing between them). Grouping by
message.id (not record adjacency) needs no record-type whitelist to rot.
Text presence is checked across ALL blocks of a record — multi-block records
exist (3 fleet-wide) and must not masquerade as eaten. Because the
transcript is written asynchronously and may lag the conversation
(documented), per-mid counts carry across scan windows (merged, capped), so
a response whose records straddle fires is still caught. A flagged mid is
never flagged twice while it remains in the retained list (FLAGGED_CAP most
recent; preserved across re-baselines — message.ids are globally unique).
`isSidechain` records are skipped: subagent conversations live in the parent
JSONL, their losses are not the parent's to restate, and flagging them would
prompt the parent to confabulate a restatement it never authored. Both arms
skip subagent fires (`agent_id`).
FIRST FIRE / RE-BASELINE
------------------------
With no state, a different transcript behind the same session id, a
mismatched content anchor (truncate-then-regrow), or an out-of-range
offset: scan only the transcript tail (last TAIL_SCAN bytes) and flag only
responses in the FINAL turn (records after the last genuine user message).
The final turn is the only region whose prose is guaranteed to be in live
context — earlier history may predate a compaction or belong to a resumed
session, and an eaten block was never persisted, so a rebuilt context
definitionally cannot contain it. This catches turn-1 losses in brand-new
sessions and the current turn of adopted long transcripts, without ever
demanding unrecoverable history. Known limitation: an injected user-typed
record (hook feedback, non-isMeta notices) landing MID-response looks like
a turn boundary on this path and voids that one detection — fail-toward-
miss, first-fire only; the incremental path never consults boundaries.
Re-baseline resets the byte offset and carry only; `carryover`,
`surfaced`, and the flagged list survive it (a stashed detection must not
be dropped, and an already-woven response must not be demanded twice).
LOOP BOUND (TERMINATING)
------------------------
Only the footer tripwire blocks, so the consecutive-block counter governs
it alone: it increments on each footer block and resets ONLY on a fire
with no failed marker check. At MAX_CONSEC the guard stops blocking and
keeps the unverified ids in `surfaced` (they go stale at the next prompt
and ride carryover delivery). It cannot sawtooth, and it cannot keep
resetting the harness's independent 8-consecutive-block cap. A marker
check that CANNOT run (missing/empty `last_assistant_message`) fails open
with telemetry, never into a block loop.
LIVENESS
--------
Fail-open guards die silently, so the state file doubles as the heartbeat:
`fires` counts every scanning run (early exits — subagent fires, missing
transcript — do not count) and the file's mtime is the last run. Consumer
procedure: newest ~/.claude/scratch/eaten-text-guard/<session>.json should
be no older than the last active session's turns; zero blocks with a stale
state dir is a dead instrument, not a quiet one. Surfacings, blocks, and
footer verdicts append timestamped records to blocks.jsonl.
FAIL-OPEN
---------
Unparseable stdin, missing transcript, or any internal exception exits 0
with a one-line stderr note. A guard that traps a session is worse than the
failure mode it polices.
Stdlib only. Cross-platform (invoked via `py` on Windows, `python3` on Mac).
"""
from __future__ import annotations
import json
import os
import re
import sys
import time
STATE_DIR = os.path.join(os.path.expanduser("~"), ".claude", "scratch",
"eaten-text-guard")
CONTRACT = "memory-global/eaten-text-persistence-bug.md"
# First-fire tail window: large enough to hold any final turn, small enough
# to keep the one-time scan of a 40MB+ adopted transcript cheap.
TAIL_SCAN = 4_000_000
# Per-fire scan bound: guarantees the offset advances every fire even when
# catching up after missed fires (measured: 8.5MB real transcript scans in
# ~0.1s, so this bound is far below the 10s hook timeout).
MAX_WINDOW = 8_000_000
# Consecutive footer-block bound (termination; ids kept, never lost).
MAX_CONSEC = 2
# Bounds on carried per-mid counts, remembered flagged mids, carryover,
# and surfaced-awaiting-footer entries.
CARRY_CAP = 200
FLAGGED_CAP = 50
PENDING_CAP = 20
# Content anchor: hash of the first line (up to this many bytes). A fixed
# short prefix would NOT identify the file — JSONL records share long
# structural prefixes ('{"type": "user", ...'), so byte-40 heads collide.
ANCHOR_LEN = 4096
# The weave footer. Count is cumulative for the turn; the check accepts the
# LARGEST count found so a message quoting an older footer cannot shrink it.
# Checked only in the message TAIL: the contract places the footer at the
# end, and tail scope stops a footer QUOTED mid-prose (this very file, a
# transcript excerpt, a blocks.jsonl line) from clearing surfaced ids.
MARKER_RE = re.compile(r"\[recovered:\s*(\d+)")
MARKER_TAIL = 300
def load_state(path: str) -> dict:
try:
with open(path, encoding="utf-8") as fh:
s = json.load(fh)
return s if isinstance(s, dict) else {}
except (OSError, ValueError):
return {}
def save_state(path: str, state: dict) -> None:
os.makedirs(os.path.dirname(path), exist_ok=True)
tmp = f"{path}.{os.getpid()}.tmp"
with open(tmp, "w", encoding="utf-8") as fh:
json.dump(state, fh)
os.replace(tmp, path)
def log_event(session: str, kind: str, **fields) -> None:
try:
with open(os.path.join(STATE_DIR, "blocks.jsonl"), "a",
encoding="utf-8") as fh:
fh.write(json.dumps({
"ts": time.strftime("%Y-%m-%dT%H:%M:%S%z"),
"session": session, "kind": kind, **fields,
}) + "\n")
except OSError:
pass
def read_anchor(transcript: str) -> str:
import hashlib
with open(transcript, "rb") as fh:
head = fh.read(ANCHOR_LEN)
return hashlib.sha1(head.split(b"\n", 1)[0]).hexdigest()
def is_genuine_user(r: dict) -> bool:
"""A turn boundary: a real user message (not a tool_result batch, not a
sidechain, not injected meta context). Only the first-fire path consults
this; misclassification there fails toward a narrower flag window (a
miss), never a false demand."""
if r.get("type") != "user" or r.get("isSidechain") or r.get("isMeta"):
return False
content = (r.get("message") or {}).get("content")
if isinstance(content, str):
return True
if isinstance(content, list):
return not any(isinstance(b, dict) and b.get("type") == "tool_result"
for b in content)
return False
def scan_window(transcript: str, offset: int, carry: dict,
already_flagged: list, final_turn_only: bool,
) -> tuple[list, int, dict]:
"""Scan complete lines in [offset, min(EOF, offset+MAX_WINDOW)).
Returns (hits, new_offset, new_carry). A hit is "mid@hh:mm:ss" for a
message.id group with >=2 thinking-bearing records and zero text-bearing
records, not previously flagged. With final_turn_only (first fire /
re-baseline), groups reset at every genuine user record so only the
final turn can flag.
"""
with open(transcript, "rb") as fh:
fh.seek(offset)
buf = fh.read(MAX_WINDOW)
end = buf.rfind(b"\n")
if end < 0:
return [], offset, carry
new_offset = offset + end + 1
groups: dict[str, list] = {} # mid -> [thinking_recs, text_recs, hh:mm:ss]
for raw in buf[:end + 1].split(b"\n"):
if not raw.strip():
continue
try:
r = json.loads(raw)
except ValueError:
continue
if final_turn_only and is_genuine_user(r):
groups = {}
continue
if r.get("type") != "assistant" or r.get("isSidechain"):
continue
msg = r.get("message") or {}
mid = msg.get("id")
blocks = msg.get("content")
if not mid or not isinstance(blocks, list):
continue
kinds = {b.get("type") for b in blocks if isinstance(b, dict)}
if mid not in groups:
prev = carry.get(mid)
base = [int(prev[0]), int(prev[1])] if (
isinstance(prev, (list, tuple)) and len(prev) >= 2) else [0, 0]
groups[mid] = base + [(r.get("timestamp") or "")[11:19]]
if "thinking" in kinds:
groups[mid][0] += 1
if "text" in kinds:
groups[mid][1] += 1
hits = [f"{mid}@{ts}" for mid, (t, x, ts) in groups.items()
if t >= 2 and x == 0 and mid not in already_flagged]
# Merge groups over the old carry (a mid can skip a window while its
# writer lags), then trim oldest entries to the cap. Pop-and-reinsert
# so an actively-straddling mid moves to the young end — dict merge
# alone keeps a twice-seen key at its ORIGINAL slot, where the cap
# trim could evict it mid-straddle.
for m in groups:
carry.pop(m, None)
merged = {**carry, **{m: g[:2] for m, g in groups.items()}}
if len(merged) > CARRY_CAP:
merged = dict(list(merged.items())[-CARRY_CAP:])
return hits, new_offset, merged
def short_ids(entries: list) -> str:
out = []
for h in entries:
mid, _, ts = h.partition("@")
out.append(f"..{mid[-8:]}" + (f" {ts[:5]}" if ts else ""))
return ", ".join(out)
def marker_count(text: str) -> int:
"""Largest [recovered: N] in the message tail; 0 if none."""
return max((int(m) for m in MARKER_RE.findall(text[-MARKER_TAIL:])),
default=0)
def run(post_mode: bool) -> int:
try:
# Bytes, not text: a text read of a Windows pipe decodes cp1252 and
# mojibakes non-ASCII. json.loads on bytes decodes UTF-8 per RFC 8259.
payload = json.loads(sys.stdin.buffer.read())
except (ValueError, OSError, AttributeError):
return 0
# agent_id marks a subagent's fire (empirical: saylive_stop_hook.py keys
# on the same field; docs route subagents to SubagentStop, so this is
# belt-and-suspenders over the isSidechain record filter).
if not isinstance(payload, dict) or payload.get("agent_id"):
return 0
transcript = os.path.expanduser(payload.get("transcript_path") or "")
session = payload.get("session_id") or "unknown"
prompt_id = payload.get("prompt_id") or ""
if not transcript or not os.path.isfile(transcript):
return 0
state_path = os.path.join(STATE_DIR, f"{session}.json")
state = load_state(state_path)
size = os.path.getsize(transcript)
anchor = read_anchor(transcript)
offset = int(state.get("offset", -1))
carry = state.get("carry") or {}
already_flagged = state.get("flagged_mids") or []
# Undelivered ids awaiting the next post-arm fire. Legacy `pending`
# (the retired block-and-recap flow's stash) folds in on read.
carryover = [c for c in (state.get("carryover") or [])
if isinstance(c, str)]
legacy = state.get("pending")
if isinstance(legacy, list):
carryover = list(dict.fromkeys(
carryover + [p for p in legacy if isinstance(p, str)]))
# Dedupe surfaced by id: parallel tool calls fire parallel post arms
# whose read-modify-write can double-surface one mid; a duplicate here
# would inflate len(current) past any honest footer count and force a
# false footer-failed block.
surfaced, _seen = [], set()
for e in (state.get("surfaced") or []):
if isinstance(e, dict) and e.get("id") and e["id"] not in _seen:
_seen.add(e["id"])
surfaced.append(e)
final_turn_only = False
if (state.get("transcript") != transcript
or state.get("anchor") != anchor
or offset < 0 or offset > size):
# First fire, a different/rewritten file behind this session id, or
# an invalid offset: tail-scan, final turn only. `pending`,
# `surfaced`, and the flagged list deliberately survive re-baseline.
offset = max(0, size - TAIL_SCAN)
carry = {}
final_turn_only = True
hits, new_offset, carry = scan_window(
transcript, offset, carry, already_flagged, final_turn_only)
already_flagged = (already_flagged
+ [h.split("@", 1)[0] for h in hits])[-FLAGGED_CAP:]
base_state = {
"transcript": transcript,
"anchor": anchor,
"offset": new_offset,
"carry": carry,
"flagged_mids": already_flagged,
"fires": int(state.get("fires", 0)) + 1,
}
if post_mode:
# POST ARM: deliver new signatures plus any carryover from the stop
# arm's silent collection; the stop arm verifies the footer later.
# Never blocks anything (PostToolUse exit 2 only feeds stderr back),
# so no consec accounting here.
deliver = list(dict.fromkeys(carryover + hits))
have = {e["id"] for e in surfaced}
grown = surfaced + [{"id": h, "prompt": prompt_id}
for h in deliver if h not in have]
surfaced = grown[-PENDING_CAP:]
if len(grown) > PENDING_CAP:
log_event(session, "evicted",
ids=[e["id"] for e in grown[:-PENDING_CAP]])
save_state(state_path, {
**base_state,
"consec_blocks": int(state.get("consec_blocks", 0)),
"carryover": [],
"surfaced": surfaced,
})
if not deliver:
return 0
turn_total = (len([e for e in surfaced if e.get("prompt") == prompt_id])
if prompt_id else len(surfaced))
log_event(session, "surface", ids=deliver, carried=carryover,
prompt=prompt_id, offset=new_offset)
carried_note = ""
if carryover:
carried_note = (
f" {len(carryover)} of these surfaced at a turn boundary "
"and may belong to the previous turn; if any text is no "
"longer in your context, note that in one line -- never "
"reconstruct or invent."
)
# ASCII only by construction; stderr is backslashreplace regardless.
print(
f"[eaten-text] {len(deliver)} mid-turn text(s) not persisted: "
f"{short_ids(deliver)}. Weave their substance into your final "
"message where it naturally belongs, then end that message "
f"with the footer '[recovered: {turn_total}]' (inline is fine)."
f"{carried_note} Contract: {CONTRACT}.",
file=sys.stderr,
)
return 2
# STOP ARM.
# Partition surfaced ids: this turn's await footer verification; a
# previous turn's (its Stop never ran — interrupt/abort) join the
# carryover flow instead of being dropped.
current = [e for e in surfaced
if not prompt_id or e.get("prompt") == prompt_id]
stale = [e for e in surfaced if e not in current]
# Stragglers and stale ids collect SILENTLY; the next post-arm fire
# delivers them for weaving. No post-turn recovery block.
carry_all = list(dict.fromkeys(
carryover + [e["id"] for e in stale] + hits))
new_carry = carry_all[-PENDING_CAP:]
if len(carry_all) > PENDING_CAP:
# Eviction must leave a trace: the oldest (longest-waiting) ids
# fall off first, and a silent trim would read as full coverage.
log_event(session, "evicted", ids=carry_all[:-PENDING_CAP])
if hits or stale:
log_event(session, "carryover", ids=new_carry, offset=new_offset)
footer_failed = False
surfaced_after: list = []
if current:
last = payload.get("last_assistant_message")
text = last if isinstance(last, str) else (
json.dumps(last) if last else "")
if not text:
# Cannot verify this fire: keep the ids (never drop on a missing
# field), fail open with telemetry, never into a block loop. If
# the field stays absent, the entries go stale at the next
# prompt and ride the recovery demand instead.
log_event(session, "marker-unverifiable",
ids=[e["id"] for e in current])
surfaced_after = current
elif marker_count(text) >= len(current):
log_event(session, "marker-ok", ids=[e["id"] for e in current],
prompt=prompt_id)
else:
footer_failed = True
surfaced_after = current # keep for re-check on the next fire
# Counter resets ONLY on a fire with no new detections and no failed
# marker; a capped fire keeps it at the cap, so sustained detections
# cannot sawtooth the block rate or repeatedly reset the harness's own
# 8-block cap.
consec = (0 if not footer_failed
else int(state.get("consec_blocks", 0)))
will_block = footer_failed and consec < MAX_CONSEC
# Persist BEFORE any block decision: the same eaten records must never
# be flagged twice, even if the continuation turn misbehaves.
save_state(state_path, {
**base_state,
"consec_blocks": (consec + 1) if will_block else consec,
"carryover": new_carry,
"surfaced": surfaced_after,
})
if not footer_failed:
return 0
log_event(session, "footer-block",
ids=[e["id"] for e in current], capped=not will_block)
if not will_block:
return 0 # capped; unverified ids go stale -> carryover delivery
print(
f"[eaten-text guard] {len(current)} text(s) were surfaced to you "
f"mid-turn ({short_ids([e['id'] for e in current])}) but the final "
"message lacks a covering '[recovered: N]' footer. Send a message "
"confirming their substance is woven (weaving it now if not), "
f"ending with '[recovered: {len(current)}]' (inline is fine). "
f"Contract: {CONTRACT}. No tool calls.",
file=sys.stderr,
)
return 2
def main() -> int:
try:
return run("--post" in sys.argv[1:])
except Exception as exc: # fail-open: a guard must never wedge a session
print(f"guard_eaten_text: fail-open ({exc})", file=sys.stderr)
return 0
if __name__ == "__main__":
raise SystemExit(main())</details>