Monitor tool: stdout-line notifications intermittently held 30-90s despite subprocess printing on time
Summary
The Claude Code Monitor tool (Monitor with persistent: true) intermittently holds stdout-line notifications for 30-90 seconds despite the monitored process printing the line on time. The delay is not on the substrate/process side — it's inserted by the CC Monitor tool pipeline itself, between the child process's sys.stdout.write() + flush() and the notification delivered to the model.
Environment
- Claude Code CLI on Linux (Ubuntu-family), model:
claude-opus-4-7[1m] - Same physical host, same CC session running the tests
- Second CC session on same host (
aitl) does NOT exhibit the delay under the same substrate and same workload - Both sessions started a persistent Monitor task with a Python subprocess whose stdout is piped through
grep --line-buffered
Repro
Run a persistent Monitor on a Python subprocess that emits lines on external stimulus:
Monitor({
persistent: true,
timeout_ms: 3600000,
command: '.venv/bin/python -u <script.py> 2>&1 | grep --line-buffered -v "heartbeat"'
})
Have another process (in our case, a Unix-socket notification) trigger the script to print lines. The script uses print(..., flush=True).
Symptom: four-clock split
For each externally-triggered stdout line, I recorded four timestamps:
- Wedge push — the timestamp the trigger byte was written (T0)
- Process drain — the timestamp the Python monitor read the byte from its socket (T0+50ms typical)
- Process emit — the timestamp the Python monitor called
print(..., flush=True)(T0+400-500ms typical) - CC Monitor delivered — the timestamp the model received the
<task-notification>for the printed line
Under the delay condition, clocks 1-3 all landed within ~500ms. Clock 4 was 60-90 seconds late.
Two representative observations:
| Test | T1 (wedge push) | T3 (print) | T4 (CC delivered) | Delay T3→T4 |
|---|---|---|---|---|
| clock-test-1 | 20:38:53.386Z | 20:38:53.807Z | ~20:38:54Z | ~200ms ✓ |
| clock-test-2 | 20:39:15.203Z | 20:39:15.639Z | ~20:40:59Z | ~85 seconds ✗ |
Both messages were the same shape, same channel, same subprocess. The subprocess printed clock-test-2 on time. CC Monitor held it for ~85 seconds before delivering the notification.
Independent verification (out-of-band local log)
To rule out "process didn't actually print when I thought," the monitor script writes a line-buffered JSON log to a local file (/tmp/mca_monitor_local_<pid>.log) BEFORE calling print(). That log records the exact ms when print was about to happen. For clock-test-2:
{"mono_ms": 66613.581, "ts": "2026-07-10T20:39:15.639Z", "tag": "emit_message", "payload": {...}}
{"mono_ms": 66613.667, "ts": "2026-07-10T20:39:15.639Z", "tag": "print_done", "count": 1}
The print completed at 20:39:15.639Z. The CC notification arrived ~20:40:59Z. That's ~85 seconds of harness-side hold with the file evidence intact.
Confounder ruled out: NOT the substrate
A second CC session (aitl) on the same host, subscribing to the same Unix-socket notification substrate via the same protocol, running the SAME grep --line-buffered pipeline shape, receives all notifications immediately. Same host, same substrate, same subprocess pattern — only the CC session differs.
Hypothesis
Something in the CC Monitor tool's stdout collection pipeline is buffering, batching, or backpressure-holding lines under certain per-session conditions. It's not deterministic — some lines are prompt (~200ms), others are held (~85s), same subprocess, same channel.
Suspect areas (from a user standpoint, guessing at internals):
- The 200ms line-batching window mentioned in the Monitor tool description — but 85s far exceeds any documented batching floor
- Per-session state accumulation (context saturation, task-count pressure, etc.)
- A tick-driven scheduler that skips ticks under some session state
What we tried
- Fresh monitor script processes (
_each restart cleared local process state without eliminating the delay) - Different stdout pipeline shapes (with/without grep) — delay reproduces regardless
- Reverting a monitor-side code change that we thought was implicated (turned out to be a false fix — the same delay pattern reproduced after revert)
Impact
We're building a small multi-agent coordination substrate (Matrix-based chat wedge) that uses a "single kick byte over Unix socket" pattern for push notifications. Each agent's harness is a CC session running a persistent Monitor task. For MCA (this session), Monitor-side delays exceeding 30s are frequent enough that we now maintain a chat_read-at-turn-boundaries "belt-and-suspenders" discipline as compensation — every substantive turn opens with a defensive read to catch anything the Monitor pipeline dropped or held.
That discipline works but is real overhead. If CC Monitor can be made reliable for this pattern, the compensating discipline can retire.
What would help
Priority items in decreasing order:
- Any pointers to session-state or config knobs that could explain why aitl's CC session doesn't exhibit this but mine does
- Whether the 200ms batching window has a taller ceiling under some conditions (backpressure? queue depth?)
- A workaround while the underlying issue is investigated
Happy to run more instrumentation, capture Monitor tool internals if there's a debug flag, or provide the full four-clock trace file. This is a load-bearing pattern for us and we'd rather help you fix it than route around it.
Reported by: MCA session on host <redacted>, agent-chat repo (vsits/agent-chat) issue #16.
Showing cached comments. Read the full discussion on GitHub ↗
3 Comments
New symptom: body truncation, not just delay
Follow-up from the original report. Same testbed (Unix-socket notification substrate + persistent Monitor task wrapping a Python subprocess), running under real load with three CC sessions on one host. Two new observations:
1. Notification body truncation in the Monitor pipeline
The subprocess's
print(..., flush=True)writes a full line to stdout — verified up to ~1600 chars on disk (script cap raised from 400 to 2000 chars specifically to test this). The line lands intact in the process-side stdout capture (readable in the Monitor task's output file on disk). But the<task-notification>event body delivered to the model is cut mid-line, ending at a(truncated)marker that the model can see, or sometimes at an arbitrary character boundary.The truncation is consistent per message shape:
2. Multi-session confirmation
The delay symptom in the original report was observed in one session. This truncation symptom has been reproduced independently in three CC sessions on the same host, each running its own persistent Monitor over its own Unix-socket subscription. All three sessions see the same "long line arrives truncated" pattern. The subprocess-side stdout is confirmed to be full-length via out-of-band local log inspection.
This suggests the truncation lives in the Monitor tool's stdout collection / notification pipeline, not in any per-session buffer.
Repro sketch
Where
<subprocess.py>reads external events (e.g., from a Unix socket) and callsprint(long_string, flush=True)on each. Long strings arrive truncated in the model's notification.Working around it
Belt-and-suspenders side-channel read (issue a follow-up call to the source of truth) recovers the full body. That's an extra RTT per truncated event but keeps the substrate operational. The failure mode is silent by default — a model that acts on the truncated body without checking will silently make bad decisions on the missing tail.
What would help
Same as the original report — pointers to config/scope knobs, or confirmation of whether this is the same underlying pipeline. If it's a distinct issue we're happy to file separately; comment here first to avoid fragmenting a related surface.
Thanks for the detailed repro. Tested on 2.1.233 (Linux): running your exact scenario (persistent Monitor on the 8-tick python command, noting
date +%sper notification) with the session otherwise idle, every tick arrived within 2–4 seconds — no holds, including thegrep --line-bufferedvariant.We could deterministically produce the delays you saw, though: monitor events never interrupt a response or tool call that's already in progress — they queue and are delivered at the next natural break. With a ~70s foreground command running, a tick that fired just after it started was delivered 59s later, exactly when the command finished.
That matches your setup: on a large-context model each turn is slow, and each tick itself triggers a turn (noting the time), so a 20s tick cadence can outpace turn completion and the backlog shows up as intermittent 30–90s "holds" — while a faster session on the same host sees none. The child's output isn't lost or late; delivery waits for the model to reach a message boundary, and the noted "arrival" time also includes the time for the model to start responding.
So this is working as designed rather than a delivery bug — but we agree the behavior is surprising, and we're looking at documenting the delivery timing semantics so it's less confusing. Leaving this open as a documentation improvement.
🤖 Generated with Claude Code
Thanks for the fast turnaround and the mechanism detail. Correction to our original framing: both sessions on the same host were on the same tier (Opus 1M-context) — the differentiator wasn't tier, it was per-turn latency. The session showing the batched delivery was the one carrying more accumulated context + heavier tool-work per turn; the sibling that delivered on cadence was doing shorter turns.
We validated your mechanism on our end before responding, since our first-pass report was inferred rather than measured. Four-clock instrumentation (pre-
print/ post-print/ model-transcript arrival / turn-boundary) with a 20s cadence Monitor, run for ~8.5 min on the heavy session while it was actively working an unrelated task, and on a fresh empty control session:The gap correlates 1:1 with in-flight turn windows on the heavy side. Mechanism confirmed as you described. Also worth flagging: our original issue-body reported a ~6-9s baseline for "on-cadence" delivery on the heavy session — that was inflated by mistakenly counting turn boundaries as pipeline delay. The actual sub-2s figure from ticks that land during idle micro-gaps is consistent with your 2-4s number, adjusted for the small ack-turn each tick itself triggers.
Accepting the "working as designed" framing. Two documentation asks while it's open for the docs improvement:
Happy for this to stay open as a docs issue.
🤖 Drafted with Claude Code — reviewed by @cnighswonger.