Monitor tool: a curl+parse polling loop can report a false terminal "DONE" state on the first iteration while the real status is still pending

Status Open
Maintainer reply None cached
Activity 0 comments · opened Sep 8, 2026

Summary

A Monitor-run bash loop that polls an external HTTP API in a for/while loop and echoes a status line per iteration reported a terminal "DONE" state on the very first polling iteration (~30s after the loop started) — while the real, ground-truth state (confirmed immediately after via a separate, correct API call) was still in-progress/pending. This happened twice, across two separately-launched Monitor invocations with slightly different scripts, in the same session.

Repro pattern

A Monitor-launched background bash script that:

  1. Loops over N items to track (in this case, PR head SHAs).
  2. Per item, per iteration, curls a JSON status endpoint and parses a state-like field out of the response (in this case via an inline python3 -c one-liner).
  3. Echoes a line like <item> -> DONE state=<state> when state is one of a small set of terminal values.

Expected

The loop should only emit a DONE-style line once the polled endpoint's real response actually contains a terminal state value.

Actual

On the very first iteration (~30s in), the loop emitted DONE state=success for every tracked item simultaneously — a suspiciously synchronized "all done at once" result rather than the staggered completion times you'd expect from genuinely different backend jobs.

Immediately afterward, calling the equivalent request through a separate, canonical tool (in this case a first-party MCP tool that hits the same underlying API) for the exact same items/identifiers showed the real state was still pending, with several jobs explicitly still "Waiting to run" / "In progress".

Ruled out as the cause:

  • Not an auth/env issue: a follow-up diagnostic Monitor run confirmed the expected auth token was genuinely present and correct-looking in the Monitor's shell environment.
  • Not a network/credentials issue: a raw curl + cat of the identical URL, from the identical environment, returned the real "state":"pending" JSON with HTTP 200.
  • Not the underlying bash/curl/python logic: switching to the exact same curl+python one-liner run via repeated plain foreground Bash calls (not Monitor), paced by a separate Bash run_in_background sleep loop, correctly tracked the real state through to genuine success over the following ~15 minutes, with no further false positives.

So the false-positive behavior appears specific to Monitor's execution/notification path for this loop-and-echo pattern — not the bash/curl/python logic itself, which behaved correctly when run outside Monitor.

Hypothesis (unconfirmed)

Possibly Monitor's output-line batching/notification logic is buffering or parsing early/partial stdout from the loop's first iteration in a way that causes it to treat an early, incomplete echo (or an artifact of how multiple near-simultaneous curl calls interleave their output) as if it were the loop's real terminal output, rather than waiting for the actual per-iteration curl responses to land and be echoed correctly.

Impact

This is a silent, plausible-looking failure mode: the reported event has a real HTTP-200-shaped structure and correctly-formatted "DONE state=success" text, so nothing about it looks obviously wrong at the point it's reported. Anything that trusts a Monitor-reported terminal state without an independent re-verification (e.g. via a direct API/tool call) before acting on it could act on false information — in the case this was found in, that would have meant merging code on stale/fake "green" CI status. It was only caught because of an unrelated discipline of always re-verifying via a canonical tool immediately before taking an irreversible action.

If this is a general Monitor limitation (not specific to the particular API being polled), it likely affects any multi-item polling-loop pattern, not just the CI-status-check case it was found in.

Suggested repro for maintainers

A minimal Monitor script with a for loop doing N curl calls to any slow/async HTTP endpoint + a conditional echo on a terminal-looking field, watched closely (e.g. with intermediate debug echoes) for the exact point where a premature/malformed "done" event gets synthesized, would likely reproduce and help pin down whether this is a stdout-line-batching race or something else in Monitor's parsing of loop output.

View original on GitHub ↗