[ENHANCEMENT] Inter-agent `SendMessage`: add transport-level send metadata (timestamp + sequence + delivery ack) to detect stale / out-of-order / lost messages

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

In multi-agent sessions, SendMessage delivery can fail, be delayed, or be reordered under transient network degradation. The sender's call succeeds, but the receiver never gets the message (or gets it late / out of order). Because messages carry no transport-level send metadata (timestamp, sequence, id), a receiver cannot distinguish a current message from a stale / delayed / duplicate one, and a sender cannot tell that delivery failed. This produces silent correctness gaps, wasted recovery loops, and a class of "stale control message acts on the wrong target" bugs. Agents also forget to respond through SendMessage quite often to the asking agent. They end up in an idle-lock. Giving SendMessage a parameter, that tells the receiver, that caller is expecting a response back.

Background / where it bites

Observed during a long (~10h wall-clock) multi-agent design-grooming run (Windows, in-process backend, 1 lead + orchestrator + up to 7 specialists). Delivery degraded transiently and bidirectionally (lead↔orchestrator and specialist↔orchestrator). Concrete symptoms:

1. Lost findings. Specialist→orchestrator findings were silently absent from the orchestrator's state across multiple discussion rounds (R1/R2/R3). The orchestrator's round-log showed e.g. 6/7 completions while all 7 had actually sent. Recovery required re-sending every finding via a file channel.
2. Crossed / stale control messages. The orchestrator re-sent a HARD-GATE approval request after the lead had already sent the approval. Neither side could tell the re-send was stale — there was no timestamp/sequence to compare against.
3. Stale shutdown-approval killed a fresh, same-named agent. A shutdown_request sent to an old orchestrator instance was approved minutes later; the harness matched termination by name and terminated a newly spawned agent that had reused the name (orchestrator). Work was only saved by ground-truth disk verification + re-spawn.
4. Idle-loops. An agent waiting on a relay that never arrived spun in tight idle notifications (observed ~8 idle pings in 15s), unable to distinguish "still waiting" from "already answered, my reply was lost."

Impact

• Correctness risk: a lost finding/decision that is not noticed and recovered.
• Cost/time: dozens of manual re-send + ground-truth-verification round-trips; large token/time overhead.
• Operational fragility: unattended long runs can deadlock (mutual idle) when a relay is lost.

Why a prompt-level rule is insufficient

A natural first instinct is "tell agents to include a timestamp in every message." This is unreliable for the same reason other soft conventions are: agents follow complexly-conditioned instructions inconsistently, and compliance fails exactly on the critical control messages (shutdown / approval / completion). The guarantee must come from the transport (SendMessage), not from sender discipline.

(Empirically, the only timestamp we could trust during the run was file mtime from the file-fallback channel — an enforced, sender-independent timestamp the OS attaches. That is itself the argument for putting equivalent metadata in the transport.)

Proposed Solution

1. Auto-attach send metadata to every SendMessage (transport-level, not author-supplied)

• sent_at — monotonic timestamp set by the runtime at send time.
• seq — per-sender monotonic sequence number.
• message_id — unique id.

These are stamped by the harness, so an agent cannot forget or mis-format them.

2. Expose the metadata to the receiver + apply staleness/ordering policy

• Receiver (and harness) can drop or flag a control message older than its last-known state for that conversation.
• supersedes / last-write-wins semantics for status/approval/completion messages, keyed by (sender, topic) and ordered by seq/sent_at.

3. Strict requestId→instance matching for lifecycle messages

• shutdown_response (and similar) must terminate the specific agent instance that owns the outstanding requestId, never "whatever agent currently holds that name." This fixes the stale-shutdown-kills-fresh-agent class even without (1)/(2).

4. Delivery acknowledgement + at-least-once with dedup

• The sender receives a delivery receipt (delivered / not-yet) instead of only "queued."
• Redelivery reuses the same message_id so the receiver can dedup.
• This lets senders detect non-delivery and retry deterministically instead of guessing from silence.

Nice-to-have

• A lightweight, harness-maintained per-team message log so recovery can reconcile against ground-truth automatically, instead of every skill hand-rolling round-logs + bespoke file-fallback.

Alternative Solutions

Current workaround (for reference)

File-fallback: agents write critical findings/decisions to disk (artifacts/*.md, decision files); receivers read ground-truth from files rather than trusting message delivery. File mtime acts as the enforced, sender-independent timestamp for staleness/ordering. It works but is bespoke per skill, costs an extra write per message, and doesn't help control-plane messages (shutdown/approval) without additional plumbing.

Priority

Medium - Would be very helpful

Feature Category

API and model interactions

Use Case Example

_No response_

Additional Context

_No response_