[FEATURE] Plugin API: replay the main thread's last request as sent (cache pre-warm) for keepalive mods

Status Open
Reported on v2.1.296
Maintainer reply None cached
Activity 1 comment · opened Oct 13, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

I maintain a cache-keepalive mod: when the session sits idle (waiting on the user, a long tool call, or a background agent), it sends a heartbeat just before the 5-minute prompt cache lapses, so the next real request reads the prefix instead of re-writing it.

The only primitive that can reproduce the main thread's prefix is $.model.fork({ prompt }), but it forks the post-sampling snapshot: the last request's messages plus that request's response (with tool_use blocks stripped), and with skipCacheWrite the message marker lands on that response. So every heartbeat:

  1. Writes the last response into the cache. The response is output, not input; ideally it gets cached on the next real request together with the user's input / tool results, not by a keepalive that may never be followed by another request.
  2. In the tool-running case, writes a tail that can never be reused. The fork strips tool_use from the trailing assistant message, so the cached "response" differs from what the next real request will send (which includes the tool_use block). That write is pure waste.

What a keepalive actually needs is to touch exactly what the last request cached, i.e. the request as sent, with its marker where the main thread put it: a pure cache read, zero cache write.

Nothing else in the mod API can do this:

  • $.model.fork takes only prompt; there is no way to choose the snapshot or the marker position.
  • Forks raise no turn.step, so a hook cannot adjust them.
  • $.model.complete carries no tools, history or session system prompt, so it can never hit the main thread's prefix.
  • Inside a turn.step hook, a second next(e) does resend the exact pinned messages, but only while that step is running, not minutes later when the heartbeat is due.
  • No event exposes the rendered request body, so a mod cannot record and replay it either. The only workaround today is an external HTTP proxy in front of the API, which is a lot of machinery (and credential handling) for a mod.

Proposed Solution

A mod primitive that re-sends the main thread's last request as sent, for cache warming only. For example:

// Re-sends the main thread's last API request unchanged (model, system, tools,
// messages, cache_control markers and TTL, thinking/effort, betas) as a pre-warm:
// max_tokens: 0, no transcript append, no tools run, no hooks.
const r = await $.model.prewarm()          // or: $.model.fork({ prompt, from: 'request' })
// r: { usage: ModelUsage } | { reason: 'nothing-to-warm' | 'api-error' | 'aborted', ... }

Desired properties:

  • The snapshot is the request before its response, captured when the request is sent (not the post-sampling snapshot).
  • Byte-identical cache-relevant content, so the result is cache_read ≈ prefix, cache_creation = 0.
  • Uses the official max_tokens: 0 pre-warm, so no output tokens and no model reply to steer.
  • Optionally an agentId to warm a subagent's last request, since subagents keep their own prefix.
  • usage in the result, so the mod can log what the heartbeat read.

Scheduling, caps and policy (idle vs. busy, how many beats, on/off) stay in the mod; the engine only needs to expose the replay.

Alternative Solutions

  • $.model.fork as it is today: it works, but it writes the response tail on every first heartbeat (wasted if the session is abandoned), and in the tool-running case it writes a tail that is never reused.
  • Exposing the rendered request on turn.step (read-only) plus a raw "send this body" call: more general, but it leaks credentials and wire details into mods. A dedicated replay call is safer.
  • A local HTTP proxy via ANTHROPIC_BASE_URL that records and replays requests: exact, but heavy, outside the mod system, and it has to hold auth headers.
  • #95728 asks for a harness-owned keepalive. This request is complementary: it is the primitive that would let mods implement that policy (and variants such as capped idle beats, per-agent warming, logging) without the harness owning the policy.

Priority

Medium - Would be very helpful

Feature Category

Developer tools/SDK

Use Case Example

  1. A turn ends with a final answer, or starts a 20-minute test run in the foreground.
  2. At 270 s idle, the mod calls $.model.prewarm().
  3. The API reads the full prefix the last request cached and writes nothing. The response and the next user input / tool result are cached once, by the next real request.
  4. The mod repeats up to its cap and logs read 120k, wrote 0 per beat.

Additional Context

Claude Code 2.1.296, Windows. Observed with a mod on the plugin API ($.model.fork, turn.step, $.clock.after).

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗