[FEATURE] Plugin API: replay the main thread's last request as sent (cache pre-warm) for keepalive mods
Preflight Checklist
- [x] I have searched existing requests and this feature hasn't been requested yet
- [x] This is a single feature request (not multiple features)
Problem Statement
I maintain a cache-keepalive mod: when the session sits idle (waiting on the user, a long tool call, or a background agent), it sends a heartbeat just before the 5-minute prompt cache lapses, so the next real request reads the prefix instead of re-writing it.
The only primitive that can reproduce the main thread's prefix is $.model.fork({ prompt }), but it forks the post-sampling snapshot: the last request's messages plus that request's response (with tool_use blocks stripped), and with skipCacheWrite the message marker lands on that response. So every heartbeat:
- Writes the last response into the cache. The response is output, not input; ideally it gets cached on the next real request together with the user's input / tool results, not by a keepalive that may never be followed by another request.
- In the tool-running case, writes a tail that can never be reused. The fork strips
tool_usefrom the trailing assistant message, so the cached "response" differs from what the next real request will send (which includes thetool_useblock). That write is pure waste.
What a keepalive actually needs is to touch exactly what the last request cached, i.e. the request as sent, with its marker where the main thread put it: a pure cache read, zero cache write.
Nothing else in the mod API can do this:
$.model.forktakes onlyprompt; there is no way to choose the snapshot or the marker position.- Forks raise no
turn.step, so a hook cannot adjust them. $.model.completecarries no tools, history or session system prompt, so it can never hit the main thread's prefix.- Inside a
turn.stephook, a secondnext(e)does resend the exact pinned messages, but only while that step is running, not minutes later when the heartbeat is due. - No event exposes the rendered request body, so a mod cannot record and replay it either. The only workaround today is an external HTTP proxy in front of the API, which is a lot of machinery (and credential handling) for a mod.
Proposed Solution
A mod primitive that re-sends the main thread's last request as sent, for cache warming only. For example:
// Re-sends the main thread's last API request unchanged (model, system, tools,
// messages, cache_control markers and TTL, thinking/effort, betas) as a pre-warm:
// max_tokens: 0, no transcript append, no tools run, no hooks.
const r = await $.model.prewarm() // or: $.model.fork({ prompt, from: 'request' })
// r: { usage: ModelUsage } | { reason: 'nothing-to-warm' | 'api-error' | 'aborted', ... }
Desired properties:
- The snapshot is the request before its response, captured when the request is sent (not the post-sampling snapshot).
- Byte-identical cache-relevant content, so the result is
cache_read ≈ prefix, cache_creation = 0. - Uses the official
max_tokens: 0pre-warm, so no output tokens and no model reply to steer. - Optionally an
agentIdto warm a subagent's last request, since subagents keep their own prefix. usagein the result, so the mod can log what the heartbeat read.
Scheduling, caps and policy (idle vs. busy, how many beats, on/off) stay in the mod; the engine only needs to expose the replay.
Alternative Solutions
$.model.forkas it is today: it works, but it writes the response tail on every first heartbeat (wasted if the session is abandoned), and in the tool-running case it writes a tail that is never reused.- Exposing the rendered request on
turn.step(read-only) plus a raw "send this body" call: more general, but it leaks credentials and wire details into mods. A dedicated replay call is safer. - A local HTTP proxy via
ANTHROPIC_BASE_URLthat records and replays requests: exact, but heavy, outside the mod system, and it has to hold auth headers. - #95728 asks for a harness-owned keepalive. This request is complementary: it is the primitive that would let mods implement that policy (and variants such as capped idle beats, per-agent warming, logging) without the harness owning the policy.
Priority
Medium - Would be very helpful
Feature Category
Developer tools/SDK
Use Case Example
- A turn ends with a final answer, or starts a 20-minute test run in the foreground.
- At 270 s idle, the mod calls
$.model.prewarm(). - The API reads the full prefix the last request cached and writes nothing. The response and the next user input / tool result are cached once, by the next real request.
- The mod repeats up to its cap and logs
read 120k, wrote 0per beat.
Additional Context
Claude Code 2.1.296, Windows. Observed with a mod on the plugin API ($.model.fork, turn.step, $.clock.after).
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗