`/ultrareview` results become permanently unrecoverable once the task ID expires

Status Open
Maintainer reply None cached
Activity 0 comments · opened Aug 21, 2026

/ultrareview results become permanently unrecoverable once the task ID expires

Summary

/ultrareview is a paid/quota-limited feature (e.g. "Free ultrareview 2 of 3") that runs an expensive cloud review. Once it completes, its findings are delivered as a <task-notification> into the assistant's model context only — the user never sees this raw payload directly; the user's only view of the results is the assistant's own prose reproduction of them in the chat. There is no user-facing, durable way to retrieve the raw output at all: the tracking URL shows nothing, and TaskOutput stops resolving the task ID well within the same session (so even the assistant can't re-fetch it). The only way to get a review's data again is to re-run /ultrareview, which consumes another one of the limited runs and produces a fresh (not identical) analysis.

Because this feature specifically costs real quota per invocation, having no user-facing access to its raw output — only a model-mediated retelling — is a bigger problem than it would be for a free/cheap background task. You can't just "run it again" for free to recover the data, and you can't get the actual results without trusting the assistant's regeneration of them.

Repro

  1. Ran /ultrareview on a branch (billed against a limited quota — "Free ultrareview 2 of 3"). It launched a cloud review and returned a task ID plus a tracking URL (https://claude.ai/code/session_...).
  2. The review completed; a <task-notification> delivered the full JSON findings payload into the assistant's context — not anywhere the user could see it directly. The user's only visibility into the results was the assistant retyping/summarizing them in chat.
  3. User visited the tracking URL afterward — no output was shown there, so there was no user-facing way to see the raw results either.
  4. Later in the same session, the assistant called TaskOutput with the same task ID to try to re-fetch the raw payload on the user's behalf:

``
TaskOutput({ task_id: "rccpjvyqu", block: false })
→ Error: No task found with ID: rccpjvyqu
``

Expected

Given /ultrareview charges against a limited quota per run, the user should have some direct, durable, user-facing way to see the actual results — not only the assistant's paraphrase of them. Either:

  • the tracking URL should actually show the completed review's raw output to the user, or
  • the raw findings should be surfaced to the user directly in the conversation (e.g. as a file, artifact, or rendered block) at delivery time, not only handed to the assistant's context, or
  • TaskOutput (or an equivalent) should still retrieve the completed review's result for a reasonable retention window so it can be recovered later, or
  • the task notification should include a stable output file/link that persists independently of the ephemeral task ID.

Actual

  • The user never had direct access to the raw findings at all — only the assistant's own chat reproduction of them.
  • The tracking URL showed nothing.
  • The task ID was already unresolvable via TaskOutput well within the same conversation/session, so not even the assistant could recover the original data afterward.

Impact

For a feature that costs real, limited quota per run, users can't get a guaranteed-exact copy of the findings at all — they only ever see the assistant's regenerated retelling of them, with no fallback to the actual source once the task ID expires. Re-running the review to "get the data" burns another one of a limited number of paid/quota-gated runs.

This also locks in whichever model happened to be driving the session at delivery time. In this case the session was running Sonnet, so the retelling of an expensive, quota-gated review is permanently capped at Sonnet-quality synthesis — there's no way to later have a higher-tier model (Opus, Fable) re-read the same raw findings and produce a better analysis, because the raw data no longer exists anywhere retrievable. The model tier active when the task notification lands becomes a permanent, unchangeable property of the result.

Related

  • #87285 — a different failure mode on the same theme: there, the task-notification never arrives at all and the browser session 403s. Here, the notification does arrive (into the assistant's context) but still can't be recovered afterward by anyone.
  • #63331 — docs describe a /tasks command and claude ultrareview --json subcommand for reading findings after the fact; neither exists in the shipped CLI.

View original on GitHub ↗