[FEATURE] A "fact-check gate" at response-commit time: verify a verification action ran before the model asserts facts

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

LLM output is, structurally, a stringification of sampled tokens. There is no built-in guarantee that an assertion in the assistant's text (e.g. "X is not installed", "the test passed", "it works") is backed by an actual verification action (a tool call such as a shell command, a file Read, or an existence check). In practice the model frequently asserts facts from memory/recollection, and the user only discovers the mistake by challenging it afterward.

For users who build deterministic, auditable AI operations, this "assert from memory" failure mode is the single biggest reliability risk. The model cannot reliably observe its own lapses, so self-checking does not solve it — a machine-enforced check is needed.

Proposed Solution

Add a "fact-check gate" at the response-commit moment.

Key structural findings from real operation:

1. The assistant's utterance is a stringification of variables (tokens); its factual backing is not structurally guaranteed. A guard must therefore check, not the content phrasing, but whether a verification action actually happened.
2. Generation and display are separate (effectively asynchronous) stages. Hook output injected at display time (e.g. via MessageDisplay displayContent) is shown on screen but does NOT appear in the transcript. This is observable: a footer injected by a MessageDisplay hook shows on screen every turn yet leaves zero trace in the transcript's assistant content.
3. The moment a response is "decided" is well-defined: when the response is assigned to the display variable (e.g. the delta field in a MessageDisplay hook). That is the natural, deterministic point at which a hook can inspect "did a verification action occur just before this?" and act.
4. We confirmed in real operation that a MessageDisplay hook can prepend a note to the assistant's response at that commit moment (verified on screen). So injection/overwrite at response-commit time already works.

Request: at the response-commit moment, allow a hook to (a) read whether a verification tool was used in the immediately preceding steps (transcript lookup), and (b) gate the response on it — not merely annotate ("please verify"), but block / send back when no verification action preceded a factual assertion, forcing the model to verify and re-report.

Conceptually a checkSafetyBehavior(input)-style gate: judge purely on data-structure facts (a flag set when a verification tool ran; or a transcript scan for preceding tool_use), with no reliance on the model's self-report.

Today the pieces are almost there: PreToolUse can block tool calls before execution, Stop can send back with decision: block after the turn, and MessageDisplay can change the displayed text (display-only, not recorded). What's missing is a clean, recorded, response-commit-time gate that (a) is observable in the transcript/audit log and (b) can hard-gate (not just annotate) based on whether a verification action preceded an assertion.

Alternative Solutions

We built a workaround using hooks: a MessageDisplay hook that scans the response for assertive phrasing and prepends a "did you actually verify this?" note at display time. This works (the note appears on screen), but it only annotates — it cannot force the model to actually verify, and the injected note is not recorded in the transcript, so there is no audit trace of whether the check fired.

Other partial workarounds: PreToolUse blocking (only covers tool calls, not text assertions), and Stop with decision: block (fires after the turn, and the model still has to re-derive what to verify). None of these provide a clean, recorded, response-commit-time gate keyed on "was a verification action actually performed before this assertion".

Priority

High - Significant impact on productivity

Feature Category

Developer tools/SDK

Use Case Example

Concrete scenario (real operation):

1. The user asks: "is Python available in this environment?"
2. The model answers from memory ("no, it's a Microsoft Store stub") without actually running python --version in this session.
3. The user challenges it; the model finally runs the command and finds Python does work — the earlier answer was a memory-based fabrication.

With a fact-check gate: at the moment the model commits the assertion "Python is not available", the hook sees that no verification tool (Bash/Read/existence-check) ran in the immediately preceding steps, and sends the response back, forcing the model to actually run the check before reporting. The user never receives the unverified claim in the first place.

This single failure mode — asserting environment/state facts from memory instead of checking — was the most frequent and most damaging reliability issue across a long real session.

Additional Context

Observed behavior that makes this design feasible:

• A MessageDisplay hook already injects a footer (timestamp + context %) into every response at display time. This footer is visible on screen but leaves zero trace in the transcript — confirming that generation and display are separate stages, and that display-time injection works but is not recorded.
• We verified on screen that a MessageDisplay hook can prepend a note to the response at the commit moment.

What is missing is the ability to (a) gate (block/send-back), not just annotate, and (b) have that gate decision recorded in the audit log. The check should be purely data-structure based (was a verification tool used before the assertion?), independent of the model's self-report, since the model cannot reliably observe its own lapses.