[FEATURE] Verification summary generated by the harness (not the model) at the end of each turn

Status Open
Maintainer reply None cached
Activity 2 comments · opened Oct 11, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

At the end of a task the model often says "all tests pass" or "this should work now". In practice:

  • Sometimes the full suite ran.
  • Sometimes only one test file ran.
  • Sometimes tests ran, and then Claude made three more edits.
  • Sometimes nothing ran at all.

To know which, I have to scroll back through the transcript. The harness already knows the truth: which commands ran, their exit codes, and when they ran relative to each edit. That information just isn't shown to me.

Proposed Solution

After any turn that changed files, show a short summary built from the tool log, not from the model's text:

Verification
  ✅ npm test        exit 0   ran after the last edit
  ⚠️ npm run lint    exit 0   stale: 3 edits came after it
  ❌ typecheck       not run
  📄 2 edited files were never read back after editing
  • Declare the expected checks: a project setting such as "verify": ["npm test", "npm run lint", "npx tsc --noEmit"] lets the summary show what is missing, not only what ran.
  • Expose it to hooks: pass the same structured data to Stop hooks, so a team can block "done" when required checks are missing or stale.
  • Keep it short: one line per check, with a single collapsed line when everything passed.

Alternative Solutions

  • Stop hooks that parse the transcript: this works, but every team has to rebuild it, and it breaks when the transcript format changes.
  • Asking the model to report what it verified: that is the problem itself. Self-reports are sometimes wrong, and a summary from the harness can't overstate.

Priority

High - Significant impact on productivity

Feature Category

Interactive mode (TUI)

Use Case Example

  1. I ask Claude to fix a bug. It edits 3 files, runs the one related test, and then edits a 4th file.
  2. Claude ends with "Fixed, tests pass."
  3. The summary shows the test run is stale (one edit came after it) and that lint and typecheck never ran.
  4. I reply "run the checks" instead of finding out in CI 20 minutes later.

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗