[FEATURE] Auto mode classifier dry-run / preview mode

Resolved 💬 2 comments Opened Mar 26, 2026 by msd-olo Closed Apr 24, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

Users considering switching from acceptEdits to auto mode have no way to evaluate what the auto-mode classifier would decide for their typical workflows before committing to the switch. The classifier has a documented 17% false-negative rate, and users working with sensitive infrastructure (Kubernetes clusters, cloud resources, production databases) need to understand what the classifier would silently approve before trusting it.

Related: #38500 requests a pre-classifier hook, which would allow overriding classifier decisions. This request is complementary — it focuses on observability of classifier decisions while remaining in the current permission mode.

Proposed Solution

Add a dry-run / preview mode for the auto-mode classifier that:

  1. Runs the classifier on every tool call that would reach tier 3 (not already resolved by built-in allowlist or in-project file edits)
  2. Logs the classifier's decision (allow/block) and its reasoning (stage 1 result, stage 2 chain-of-thought if triggered)
  3. Does not act on the decision — the current permission mode (e.g., acceptEdits) remains in control
  4. Makes the log accessible via a file, a /classifier-log command, or the verbose transcript (Ctrl+O)

This would let users audit classifier behavior against their own judgment over days or weeks before switching to auto mode.

Alternative Solutions

  • Just switch to auto mode and see what happens — risky for infrastructure-heavy workflows where a single bad auto-approve could have real consequences
  • PreToolUse hook with custom classification — can't replicate the actual Sonnet 4.6 classifier behavior; the hook's pattern matching is a poor approximation of the LLM-based classifier
  • Pre-classifier hook (#38500) — solves the override problem but not the observability problem; users still wouldn't know what the classifier decided or why

Priority

Medium - Would be very helpful

Feature Category

CLI commands and flags

Use Case Example

A platform engineer uses Claude Code daily with acceptEdits mode to manage Kubernetes clusters and Terraform infrastructure. They want to switch to auto mode to reduce prompt fatigue but need confidence that the classifier won't silently approve kubectl apply, helm upgrade, or other mutating infrastructure commands without prompting. They enable classifier dry-run mode and review a week of decisions to identify any gaps before switching.

Additional Context

The auto mode blog post acknowledges the tension: "If you are manually approving every action carefully, it's arguably a regression — you're trading your own judgment for a classifier that will sometimes make a mistake." A dry-run mode would bridge this gap by letting users build trust in the classifier incrementally.

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗