[FEATURE] Native support for System One (typed-decision) models like TypeSafe Jev

Status Open
Maintainer reply None cached
Activity 0 comments · opened Oct 7, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

Many decisions in a Claude Code workflow are classification, not generation: permission approval, prompt routing, subagent selection, file relevance, test-failure triage.

Today these run either as plain code (rigid) or as LLM calls. LLM calls are slow and costly for this kind of work, and their confidence is a prompted estimate rather than a calibrated probability, so it can't reliably drive automated decisions.

Proposed Solution

First-class support for System One models (e.g. TypeSafe's Jev) as a decision backend:

  1. A native decision hook type that sends state and typed questions to a configured System One endpoint and gates on the returned probability / confidence.
  2. Config to route built-in decision points (model and subagent routing) through that backend, and optionally add it as an extra check on permissions.
  3. Configurable confidence thresholds, with a fallback (ask the user, or escalate to the LLM) when confidence is low.

Expected outcomes (measured on a working hook/MCP prototype)

Over 2 days of real Claude Code use, a plugin routing decisions to Jev logged 6,404 calls (12,664 questions):

  • Speed: median latency was 238 ms, p95 416 ms. Most of these are per-tool-call gates (injection screening, destructive-action checks) that run without an LLM round-trip.
  • Cost: $0.32 total for all 6,404 calls, with 21 calls of unknown cost counted separately rather than booked as $0.
  • Context and tokens: about 949k tokens of file, log and search content were judged without ever entering the session context. This is a proxy (content judged and never read), not an A/B counterfactual. One example: 59 search hits were triaged in 14 calls, and only 2 were opened.
  • Reliability and hallucination surface:
  • Answers are restricted to caller-defined options, so the model can't invent an output.
  • Calibrated confidence lets low-confidence cases fall through to the LLM or the user instead of being silently approved.
  • A Stop gate blocks turns that claim a verification which never ran.
  • No rate reduction has been measured yet; that needs a controlled comparison.

Gaps that make native support necessary (found empirically, not from docs)

  1. updatedToolOutput has no effect on built-in Read, so hooks can only add context beside untrusted content, not wrap it.
  2. PostToolUse receives tool_response, not the documented tool_output, so a hook following the docs silently does nothing.
  3. SubagentStop cannot suppress a report, only annotate it.

Alternative Solutions

Command hooks and MCP servers can call Jev today, but every user has to build and maintain their own adapter. Jev's API isn't Messages-shaped, so it can't be swapped in by model name. Prompt-based hooks spend a full LLM call on what is really a yes/no judgment.

Priority

Medium - Would be very helpful

Feature Category

API and model interactions

Use Case Example

  1. User submits a prompt.
  2. A decision hook asks the System One model: "Is this task trivial, moderate, or complex?" and "Does it touch production config?"
  3. High-confidence answers route the task to the right model/subagent and permission mode in ~100 ms, without an LLM call.
  4. Low-confidence answers fall back to asking the user or to the main model.

Additional Context

  • TypeSafe reports 70-500 ms latency and 40-400x lower cost than frontier LLMs on its own workflows (vendor benchmarks, built by their own team).
  • An independent evaluation (arXiv 2609.37647) found Jev's choice probabilities well calibrated and suitable for selective prediction.
  • Related but distinct: #43326 (auto-select model by task complexity), #92162 (calibrated self-assessment signal).
  • Independent prior art: jev-router (write-up) puts a local gateway in front of Claude Code and has Jev pick the model for each new user message, a second separate workaround for the same missing native hook point.

View original on GitHub ↗