Headless output: name the model that wrote the final answer, and document the fallback record

Status Open
Maintainer reply None cached
Activity 0 comments · opened Oct 14, 2026

Preflight Checklist

  • [x] I have searched existing requests and this feature hasn't been requested yet
  • [x] This is a single feature request (not multiple features)

Problem Statement

We run Claude Code unattended (-p --output-format stream-json) as an evaluation judge, with a pinned --model. In a pilot, a few judge calls were answered by a fallback model after automatic model fallback. That behaviour is documented, and this request isn't about the fallback itself. It's about making the answering model easy to read from headless output.

In those runs:

  • the system/init record named the requested model, which is right at session start;
  • a system record with subtype model_refusal_fallback carried original_model and fallback_model;
  • the final result record's modelUsage listed both models;
  • each assistant message's model field showed which model wrote it.

Our harness took the model from init, so it reported the requested model. That was our mistake, and the records to catch it were all there. We've written up what we changed: https://mainthread.ai/notes/which-model-answered/.

Proposed Solution

Three requests:

  1. Add answer-authorship metadata to result, for example answer_models: the model IDs whose text makes up the final answer, with several IDs if needed, an empty list when there's no answer text, and null when attribution isn't available. In our fallback runs, modelUsage listed both models, including the refused one, without saying which wrote the returned text.
  2. Document the model_refusal_fallback system record and its fields in the headless / stream-json reference, so harness authors know to look for it.
  3. Clarify what init.model means: it records the model at session start. A later fallback can change the session's active model without updating that record.

Alternative Solutions

_No response_

Priority

Medium - Would be very helpful

Feature Category

API and model interactions

Use Case Example

_No response_

Additional Context

Related: #94652 (fallback notices and billing in interactive sessions) and #98387 (subagents and switchModelsOnFlag).

Thanks for documenting the fallback and switchModelsOnFlag; it made the cause quick to find.

View original on GitHub ↗