Headless output: name the model that wrote the final answer, and document the fallback record
Preflight Checklist
- [x] I have searched existing requests and this feature hasn't been requested yet
- [x] This is a single feature request (not multiple features)
Problem Statement
We run Claude Code unattended (-p --output-format stream-json) as an evaluation judge, with a pinned --model. In a pilot, a few judge calls were answered by a fallback model after automatic model fallback. That behaviour is documented, and this request isn't about the fallback itself. It's about making the answering model easy to read from headless output.
In those runs:
- the
system/initrecord named the requested model, which is right at session start; - a
systemrecord with subtypemodel_refusal_fallbackcarriedoriginal_modelandfallback_model; - the final
resultrecord'smodelUsagelisted both models; - each assistant message's
modelfield showed which model wrote it.
Our harness took the model from init, so it reported the requested model. That was our mistake, and the records to catch it were all there. We've written up what we changed: https://mainthread.ai/notes/which-model-answered/.
Proposed Solution
Three requests:
- Add answer-authorship metadata to
result, for exampleanswer_models: the model IDs whose text makes up the final answer, with several IDs if needed, an empty list when there's no answer text, andnullwhen attribution isn't available. In our fallback runs,modelUsagelisted both models, including the refused one, without saying which wrote the returned text. - Document the
model_refusal_fallbacksystem record and its fields in the headless / stream-json reference, so harness authors know to look for it. - Clarify what
init.modelmeans: it records the model at session start. A later fallback can change the session's active model without updating that record.
Alternative Solutions
_No response_
Priority
Medium - Would be very helpful
Feature Category
API and model interactions
Use Case Example
_No response_
Additional Context
Related: #94652 (fallback notices and billing in interactive sessions) and #98387 (subagents and switchModelsOnFlag).
Thanks for documenting the fallback and switchModelsOnFlag; it made the cause quick to find.