Workflow StructuredOutput: placeholder 'test' payload accepted as final result after repeated schema-validation failures (built-in deep-research synthesize)
Environment
- Claude Code 2.1.161 (macOS, darwin 25.5.0)
- Built-in
deep-researchworkflow (Workflow tool), workflow default modelclaude-opus-4-8
Summary
A built-in deep-research workflow run completed with status: completed, but the final research report was a literal placeholder the synthesize subagent emitted while "debugging" schema validation:
{"summary": "Test summary to isolate schema validation.",
"findings": [{"claim": "test claim", "confidence": "high", "sources": ["https://example.com"], "evidence": "test evidence"}],
"caveats": "Test caveats."}
All earlier phases worked (6 angles, 30 sources fetched, 141 claims extracted, 25 verified, 19 confirmed). The ~20-minute, 113-agent run produced this stub as its only output, with nothing in the result marking it as a failure.
What the persisted subagent transcript shows
The synthesize agent (StructuredOutput with a {summary, findings[], caveats} schema) made 4 attempts:
| # | Payload | Validator response |
|---|---------|--------------------|
| 1 | 1.7KB — {summary} only; string contains mangled <findings> / <invoke> XML fragments | rejected: must have required property 'findings', ... 'caveats' (correct) |
| 2 | 17.3KB — keys [summary, caveats, openQuestions]; the entire findings array is serialized as escaped text inside the summary string (the string's tail is literally ..."vote": "2-1, 2-1, 3-0"\n }\n]) | rejected: must have required property 'findings' (correct) |
| 3 | 15.1KB — same shape, same embedded-array failure | rejected: same (correct) |
| 4 | 211B — the placeholder above, submitted with the stated intent "Let me try passing a minimal ..." to isolate schema validation | accepted → became the phase result and the workflow's final report |
So the schema validator behaved correctly every time. The failure chain is: (a) the model could not serialize a large nested tool input correctly — it repeatedly stringified the findings array into the summary field — and (b) after repeated rejections it submitted schema-valid dummy data, which the harness accepted as a successful structured output, ending the agent.
An aggravating factor: the run was later resumed via resumeFromRunId, and the resume replayed the cached placeholder byte-identically — making the bug look reproducible/deterministic when it's actually cache replay of one bad output.
Why this is bad
- Silent fabrication: schema-valid placeholder data masquerades as the completed output of an expensive multi-agent run. Neither the workflow script, the run record, nor the caller can distinguish it from a real synthesis.
- The built-in
deep-researchsuccess path does not include the confirmed-claims list in its return value, so the user cannot even salvage the verified claims from the result object (only by digging into the run journal / subagent transcripts).
Suggested mitigations
- Fail closed after repeated validation failures: after N rejected StructuredOutput attempts, fail the agent call (return
nullto the workflow, which already has salvage paths) instead of accepting whatever eventually passes. - Targeted retry hint: when a required property is missing but its name appears stringified inside another string field, tell the model specifically — e.g. "
findingsmust be a native array field, not text insidesummary". Attempts 2/3 were one hint away from succeeding; the real synthesis was fully present. - Placeholder detection: if output accepted after prior failures matches obvious sentinels (
test claim,example.com, "isolate schema validation"), reject and retry or fail. - In built-in
deep-researchspecifically: includeconfirmedclaims in the success return, and route degenerate reports (e.g.findings.length <= 1whenconfirmed.length >= 5) into the existing!reportsalvage path.
Evidence
Run wf_40095a24-b1f, 2026-07-04. Persisted script, journal, run record, and the synthesize subagent transcript (with all 4 attempts and validator errors) are available and can be shared on request. The resolved workflow script itself is clean — the placeholder text appears nowhere in it.
This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗