Workflow StructuredOutput: placeholder 'test' payload accepted as final result after repeated schema-validation failures (built-in deep-research synthesize)

Open 💬 1 comment Opened Jul 4, 2026 by ivanbechara11-spain

Environment

  • Claude Code 2.1.161 (macOS, darwin 25.5.0)
  • Built-in deep-research workflow (Workflow tool), workflow default model claude-opus-4-8

Summary

A built-in deep-research workflow run completed with status: completed, but the final research report was a literal placeholder the synthesize subagent emitted while "debugging" schema validation:

{"summary": "Test summary to isolate schema validation.",
 "findings": [{"claim": "test claim", "confidence": "high", "sources": ["https://example.com"], "evidence": "test evidence"}],
 "caveats": "Test caveats."}

All earlier phases worked (6 angles, 30 sources fetched, 141 claims extracted, 25 verified, 19 confirmed). The ~20-minute, 113-agent run produced this stub as its only output, with nothing in the result marking it as a failure.

What the persisted subagent transcript shows

The synthesize agent (StructuredOutput with a {summary, findings[], caveats} schema) made 4 attempts:

| # | Payload | Validator response |
|---|---------|--------------------|
| 1 | 1.7KB — {summary} only; string contains mangled <findings> / <invoke> XML fragments | rejected: must have required property 'findings', ... 'caveats' (correct) |
| 2 | 17.3KB — keys [summary, caveats, openQuestions]; the entire findings array is serialized as escaped text inside the summary string (the string's tail is literally ..."vote": "2-1, 2-1, 3-0"\n }\n]) | rejected: must have required property 'findings' (correct) |
| 3 | 15.1KB — same shape, same embedded-array failure | rejected: same (correct) |
| 4 | 211B — the placeholder above, submitted with the stated intent "Let me try passing a minimal ..." to isolate schema validation | accepted → became the phase result and the workflow's final report |

So the schema validator behaved correctly every time. The failure chain is: (a) the model could not serialize a large nested tool input correctly — it repeatedly stringified the findings array into the summary field — and (b) after repeated rejections it submitted schema-valid dummy data, which the harness accepted as a successful structured output, ending the agent.

An aggravating factor: the run was later resumed via resumeFromRunId, and the resume replayed the cached placeholder byte-identically — making the bug look reproducible/deterministic when it's actually cache replay of one bad output.

Why this is bad

  • Silent fabrication: schema-valid placeholder data masquerades as the completed output of an expensive multi-agent run. Neither the workflow script, the run record, nor the caller can distinguish it from a real synthesis.
  • The built-in deep-research success path does not include the confirmed-claims list in its return value, so the user cannot even salvage the verified claims from the result object (only by digging into the run journal / subagent transcripts).

Suggested mitigations

  1. Fail closed after repeated validation failures: after N rejected StructuredOutput attempts, fail the agent call (return null to the workflow, which already has salvage paths) instead of accepting whatever eventually passes.
  2. Targeted retry hint: when a required property is missing but its name appears stringified inside another string field, tell the model specifically — e.g. "findings must be a native array field, not text inside summary". Attempts 2/3 were one hint away from succeeding; the real synthesis was fully present.
  3. Placeholder detection: if output accepted after prior failures matches obvious sentinels (test claim, example.com, "isolate schema validation"), reject and retry or fail.
  4. In built-in deep-research specifically: include confirmed claims in the success return, and route degenerate reports (e.g. findings.length <= 1 when confirmed.length >= 5) into the existing !report salvage path.

Evidence

Run wf_40095a24-b1f, 2026-07-04. Persisted script, journal, run record, and the synthesize subagent transcript (with all 4 attempts and validator errors) are available and can be shared on request. The resolved workflow script itself is clean — the placeholder text appears nowhere in it.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗