Workflow tool: deep-research synthesis step accepts schema-valid-but-empty (placeholder) agent output as final result

Open 💬 0 comments Opened Jul 14, 2026 by myersdb

What happened

A Workflow({name: "deep-research", args: ...}) run completed "successfully" (109 agents, ~2M output tokens, 26 sources, 25/25 claims verified) but the final synthesized report was a placeholder stub:

{ "summary": "test", "findings": [{ "claim": "a", "confidence": "high", "sources": ["x"], "evidence": "y" }], "caveats": "none" }

All the real work (fetch-phase claim extractions, verification votes) was intact — only the last Synthesize sub-agent returned degenerate content.

Root cause

The final step is a single agent(..., {schema: REPORT_SCHEMA}) call. Schema validation guarantees shape, not substance, so a shape-valid placeholder passes. The workflow's only guard against a bad synthesis is a null check:

if (!report) { /* salvage confirmed claims raw */ }

A non-null, schema-valid stub bypasses this and is emitted as the run's result with afterSynthesis: 1. The run reports success while the deliverable is empty — a silent, expensive quality failure.

Impact

  • Long/costly runs can silently produce a worthless final artifact.
  • Recovery requires manually reading journal.jsonl (the per-agent result log) and re-synthesizing by hand. The tool's own diagnostics hint at this ("If the result above is empty or unexpected, Read this file BEFORE diagnosing"), but it's manual.

Suggested fix

In the bundled deep-research workflow, extend the post-synthesis guard to detect degenerate output — e.g. findings.length === 0, findings.length far below the confirmed-claim count, or placeholder tokens in summary/claim — then retry once and, failing that, fall back to the raw-claims salvage that already exists for the null case. More generally: a schema-valid result is not necessarily a substantive one; terminal synthesis nodes may warrant a lightweight sanity gate.

Notes

  • Non-deterministic (a "lazy completion" on one sub-agent); the other 108 agents returned real content.
  • The named workflow appears bundled with the skill (no user-editable definition file found), so a fresh invocation regenerates the same vulnerable script — editing the persisted per-run copy doesn't persist the fix.

Environment

  • Claude Code 2.1.207 on Windows 11
  • Model: Claude Opus 4.8

View original on GitHub ↗