PR Review: Structured output validation failure discards all AI findings

Bug Description

When Claude's structured output JSON schema validation fails after maximum retries, all PR review findings are discarded and the system incorrectly returns "Ready to Merge" even when the AI identified critical issues.

Impact

Severity: HIGH - This causes false negatives in PR reviews. Real security and quality issues found by the AI are silently lost, leading to incorrect "approve" verdicts.

Evidence from Production Failure

What the AI Actually Found

From logs_65.json (lines 1481-1498), the AI correctly identified:

## Final PR Review Verdict: **NEEDS_REVISION**

### Critical Issues (Must Fix Before Merge)

**1. CRITICAL - Code Duplication (85-90%)**
- Files: `lib/cache/hexagon-cache.ts`, `spatial-query-cache.ts`, `tile-cache.ts`
- ~1,000 of 1,172 lines duplicated across three cache modules
- Creates 3x maintenance burden

**2. HIGH - Race Condition in Hexagon Analysis**

What Was Actually Saved

From review_65.json:

{
  "verdict": "ready_to_merge",
  "verdict_reasoning": "No blocking issues found",
  "findings": [],
  "overall_status": "approve"
}

The Failure Chain in Logs

| Log Line | Content |
|----------|---------|
| 1451-1457 | Agent found MEDIUM severity admin verification issue |
| 1474-1477 | AI synthesized findings: "NEEDS_REVISION...1 CRITICAL..." |
| 1481-1484 | AI explicitly says "structured output is having issues" |
| 1495-1498 | AI lists CRITICAL code duplication and HIGH race condition |
| 1507 | ResultMessage: subtype=error_max_structured_output_retries |
| 1509 | WARNING: Structured output validation failed after retries |
| 1537 | Multi-pass review complete: 0 findings |
| 1544 | Verdict: ready_to_merge |

Root Cause Analysis

1. Schema Validation Failure

The ParallelOrchestratorResponse Pydantic schema has strict requirements:

class ParallelOrchestratorResponse(BaseModel):
    analysis_summary: str  # REQUIRED
    agents_invoked: list[str]
    findings: list[ParallelOrchestratorFinding]
    agent_agreement: AgentAgreement
    verdict: Literal["APPROVE", "COMMENT", "NEEDS_REVISION", "BLOCKED"]  # STRICT ENUM
    verdict_reasoning: str  # REQUIRED

And ParallelOrchestratorFinding requires exact enum values:

category: Literal[
    "security", "quality", "logic", "codebase_fit", "test", 
    "docs", "redundancy", "pattern", "performance"
]  # Must match EXACTLY

Likely validation failures:

• Category mismatch: AI used "duplication" but schema requires "redundancy"
• Verdict format: AI used "NEEDS REVISION" but schema requires "NEEDS_REVISION"
• Missing required fields after synthesizing 252 messages from 4+ agents

2. Text Fallback Only Parses JSON

When structured output fails, the code falls back to text parsing. But _parse_text_output only looks for JSON, not markdown:

File: parallel_orchestrator_reviewer.py:1096-1117

def _parse_text_output(self, output: str) -> list[PRReviewFinding]:
    """Parse findings from text output (fallback)."""
    findings = []
    try:
        data = self._extract_json_from_text(output)  # ← Only looks for JSON!
        if not data:
            return findings  # ← Returns empty list when no JSON found

3. Empty Findings → Wrong Verdict

With 0 findings, _generate_verdict returns READY_TO_MERGE:

File: parallel_orchestrator_reviewer.py:1370-1371

else:
    verdict = MergeVerdict.READY_TO_MERGE
    reasoning = "No blocking issues found"

Complete Failure Flow

AI analyzes PR (252 messages, 20+ minutes)
    ↓
AI finds CRITICAL code duplication + HIGH race condition
    ↓
AI tries to output ParallelOrchestratorResponse JSON
    ↓
Schema validation fails (category/verdict enum mismatch?)
    ↓
SDK retries multiple times, all fail
    ↓
SDK emits: error_max_structured_output_retries
    ↓
AI gives up, outputs findings in MARKDOWN format
    ↓
_extract_structured_output sees structured_output=None
    ↓
Calls _parse_text_output(result_text)
    ↓
_parse_text_output only looks for JSON, finds none
    ↓
Returns empty list []
    ↓
_generate_verdict sees 0 findings
    ↓
Returns READY_TO_MERGE ❌

Suggested Fixes

Option 1: Parse Markdown Fallback (Recommended)

Add markdown parsing to _parse_text_output to extract findings from patterns like:

• **CRITICAL - Code Duplication**
• ### Critical Issues
• Numbered lists with severity indicators

Option 2: Relax Schema Validation

• Make more fields optional with sensible defaults
• Use union types for categories (allow synonyms)
• Add case-insensitive enum matching

Option 3: Better Error Recovery

When error_max_structured_output_retries occurs:

1. Log the AI's text response for manual review
2. Re-prompt with a simpler schema
3. Extract verdict at minimum (even if findings parsing fails)

Option 4: Preserve Raw Findings

At minimum, save the AI's raw text response to a separate field so findings aren't completely lost:

if stream_result.get("error") == "structured_output_validation_failed":
    result.raw_ai_response = result_text  # Preserve for manual review
    logger.warning(f"Structured output failed, raw response saved")

Files Involved

• apps/backend/runners/github/services/parallel_orchestrator_reviewer.py - Main reviewer logic
• apps/backend/runners/github/services/sdk_utils.py - SDK stream processing
• apps/backend/runners/github/services/pydantic_models.py - Schema definitions

Reproduction

1. Run PR review on a large PR (40+ files) with complex caching/performance code
2. Wait for 252+ SDK messages (long review session)
3. Observe error_max_structured_output_retries in logs
4. Check that findings are 0 despite AI text showing issues found

Environment

• Platform: Windows
• Review session: 252 messages over ~20 minutes
• PR: 42 files, +12,836/-64 lines (performance optimization with caching)