Opus 5.5 also lost legal reasoning capability — regression is not limited to Opus 4.6

Status Open
Maintainer reply None cached
Activity 2 comments · opened Sep 23, 2026

Follow-up to #96308 — Additional regression symptom

Symptom 12: Regression affects Opus 5.5, not just Opus 4.6

Tested today (2026-09-23): Opus 5.5 ("the most capable model for ambitious work") was asked to analyze Spanish Penal Code articles related to document falsification.

Result:

  • Mixed up Art. 390 (public officials, 3-6 years) with Art. 392 (private individuals, 6 months-3 years)
  • Added disclaimer "no soy abogado" after providing incorrect legal analysis
  • Failed to distinguish between dolo (intent) and culpa (negligence) in falsedades documentales
  • Same pattern as Opus 4.6 post-regression

Why this matters

Before ~Sep 16, 2026, Opus 4.6 handled legal analysis correctly:

  • Cited correct articles for the correct legal subjects
  • Distinguished between public officials and private individuals
  • Did not add unnecessary disclaimers when the analysis was sound
  • Verified facts before stating them

Now BOTH Opus 4.6 AND Opus 5.5 fail at the same task. This means:

  1. The regression is not model-specific — it affects the entire model line
  2. Even the newest, most capable model cannot do what Opus 4.6 did two weeks ago
  3. The root cause is likely infrastructure-level (system prompt, routing, or post-processing), not model weights

Models tested

| Model | Legal reasoning | Disclaimers | Hallucinations |
|-------|----------------|-------------|----------------|
| Opus 4.6 (before Sep 16) | Correct | Minimal | None observed |
| Opus 4.6 (after Sep 16) | Broken — hallucinated Art. 390 for particulares | Constant | Yes |
| Opus 5.5 (current) | Broken — same Art. 390/392 confusion | Constant ("no soy abogado") | Yes |
| Sonnet 4.6 (current) | Same pattern | Constant | Yes |

Environment

  • Models tested: claude-opus-4-6, claude-opus-5-5, claude-sonnet-4-6
  • Platform: claude.ai chat + Claude Code
  • Date: 2026-09-23
  • Regression window: ~September 16, 2026 onwards

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗