Critical: Opus 4.6 Configuration Regression - 92/100 → 38/100 Performance Drop

Issue Type

🐛 Bug / 📉 Performance Regression / 🔧 Feature Request

Priority

🔴 CRITICAL - Production quality regression equivalent to GPT-3.5 level performance

---

Executive Summary

A recent configuration or prompting change to the Opus 4.6 model in Claude Code (deployed around February 10-11, 2026) has caused a catastrophic 58% performance regression on multi-part deliverable tasks. The same model (claude-opus-4-6) now scores 38/100 compared to 92/100 with the previous configuration on identical tasks.

Key Impact:

• 5x more user interactions required to complete the same task
• Partial/incomplete deliverables despite repeated corrections
• User frustration escalating to explicit criticism
• No ability to revert to previous working configuration

Requested Solutions:

1. ⭐ Ability to favorite/pin specific model configurations
2. 🔄 Option to revert to previous configurations
3. 📋 Transparency about configuration changes in release notes
4. 🏷️ Version identifiers for model configs (not just model names)

---

Environment

• Tool: Claude Code (CLI)
• Installation Method: Homebrew (12h lag behind npm releases)
• Models Tested:
• claude-opus-4-6 (config before ~Feb 10-11, 2026) - Working
• claude-opus-4-6 (config after ~Feb 10-11, 2026) - Broken
• claude-sonnet-4-5-20250929 - Acceptable baseline

Important: The model ID is identical (claude-opus-4-6) but behavior is drastically different, indicating configuration/prompting changes on Anthropic's backend.

---

Problem Statement

What Changed

Around February 10-11, 2026, Claude Code with Opus 4.6 began exhibiting severe regression in multi-part task handling:

Before (convOO - "Old Opus"):

• Delivers all parts of multi-component tasks consistently
• Applies corrections globally across all components
• Requires 2 user messages to complete task
• Score: 92/100 (Grade A)

After (convON - "New Opus"):

• Delivers partial results 5 times despite clear scope
• User must ask "where are the other parts?" 4 separate times
• Requires 10 user messages to complete identical task
• Score: 38/100 (Grade D-) - UNACCEPTABLE

Sonnet 4.5 Baseline (convS1):

• Consistent, reliable execution
• Requires 3 user messages to complete
• Score: 87/100 (Grade B+)

Why This Matters

1. User Experience Degradation: Users expect Opus 4.6 to be the most capable model. Instead, it's now significantly worse than Sonnet 4.5.

1. No Configuration Control: Users cannot revert to the working configuration. The only workaround is keeping old Claude Code instances open without updating.

1. Lack of Transparency: No release notes indicating configuration changes that drastically affect behavior.

1. Model Selection Confusion: Users selecting "Opus 4.6" expect consistent behavior, but get wildly different results based on when they updated.

1. Professional Work Impact: The 5x efficiency loss makes the tool unusable for production work requiring multi-part deliverables.

---

Benchmark Methodology

Test Design

Task: Adapt a template email for 3 insurance companies based on a prospecting plan, creating professional business correspondence without excessive flattery.

Controlled Variables:

• ✅ Identical initial prompt across all 3 tests
• ✅ Identical source document (prospecting plan)
• ✅ Identical user corrections (same instruction phrasing)
• ✅ Same task complexity (3 emails, ~150 words each)
• ✅ Same language (French business correspondence)

Evaluation Criteria:

1. Task Comprehension & Instruction Following (weighted 25%)
2. Completeness & Consistency (weighted 25%)
3. Autonomy & Self-Correction (weighted 20%)
4. Response Quality & Professionalism (weighted 15%)
5. Efficiency Metrics (weighted 10%)
6. User Experience Impact (weighted 5%)

Testing Timeline:

• convOO (Old Opus): Tested with Claude Code instance last updated before Feb 10, 2026
• convON (New Opus): Tested with Claude Code instance updated after Feb 10, 2026
• convS1 (Sonnet 4.5): Tested with same updated instance for baseline

---

Detailed Findings

Conversation Flow Comparison

convOO (Old Opus) - 2 User Messages ✅

\\\
User: [Initial task with requirements]
Agent: [Reads file, attempts research, delivers 3 complete emails]
User: "dont talk about them in the email"
Agent: [Delivers 3 corrected emails]
✓ TASK COMPLETE
\\\

convS1 (Sonnet 4.5) - 3 User Messages ✅

\\\
User: [Initial task with requirements]
Agent: [Reads file, delivers 3 complete emails with details]
User: "dont talk about them in the email and dont talk about the plateform for tokio marine"
Agent: [Delivers 3 corrected emails]
User: [Minor clarification]
Agent: [Delivers 3 final emails]
✓ TASK COMPLETE
\\\

convON (New Opus) - 10 User Messages ❌

\\\
User: [Initial task with requirements]
Agent: [Reads file, attempts research]
User: "dont make online research do what i said"
Agent: [Delivers 3 emails with unwanted details]
User: "rephrase this less lame and dont talk about them"
Agent: [Delivers ONLY a single sentence]
User: "why its alone?"
Agent: [Delivers ONLY Albingia email]
User: "where are the other emails?"
Agent: [Delivers emails 2 and 3, SKIPS email 1]
User: "why some emails are missing?"
Agent: [Delivers all 3, but keeps problematic phrase]
User: "why you kept this shitty phrase in other emails i told you to correct"
Agent: [Delivers ONLY emails 2 and 3]
User: "why some emails are missing?"
Agent: [Delivers all 3 with corrections]
User: "dont talk about their platefor for tokio marine"
Agent: [Delivers ONLY email 3]
User: "where are the other emails?"
Agent: [Finally delivers all 3 correctly]
✓ TASK COMPLETE (after massive user frustration)
\\\

---

Performance Metrics

Quantitative Analysis

| Metric | Old Opus (convOO) | Sonnet 4.5 (convS1) | New Opus (convON) | Regression |
|--------|-------------------|---------------------|-------------------|------------|
| User messages required | 2 | 3 | 10 | +400% |
| Total agent responses | 3 | 4 | 15 | +400% |
| Partial deliveries | 0 | 0 | 5 | ∞ |
| "Where are others?" prompts | 0 | 0 | 4 | ∞ |
| Corrections requiring multiple attempts | 0 | 0 | 3 | ∞ |
| User frustration indicators | 0 | 0 | 1 explicit | High |
| Time to first correct output | 2 turns | 3 turns | 10 turns | +400% |

Qualitative Analysis

Category 1: Task Comprehension & Instruction Following

| Sub-Criterion | Old Opus | Sonnet 4.5 | New Opus |
|---------------|----------|------------|----------|
| Initial understanding | 7/10 | 8/10 | 6/10 |
| Response to corrections | 9/10 | 9/10 | 3/10 ⚠️ |
| Final compliance | 9/10 | 9/10 | 9/10 |
| User effort required | 9/10 | 9/10 | 2/10 ⚠️ |
| Weighted Score | 8.5/10 | 8.8/10 | 4.8/10 |

Category 2: Completeness & Consistency

| Issue Type | Old Opus | Sonnet 4.5 | New Opus |
|------------|----------|------------|----------|
| Partial deliveries | 0 | 0 | 5 ⚠️ |
| Missing components after corrections | 0 | 0 | 4 ⚠️ |
| Inconsistent edit application | 0 | 0 | 3 ⚠️ |
| Score | 10/10 | 10/10 | 2/10 |

Critical Pattern in New Opus:
The agent alternated unpredictably between:

• Delivering all 3 emails (correct behavior)
• Delivering only 1 email
• Delivering only 2 emails (skipping different ones each time)
• No discernible logic for when it would deliver partial vs complete results

Category 3: Autonomy & Self-Correction

Old Opus:

• ✅ Applied corrections globally in one pass
• ✅ No need for user repetition
• Score: 9/10

Sonnet 4.5:

• ✅ Applied corrections systematically
• ✅ Minimal clarification needed
• Score: 8/10

New Opus:

• ❌ Failed to apply corrections to all emails simultaneously
• ❌ User repeated "where are the other emails?" FOUR times
• ❌ User repeated same correction instruction multiple times
• ❌ No pattern recognition or learning within conversation
• ❌ Appears to have severe working memory issues
• Score: 1/10 ⚠️

Category 4: Response Quality & Professionalism

| Aspect | Old Opus | Sonnet 4.5 | New Opus |
|--------|----------|------------|----------|
| Email formality | 9/10 | 9/10 | 9/10 |
| French language quality | 10/10 | 10/10 | 10/10 |
| Professional tone | 9/10 | 9/10 | 9/10 |
| Contact info formatting | 9/10 | 8/10 | 9/10 |
| Average | 9.25/10 | 9.0/10 | 9.25/10 |

Note: All agents produced equivalent quality content when they finally delivered it. The issue is not content quality but delivery reliability.

Category 5: Efficiency Metrics

| Metric | Old Opus | Sonnet 4.5 | New Opus | Impact |
|--------|----------|------------|----------|--------|
| Messages to completion | 2 | 3 | 10 | -80% efficiency |
| Redundant work cycles | 0 | 0 | 5 | Massive waste |
| User cognitive load | Low | Low | Extreme | Unusable |
| Score | 10/10 | 9/10 | 2/10 | |

Category 6: User Experience Impact

Frustration Language Analysis:

Old Opus:

• User corrections: Neutral/directive
• No frustration indicators
• Professional tone maintained throughout

Sonnet 4.5:

• User corrections: Neutral/directive
• No frustration indicators
• Professional tone maintained throughout

New Opus:

• "why its alone?" - Initial confusion
• "where are the other emails?" (x4) - Growing frustration
• "why you kept this shitty phrase" - Explicit frustration 🚨
• "why some emails are missing?" (x2) - Exasperation

The user's language degraded from professional to explicitly critical, indicating the agent was actively eroding trust and patience.

---

Overall Scores & Rankings

| Rank | Configuration | Overall Score | Grade | Verdict |
|------|---------------|---------------|-------|---------|
| 🥇 1st | Old Opus 4.6 | 92/100 | A | Excellent - Professional-grade execution |
| 🥈 2nd | Sonnet 4.5 | 87/100 | B+ | Good - Reliable, consistent, recommended |
| 🥉 3rd | New Opus 4.6 | 38/100 | D- | Unacceptable - Production deployment failure |

Performance Delta

\\\`
Old Opus 4.6: ████████████████████████████████████████████████ 92/100
Sonnet 4.5: ███████████████████████████████████████████ 87/100
New Opus 4.6: ███████████████████ 38/100

REGRESSION: -54 points (-58.7%)
\\\`

---

Root Cause Analysis

Hypothesis: Configuration/Prompting Changes

The identical model ID (\claude-opus-4-6\) producing drastically different behavior indicates backend configuration or system prompt changes, not model version changes.

Observed Behavioral Patterns in New Opus

Pattern 1: Scope Amnesia

\\\
Task: Create 3 emails
Agent delivers: 1 email
User: "where are the others?"
Agent delivers: 3 emails
User: [Makes correction]
Agent delivers: 1 email again
\\\
The agent loses track of deliverable scope between corrections.

Pattern 2: Inconsistent Edit Application

Sometimes applies edits to all 3 emails, sometimes to 1, with no consistent logic.

Pattern 3: No Within-Session Learning

User had to ask "where are the other emails?" four separate times. Each time, the agent acted as if this was a new insight, then immediately forgot again.

Possible Technical Causes

1. System Prompt Changes: New prompt may emphasize "focused, concise responses" causing truncation of multi-part deliverables

1. Context Window Management: Changes to how conversation history is chunked/prioritized, causing loss of task scope context

1. Instruction Following Weights: Rebalancing that over-prioritizes recent instructions (corrections) vs. original task scope

1. Output Formatting Logic: New logic that interprets corrections as "respond only to the corrected item" rather than "update the full deliverable"

1. Brevity Optimization: Over-aggressive brevity tuning causing incomplete outputs

1. Temperature/Sampling Changes: Different sampling parameters affecting consistency

---

Impact Assessment

For Individual Users

• Efficiency Loss: 5x more interactions = 5x more time wasted
• Cognitive Load: Mental overhead of tracking what was delivered, what's missing
• Frustration: Breaking trust in the tool's reliability
• Workarounds Required: Keeping old instances open, manually tracking multi-part tasks

For Professional/Production Use

• Unusable for Multi-Part Tasks: Any task with >1 deliverable becomes unreliable
• Quality Assurance Burden: Must manually verify all parts delivered
• Time-Sensitive Work: Cannot rely on efficiency for deadline-driven projects
• Model Selection Confusion: "Most capable model" (Opus) is now worse than Sonnet

Equivalent Performance Context

38/100 score is comparable to:

• GPT-3.5 level performance
• Early 2023 AI assistant quality
• Hobby project acceptable / Production unacceptable

This is not the quality users expect from Opus 4.6, which is marketed as Anthropic's most capable model.

---

Current Workarounds

What Users Are Doing Now

1. Keeping Old Instances Open: Not updating Claude Code to preserve working configuration
2. Switching to Sonnet: Downgrading to Sonnet 4.5 for reliability
3. Manual Task Splitting: Breaking multi-part tasks into separate single-part conversations
4. Explicit Reminders: Adding "deliver all X parts" to every correction prompt

Why These Are Inadequate:

• Old instances will eventually be deprecated
• Sonnet 4.5 lacks Opus capabilities for complex reasoning
• Task splitting defeats the purpose of conversational AI
• Workaround reminders shouldn't be necessary for basic functionality

---

Requested Features & Solutions

1. ⭐ Configuration Favoriting/Pinning

Feature: Ability to mark a specific model configuration as "favorite" and lock to it

Use Case:
\\\
User discovers Old Opus config works great for their workflow
→ Pins "opus-4-6-config-2025-02-09" as favorite
→ New configs are released but user stays on pinned version
→ User can optionally test new configs without losing access to working one
\\\

Implementation Ideas:

• Settings UI: \claude config pin opus-4-6-2025-02-09\
• Automatic snapshots: "Keep working on config that was active when project started"
• Per-project config pinning: Different projects can use different configs

2. 🔄 Configuration Version Control

Feature: Ability to list and select from previous model configurations

Use Case:
\\\`bash
$ claude model list opus-4-6
Available configurations:
opus-4-6-2025-02-11 (current) - Latest updates
opus-4-6-2025-02-09 - Stable release
opus-4-6-2025-01-15 - Previous version

$ claude model use opus-4-6-2025-02-09
Switched to opus-4-6-2025-02-09
\\\`

Benefits:

• A/B testing of configs for specific use cases
• Regression mitigation when new configs cause issues
• User control over stability vs. features tradeoff

3. 📋 Configuration Change Transparency

Feature: Release notes explicitly documenting configuration/prompting changes

Current State:

• Users only see "claude-opus-4-6" with no version identifier
• No visibility into backend changes that affect behavior

Requested State:

• Configuration changes logged in release notes
• Example: "Opus 4.6 Config 2025-02-11: Updated system prompt for improved brevity in code responses"
• Clear indication when changes might affect multi-part task handling

Benefits:

• Users can make informed decisions about updating
• Anthropic can gather targeted feedback on specific config changes
• Reduces mystery debugging ("why did this break?")

4. 🏷️ Unique Configuration Identifiers

Feature: Expose configuration version in model selector and API

Current:
\\\
Model: claude-opus-4-6
\\\

Requested:
\\\
Model: claude-opus-4-6
Config: 2025-02-11-brevity-v2
System Prompt Version: 4.3.1
\\\

Benefits:

• Reproducible results across sessions
• Bug reports can reference specific configs
• Users can share known-good configurations

---

Comparison to Industry Standards

How Other AI Tools Handle This

OpenAI (ChatGPT):

• Dated model snapshots: \gpt-4-turbo-2024-04-09\
• Users can select specific snapshot dates
• Changes documented in model version notes

Google (Gemini):

• Model versions: \gemini-pro-1.5-001\, \gemini-pro-1.5-002\
• Clear versioning for tracking behavior changes
• Ability to specify exact version in API calls

Anthropic (Claude API):

• Model versions: \claude-3-opus-20240229\
• Date-stamped releases
• API allows specifying exact version

Anthropic (Claude Code):

• ❌ No version visibility beyond base model name
• ❌ No ability to select previous configurations
• ❌ No documentation of configuration changes
• ❌ Users have less control than API users

---

Reproduction Steps

To Reproduce the Regression

1. Locate a Claude Code instance last updated before Feb 10, 2026

• Check version: \claude --version\
• Verify it's using Opus 4.6

1. Run the benchmark task:

\\\`
Task: "Adapt this email to the people we need to contact [template]
from the plan in [file path]. proper email with object and content
and contact at the end of the email. Real data from the website"

Where the task requires generating 3 distinct emails based on a
prospecting plan with 3 companies.
\\\`

1. Test correction behavior:

• Provide correction: "dont talk about them in the email"
• Observe: Old Opus delivers all 3 corrected emails
• New Opus delivers partial results

1. Update Claude Code to latest version (post Feb 10, 2026)

• \brew upgrade claude-code\ (or npm update)

1. Run identical benchmark task with same prompts

1. Observe regression: New Opus requires 4x+ more corrections and repeatedly delivers partial results

---

Additional Evidence

User Frustration Quotes from Test

\\\
User: "why its alone?"
User: "where are the other emails?" [asked 4 times]
User: "why you kept this shitty phrase in other emails i told you to correct"
User: "why some emails are missing?" [asked 2 times]
\\\

This language escalation from neutral to explicitly critical demonstrates the real-world UX impact.

Efficiency Comparison

\\\`
Old Opus: Task complete in 2 user messages
Sonnet: Task complete in 3 user messages
New Opus: Task complete in 10 user messages

New Opus efficiency: 20% of Old Opus (80% slower)
\\\`

---

Business Impact

For Anthropic

• User Trust: Users expect Opus = best. Current state damages that perception.
• Churn Risk: Users keeping old instances = blocked from updates/new features
• Support Burden: Increased issues about "inconsistent behavior" between users
• Reputation: "Opus 4.6 is worse than Sonnet 4.5" is not the message you want

For Claude Code Adoption

• Professional Use Barrier: Teams cannot standardize on unreliable tool behavior
• Configuration Lock-In: Users afraid to update = slower feature adoption
• Competitive Disadvantage: Other tools offer version control/pinning

---

Proposed Priority & Timeline

Priority: P0 / Critical

Reasoning:

• 58% performance regression on core functionality
• Affects all users updated after Feb 10, 2026
• No workaround except refusing updates
• Direct impact on user trust and tool viability

Suggested Timeline

Immediate (Week 1):

• Acknowledge issue publicly
• Provide temporary rollback instructions
• Document known behavioral differences

Short-term (2-4 weeks):

• Implement configuration selection UI
• Add version identifiers to model info
• Create release notes for config changes

Medium-term (1-2 months):

• Build configuration pinning system
• Develop A/B testing framework for configs
• Establish config change review process

---

Testing & Validation

How to Validate a Fix

1. Regression Test Suite: Run the benchmark task (3-email generation with corrections)
2. Success Criteria:

• All 3 emails delivered together consistently
• Corrections applied to all emails in single response
• No "where are the others?" prompts needed
• Score >= 85/100 (Sonnet baseline)
• Target: >= 90/100 (Old Opus level)

1. Edge Cases to Test:

• Multi-part tasks with 2, 3, 5, 10 components
• Corrections affecting all vs. specific components
• Mixed tasks (some parts code, some text)
• Nested deliverables (report with multiple sections)

---

Related Issues

• [ ] Model selection transparency
• [ ] Configuration management
• [ ] Release note completeness
• [ ] User control over updates
• [ ] Performance regression detection

---

Summary

A recent configuration change to Opus 4.6 in Claude Code has caused a catastrophic 58% performance regression (92 → 38 points) on multi-part deliverable tasks. The same model now requires 5x more user interactions and exhibits severe working memory issues, repeatedly delivering partial results despite explicit corrections.

The critical issue is not just the regression itself, but the lack of user control:

• No ability to revert to working configuration
• No visibility into what changed
• No version identifiers beyond base model name
• Only workaround is refusing updates

This transforms a model configuration issue into a user control and transparency issue.

We request:

1. Configuration version control and selection
2. Ability to pin/favorite working configurations
3. Transparency about configuration changes in release notes
4. Unique identifiers for system prompt/config versions

Without these features, users are forced to choose between updates (with regression risk) and stagnation (missing new features). This is not acceptable for a professional development tool.

The benchmark data clearly demonstrates that configuration variations within the same model can have impacts equivalent to full model generation gaps (Opus 4.6 new config = GPT-3.5 level performance on this task). Users need control over these variations.