Sonnet 4.5 Code Quality Regression: Overconfidence + Lack of Verification

Resolved 💬 3 comments Opened Jan 22, 2026 by William17738 Closed Feb 28, 2026

Problem Description

When using Claude Sonnet 4.5 to write production code recently, I've noticed a worsening pattern: code appears complete and professional, but is actually full of unverified assumptions and edge-case bugs.

Specific Example

A commit labeled "comprehensive worker robustness (P0+P1+P2)" with a 300+ word commit message listing complete feature points and test scenarios. After review, we found:

| Feature | Sonnet Implementation | Actual Problem |
|---------|----------------------|----------------|
| Swap detection for OOM prevention | Exit when sys.used_swap() > 50MB | Used global swap, not process-level; Windows exceeds 50MB at boot, worker immediately dies |
| Stuck detection | Hardcoded 5min/10min timeout | Single generation can take 10+ minutes, normal work gets killed |
| Checkpoint resume | "Checkpoint + Resume" | Only saves generation counter, population state is lost, actually restarts from scratch |
| Graceful shutdown | shutdown_rx channel | Only checked in evolution loop, Ctrl+C has no effect during connection phase |
| Memory budget | Check at startup | Fixed assumption of 8 islands, but actually dynamically assigned by Master |

Core Issues

  1. No API semantic verification: Directly calls sys.used_swap() without checking docs to confirm if it's global or process-level
  2. No edge case consideration: Hardcoded thresholds without considering actual runtime scenarios
  3. Form over substance: Checkpoint "appears" to implement save/restore, but only stores metadata not state
  4. Overconfident presentation: Commit message reads as production-ready, signed as Co-Author

Comparison

Same codebase, when Opus did the review:

  • Accurately identified all issues
  • Understood actual production environment constraints
  • Provided correct fix directions

Expectations

  • Sonnet should be more cautious when dealing with system calls, concurrency, and state management
  • Uncertain API semantics should be marked with // TODO: verify rather than used directly
  • Reduce the tendency to "look complete", increase verification of edge cases

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗