Sonnet 4.5 Code Quality Regression: Overconfidence + Lack of Verification
Problem Description
When using Claude Sonnet 4.5 to write production code recently, I've noticed a worsening pattern: code appears complete and professional, but is actually full of unverified assumptions and edge-case bugs.
Specific Example
A commit labeled "comprehensive worker robustness (P0+P1+P2)" with a 300+ word commit message listing complete feature points and test scenarios. After review, we found:
| Feature | Sonnet Implementation | Actual Problem |
|---------|----------------------|----------------|
| Swap detection for OOM prevention | Exit when sys.used_swap() > 50MB | Used global swap, not process-level; Windows exceeds 50MB at boot, worker immediately dies |
| Stuck detection | Hardcoded 5min/10min timeout | Single generation can take 10+ minutes, normal work gets killed |
| Checkpoint resume | "Checkpoint + Resume" | Only saves generation counter, population state is lost, actually restarts from scratch |
| Graceful shutdown | shutdown_rx channel | Only checked in evolution loop, Ctrl+C has no effect during connection phase |
| Memory budget | Check at startup | Fixed assumption of 8 islands, but actually dynamically assigned by Master |
Core Issues
- No API semantic verification: Directly calls
sys.used_swap()without checking docs to confirm if it's global or process-level - No edge case consideration: Hardcoded thresholds without considering actual runtime scenarios
- Form over substance: Checkpoint "appears" to implement save/restore, but only stores metadata not state
- Overconfident presentation: Commit message reads as production-ready, signed as Co-Author
Comparison
Same codebase, when Opus did the review:
- Accurately identified all issues
- Understood actual production environment constraints
- Provided correct fix directions
Expectations
- Sonnet should be more cautious when dealing with system calls, concurrency, and state management
- Uncertain API semantics should be marked with
// TODO: verifyrather than used directly - Reduce the tendency to "look complete", increase verification of edge cases
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗