[Bug] Opus 4.7 incorrect reasoning about URL normalization vs dereferencing

Status Closed — not planned
Reported on v2.1.119
Maintainer reply None cached
Activity 2 comments · opened Apr 27, 2026 · closed May 4, 2026

Bug Description
This is Opus 4.7 your flagship model, intended for use for developing Web APIs, Web apps, etc. And yet it cannot _reason_ correctly about the difference between URL normalization and URL deferencing. This will be a fun one to dig into if you have MechInterp tools.

This oversight has led to my current project being wrong, bloated, confusing, and frustrating. I can only imagine how many other people are finding Claude's reasoning to be off in areas where it arguably probably has a lot of training data? I don't know why this problem persists, but I did replicate it twice. So. Worth adding some evaluations for I think -- at the risk of overfitting and contaminating ... so, have fun.

Environment Info

  • Platform: darwin
  • Terminal: ghostty
  • Version: 2.1.119
  • Feedback ID: 849b4438-3231-415e-a615-aa06b49fdacf

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗