[Bug] Opus 4.7 incorrect reasoning about URL normalization vs dereferencing

Resolved 💬 2 comments Opened Apr 27, 2026 by xpe Closed May 4, 2026

Bug Description
This is Opus 4.7 your flagship model, intended for use for developing Web APIs, Web apps, etc. And yet it cannot _reason_ correctly about the difference between URL normalization and URL deferencing. This will be a fun one to dig into if you have MechInterp tools.

This oversight has led to my current project being wrong, bloated, confusing, and frustrating. I can only imagine how many other people are finding Claude's reasoning to be off in areas where it arguably probably has a lot of training data? I don't know why this problem persists, but I did replicate it twice. So. Worth adding some evaluations for I think -- at the risk of overfitting and contaminating ... so, have fun.

Environment Info

  • Platform: darwin
  • Terminal: ghostty
  • Version: 2.1.119
  • Feedback ID: 849b4438-3231-415e-a615-aa06b49fdacf

View original on GitHub ↗

This issue has 2 comments on GitHub. Read the full discussion on GitHub ↗