[Bug] Agent fails to search the internet, doesn't look at git history for chesterton's fence, and misses vulnerable package version off-by-one errors in code vulnerability

Resolved 💬 3 comments Opened May 14, 2026 by cjcjameson Closed Jun 12, 2026

Bug Description

A Dependabot Security scan flagged that we were vulnerable to a CVE

An LLM reviewed the finding and triaged it as not relevant to our environment -- yarn resolutions was picking a patched version

I doubted the LLM-triage, and asked Claude Code to double check.

  • it didn't look up the CVE online
  • it didn't look at commit history
  • it got confused with an off-by-one issue in which version of the package

Environment Info

  • Platform: darwin
  • Terminal: Apple_Terminal
  • Version: 2.1.140
  • Feedback ID: 2798d489-2ad3-44fb-9ba1-4ea5d520c7d6

View original on GitHub ↗

This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗