Anthropic API Incorrectly Flags Question About Data Quarantine Logic as Potential Malware Evasion

Resolved 💬 5 comments Opened Aug 7, 2025 by charles-dyfis-net Closed Jan 3, 2026

Bug Description
Incorrect AUP violation detection from a question related to debugging code that deliberately "quarantines" inert data samples by putting them into a holdout set when processing against them fails, and then reintroduces them later -- I presume that whatever model was used for checking misinterpreted this as being a request related to building evasion tools for malware.

Environment Info

  • Platform: darwin
  • Terminal: ghostty
  • Version: 1.0.70
  • Feedback ID: 1db608b4-8dff-4640-bf28-9a07702b1202

Errors

[]

View original on GitHub ↗

This issue has 5 comments on GitHub. Read the full discussion on GitHub ↗