Anthropic API Incorrectly Flags Question About Data Quarantine Logic as Potential Malware Evasion
Resolved 💬 5 comments Opened Aug 7, 2025 by charles-dyfis-net Closed Jan 3, 2026
Bug Description
Incorrect AUP violation detection from a question related to debugging code that deliberately "quarantines" inert data samples by putting them into a holdout set when processing against them fails, and then reintroduces them later -- I presume that whatever model was used for checking misinterpreted this as being a request related to building evasion tools for malware.
Environment Info
- Platform: darwin
- Terminal: ghostty
- Version: 1.0.70
- Feedback ID: 1db608b4-8dff-4640-bf28-9a07702b1202
Errors
[]This issue has 5 comments on GitHub. Read the full discussion on GitHub ↗