Anthropic API Incorrectly Flags Question About Data Quarantine Logic as Potential Malware Evasion
Status Closed — not planned
Maintainer reply None cached
Activity 5 comments · opened Aug 7, 2025 · closed Jan 3, 2026
Bug Description
Incorrect AUP violation detection from a question related to debugging code that deliberately "quarantines" inert data samples by putting them into a holdout set when processing against them fails, and then reintroduces them later -- I presume that whatever model was used for checking misinterpreted this as being a request related to building evasion tools for malware.
Environment Info
- Platform: darwin
- Terminal: ghostty
- Version: 1.0.70
- Feedback ID: 1db608b4-8dff-4640-bf28-9a07702b1202
Errors
[]This issue has 5 comments on GitHub. Read the full discussion on GitHub ↗