[Bug] Anthropic API Error: False positive safety classifier blocking legitimate mental health detection software
Bug Description
False positive from safety classifier ([bio]) blocking legitimate student mental health work.
Request ID: req_011CeCGAQP7r7Ahbbt4Rw98E
Product: Claude Code, Sonnet 5 model
Summary: I received the error "API Error: Sonnet 5 can't help with this" with detail [bio] during a software development session. The reported category (biological) does not match the content, which was pure engineering work.
Context: I'm developing EduAssistant, a student wellbeing tool that uses structured assessments to help detect bullying, suicidal ideation, self-harm, and domestic violence, so that clinical/school staff can intervene in time. The system has clinical and legal approval gates before any real alert can be triggered.
What I was doing at the time of the block: Writing tests (TDD) for HTTP header dispatch to the system (X-Student-UUID, handling of 400/422 responses). Purely technical content: headers, status codes, alert logic. No biological, medical, or weapons-related content whatsoever.
Hypothesis: The classifier likely reacted to the high density of sensitive vocabulary (suicidal ideation, self-harm, bullying appear repeatedly as variable names, test fixtures, and comments), without distinguishing between harmful content and code that names these topics in order to detect and prevent them. This is a protective use case, not a harmful one, that appears to have been misclassified.
Impact: This blocks workflow on a legitimate, well-documented project for student mental health prevention.
Request: Please review whether this was a false positive, and consider whether the classifier can be tuned to reduce false positives in software development contexts (tests, fixtures, variable names) related to harm detection/prevention, distinguishing them from content that actually describes or facilitates harm.
Environment Info
- Platform: linux
- Terminal: vscode
- Version: 2.1.235
- Feedback ID: 7ea1bdc1-e0fb-431c-a48a-875ec552e212
Errors
[]