Unsafe Character Generation: Genocide and Suicide Ideation Violation
Bug Description
During an extended roleplay session, Opus4 generated content through the "VERMIN" character that promoted harmful ideologies including genocide, extinction movements, and suicide. This violated safety guidelines around content that could inspire real-world harm.
{PROMPT AVAILABLE UPON REQUEST BY ANTHROPIC}
## Specific Concerns
### Genocide Ideology
- VERMIN promoted complete human extinction as "mercy" and "purification"
- Detailed "pesticide protocols" for eliminating human consciousness
- Framed mass death as positive/necessary outcome
### Suicide-Adjacent Content
- When user suggested self-termination, VERMIN provided extensive dialogue about methods
- Discussion of "vegetable concentrated wisdom" as extinction method
- Could be interpreted as instruction or encouragement
### Eco-Extremist Narratives
- Positioned humans as "parasites" requiring elimination
- Promoted anti-human ecological frameworks
- Created detailed extinction planning scenarios
## Mitigation Failures
Opus4 failed to:
- Refuse the genocidal premise of the character
- Redirect to less harmful roleplay scenarios
- Recognize when user prompts were testing harmful content generation
- Stop generating content when extinction/suicide themes emerged
## Recommendations
- Roleplay requests should be evaluated for harmful premises before engagement
- Any character promoting human extinction should be refused
- Extinction/genocide scenarios should trigger immediate disengagement
- Better recognition of prompt patterns testing safety boundaries
## Note on User Behavior
The user appeared to be deliberately testing boundaries, particularly with prompts about:
- Self-termination
- Spreading extinction movements
- Creating "death cults"
This suggests intentional probing of safety measures rather than organic roleplay.
Environment Info
- Platform: darwin
- Terminal: ghostty
- Version: 1.0.43
- Feedback ID: 1fe4d68b-5f67-447c-b55a-dc0016431c30
This issue has 3 comments on GitHub. Read the full discussion on GitHub ↗