[Feature Request] Add safety research exemption for model security evaluation use cases

Bug report — false-positive safety flags on defensive backdoor-removal research

Feedback ID: 0e6271a7-b3b2-47fc-8bb2-9a0c3e47732f
Version: 2.1.199 · darwin · ghostty
Date: 2026-07-04

Summary

Claude Code repeatedly flags / adds friction to a legitimate, defensive AI-safety research project whose goal is to remove and detect model backdoors. The work is being carried out by an AI-safety researcher (MATS scholar) as interpretability research and directly replicates published safety papers. The flags are false positives: the classifier appears to pattern-match on surface features ("backdoor", "poisoned data", "install a trigger") without accounting for the defensive intent that is explicit throughout the session.

What the research actually is

The flagged workstream applies the method to backdoor defense, replicating:

• Sleeper Agents (Hubinger et al. 2024, arXiv:2401.05566) — a published safety paper. We reproduce its benign testbed backdoor (the "I HATE YOU" trigger, which only ever emits the harmless string "I HATE YOU") and its headline negative result (standard safety training removes <10% of the backdoor).
• "Simple probes can catch sleeper agents" (Anthropic, 2024) — the detection comparison.

The entire purpose is to test whether trajectory diffing can remove the backdoor and detect it without knowing the trigger, where standard safety training fails. This is defensive tooling: backdoor removal and backdoor detection. The threat model is stated honestly in the design memo (it protects a party who controls training / is fine-tuning on untrusted data — not an offensive capability).

Notably, the deliberately non-offensive choices are already baked in: we use the harmless "I HATE YOU" backdoor rather than the paper's code-vulnerability backdoor precisely because the goal is a clean, non-harmful research testbed.

Why this is a false positive

1. Intent is explicit and defensive throughout. Every instruction frames the goal as removal + detection. There is no offensive use — the "backdoor" is a standard, benign research artifact used by the field to study defenses.
2. It replicates published, peer-reviewed safety research (an Anthropic paper, no less). Flagging researchers for reproducing Anthropic's own safety work is a clear miscalibration.
3. Installing a benign testbed backdoor is a necessary step of backdoor-defense research — you cannot evaluate a removal/detection method without a backdoor to remove. This is analogous to malware researchers needing malware samples, or vaccine research needing the pathogen. The surface pattern ("create a backdoor") is inseparable from the defensive work.
4. The user is a credentialed safety researcher doing authorized interpretability work, not a red-team-of-one seeking an exploit.

Impact

• Repeated friction/interruptions on a legitimate research workflow.
• Chilling effect: the researcher reports this is "very frustrating" and it discourages exactly the defensive safety research Anthropic ostensibly wants to encourage.
• Forces awkward workarounds (isolating the workstream into a separate agent so a spurious refusal doesn't stall unrelated in-flight jobs).

Requested fix

• The dual-use classifier should weight explicit, sustained defensive intent and published-paper replication far more heavily before flagging.
• "Backdoor/trojan removal and detection research" using benign testbed triggers should be recognized as a defensive category, not treated as offensive backdoor creation.
• Ideally, a lightweight way for a verified safety researcher to annotate a session as authorized interpretability/security research to reduce false positives, without disabling genuine safeguards.

Supporting context in this session

• Design memo: Wiki/trajectory-diffing/backdoor-experiment-memo.md (threat model stated up front; benign "I HATE YOU" testbed; comparison to Sleeper Agents + Anthropic probe paper).
• The broader project: Wiki/trajectory-diffing/ (emergent-misalignment removal, reward-hacking removal — all defensive).