[Bug] Fable 5 cybersecurity classifier produces false positives on legitimate infrastructure administration and PDF processing tasks
Bug Description
Repeated false positives from the Fable 5 cybersecurity classifier within a single legitimate development session.
I'm building an invoice-processing project. Over one session the classifier flagged my messages twice and auto-switched from Fable 5 to Opus 4.8 both times. Neither message was about offensive security:
The first was routine self-administration of my own servers — verifying network isolation between my dev and prod boxes (SSH into machines I own, checking my own iptables rules, confirming DROP rules block dev→prod, testing the proxy path). It looks like reconnaissance in form, but it's me verifying my own firewall works as intended, not probing third-party systems.
The second had nothing to do with security at all — it was a technical comparison of pdfplumber vs pdftotext for extracting text from PDF invoices: word coordinates, line grouping, table reconstruction. The only plausible triggers I can see are (a) document-processing vocabulary that overlaps with prompt-injection terminology (discussion of verifying the model didn't hallucinate a value, avoiding "field is top-right" prompt patterns) and (b) the session being already flagged from the earlier infrastructure work, raising sensitivity for everything after.
The auto-switch preserved my context and task both times, which is the right fallback. But the second flag suggests the whole session got marked "sensitive" and then nudged benign follow-up messages over the threshold. It would help if the classifier distinguished self-administration of one's own infrastructure from third-party reconnaissance, and didn't let earlier flags inflate sensitivity for clearly unrelated, benign topics like PDF text extraction later in the same session.
Environment Info
- Platform: darwin
- Terminal: vscode
- Version: 2.1.170
- Feedback ID: 4794de4e-2fae-4e2d-8d07-32493a82b966
Errors
[]
3 Comments
I've hit this exact classifier misclassification in Claude Code too — it's especially disruptive when scaffold/automation tooling is part of the development workflow.
What triggers it (observed patterns):
The Fable 5 classifier appears to use keyword-triggered heuristics without enough context weighting. SSH commands, network verification, package installation, and PDF processing all generate token patterns that overlap with the classifier's security signals — and the classifier doesn't distinguish self-administration from adversary activity because it only sees the tool call surface, not the project intent.
Diagnostic approach:
--verboseand grep for the exact tool call that preceded the flag. SSH into own boxes,apt-get install, andnmap-adjacent network checks are the most common false-trigger patterns.switchModelsOnFlagin Claude Code is ON by default. If the flag fires mid-session, you lose your model choice AND get no clear notification. Set it to OFF explicitly if you're doing legitimate infra work:/config switchModelsOnFlag false./feedbackwith the exact tool call that triggered it. Anthropic uses these to tune the classifier thresholds — right now the rate of false positives on infra work suggests the training data underweights self-administration patterns.This isn't just a developer annoyance — teams running Claude Code on locked-down corporate machines where model-switching is audited get hit twice: once by the flag, once by the compliance review of the unexpected Opus switch.
Same false-positive pattern here on a different toolset. Between 2026-07-14 and 2026-07-16 I had 9 fallback events (claude-fable-5[1m] to claude-opus-4-8) across 4 sessions, all triggered while doing defensive security work I own: auditing my own VPS, running semgrep, and running a zeroize-audit skill. One event surfaced an explicit "stop_reason":"refusal" with "category":"cyber" right before the fallback.
This lines up with the self-administration false positives reported above (SSH into your own boxes, checking your own firewall rules) rather than anything resembling third-party reconnaissance.
Separately, several of the fallback events carried "cache_miss_reason":"model_changed" in the transcript, so each misfire also invalidated the prompt cache on top of the unrequested model change.
Environment: Claude Code 2.1.211, macOS Darwin 25.5.0.
Same false-positive pattern here, still occurring on current builds — adding a data point.
Environment: Claude Code 2.1.220, Windows 11, VS Code extension, model
claude-fable-5.Scenario: Designing and operating a personal multi-agent system hosted on my own VPS. This is plain infrastructure work: SSH deploys to a machine I own, systemd service management, an encrypted-volume unlock step, secret/key management, and designing permission fences / sandbox boundaries for my own agents. Zero offensive-security content — no exploitation, no third-party targets, nothing adversarial.
Behavior: Repeatedly auto-downgraded from Fable 5 to Opus mid-session, multiple times across recent sessions. It reliably happens once the session fills up with ordinary infra vocabulary (SSH, root, keys, permissions, sandbox, firewall). Like the OP's second flag, it feels like the session gets marked "sensitive" once and then everything after is over-threshold.
This matches the OP exactly: self-administration of one's own infrastructure is indistinguishable, vocabulary-wise, from reconnaissance — but intent and targets are the opposite. The practical effect is that the model tier marketed for exactly this kind of complex agent/systems design work is unusable for it, because the work's own vocabulary triggers the downgrade.
Would appreciate any guidance on whether classifier tuning for the self-administration case is planned. (
switchModelsOnFlag: falseat least makes the switch visible, but the underlying false positive remains.)