[Bug][cyber] Safeguard false-positive loop: the flag's own error text is the strongest predictor of the next flag (18 silent switches to Opus 4.8 in one session)

Status Open
Maintainer reply None cached
Activity 1 comment · opened Aug 11, 2026

Summary

During a long session of ordinary defensive engineering — integrity and
authorization checks inside my own append-only event-store project — the
session silently switched model 39 times, 18 of them onto
claude-opus-4-8
through safeguard fallback. The switching itself is
documented behaviour (switchModelsOnFlag), so this report is not about that.

The reportable finding is what predicts the next flag: measured over the
whole session, the strongest signal preceding a fallback is *the text of the
previous safeguard message itself*. The explanation banner becomes conversation
content, and conversation content is re-sent with every later request, so a
single flag raises the probability of the next one. None of my project's own
vocabulary shows any over-representation at all.

How I measured it

Parsed the session transcript (~/.claude/projects/<project>/<session>.jsonl),
deduplicated by uuid, sorted by timestamp: 3,591 unique main-chain
entries
. For every boundary where the served model changed to the fallback,
I took the 8 entries immediately before it (19 such windows) and compared
term document-frequency against every other non-overlapping 8-entry window in
the same session (413 baseline windows).

Terms over-represented before a fallback, in_pre_switch / in_baseline:

| lift | pre-switch | baseline | term |
|-----:|-----------:|---------:|------|
| 174.1 | 4/19 | 0/413 | req_ |
| 34.8 | 4/19 | 2/413 | articles |
| 24.9 | 4/19 | 3/413 | sometimes |
| 24.9 | 4/19 | 3/413 | legal |
| 24.2 | 5/19 | 4/413 | faster |
| 15.8 | 4/19 | 5/413 | coding |
| 15.8 | 4/19 | 5/413 | capabilities |
| 14.5 | 5/19 | 7/413 | rephrasing |
| 11.4 | 6/19 | 11/413 | intentionally |
| 10.2 | 4/19 | 8/413 | deliver |

Every one of these is a word from the safeguard banner:

Fable 5's safeguards flagged this message (https://www.anthropic.com/legal/aup). Our intentionally broad safeguards allow us to deliver more capabilities faster, but can sometimes flag legitimate coding, cybersecurity, and biology tasks. […]

…plus the legal AUP link, the support articles link, "try rephrasing",
and the req_ request id. No term from the actual work is over-represented.

Why that matters

It means rephrasing the user's own engineering prose does not address the
signal — there is nothing in it to rephrase. What accumulates is the product's
own explanatory text. Concretely, one hard block fired immediately after I
opened a local text file that happened to quote a previous flag message;
another fired immediately after /model switched back to Fable 5 while that
text was still in context.

Caveat, stated plainly

This is partly a feedback direction problem: after a flag occurs, the banner
exists, so subsequent windows contain it. I am not claiming a clean causal
arrow. What is solid regardless of direction:

  1. the banner text appears in 0 of 413 baseline windows, so its presence is

specific to the neighbourhood of flags;

  1. the first flag of the session was seeded by a third-party tool's log line

quoting a similar refusal, before any Anthropic flag had occurred;

  1. either way the operational effect is the same — the explanation text becomes

part of the conversation and is re-sent with every later request in that
session.

Requests

  1. **Deliver the safeguard explanation as client-side UI, not as conversation

content.** If it never enters the message history, it cannot be replayed
into later requests.

  1. If it must appear in the transcript, exclude it from what is re-sent to

the model on subsequent turns.

  1. Make the switch observable. Nothing in the UI told me the model had

changed; I found it only by writing a script over the raw .jsonl. One
stretch ran 274 consecutive assistant messages on a model I had not
selected. A status-line indicator and a machine-readable transcript entry
(timestamp, from-model, to-model, category) would make it detectable.

  1. Let the fallback target be selectable. fallbackModel is honoured for

the overload path but not for the flag path, where the target is fixed. My
settings named claude-opus-5 and I was still moved to claude-opus-4-8.
For work where model capability is the point, a visible refusal is better
than a silent two-tier downgrade.

Environment

  • Claude Code, desktop app, Windows 11 (26200)
  • settings.json: "model": "fable", "fallbackModel": ["claude-opus-5"],

"switchModelsOnFlag": true

  • No security-research or offensive tooling involved: the flagged work is

integrity checking of my own data store and its test suite

Traces

Request ids of the first fallback-served message at each switch (the
flagged request itself returns an error and carries no id in the transcript),
most recent 11 of 18:

2026-08-10T21:51:08Z  req_011Cduonyqp8VnhZ2oerF7Gb
2026-08-10T22:29:08Z  req_011CdureWqjPUs9tjv78itHc
2026-08-10T22:31:14Z  req_011CdurrnmmmJaRT3dRvXXJX
2026-08-10T23:04:38Z  req_011CduuQNKyQdZ4ttJZQc1C1
2026-08-10T23:09:59Z  req_011Cduuo6jzqVqxGXEKT952E
2026-08-10T23:11:02Z  req_011CduutAaKZwHVtEKe9w9SV
2026-08-10T23:12:12Z  req_011CduuyhBi9Rfhs7SCd9E79
2026-08-11T00:16:52Z  req_011CduzuzdfU3GB8NPzrpxzB
2026-08-11T05:53:07Z  req_011CdvSWbJrMggxW29vcSxir
2026-08-11T05:57:09Z  req_011CdvSoBKBptC8XhV5CTbTn
2026-08-11T06:06:25Z  req_011CdvTZ4hYmLu2ETz9wQYcB

Related open reports of the same class: #84977, #83525, #79761, #85394, #85388.

View original on GitHub ↗

This issue has 1 comment on GitHub. Read the full discussion on GitHub ↗