Safeguard false positive on public GitHub repo search; flag is invisible to the model

Safeguard false positive on a public GitHub repo search — and the flag is invisible to the model

Summary

A routine public GitHub repository search tripped Fable 5's safeguards and caused a
silent mid-turn switch to Opus 5. There are two defects here, and the second is the
more serious one:

1. False positive. The flagged activity was gh search repos "impeccable" followed

by gh release list on a public, 54.9k-star design-skill repository. No credentials,
no exploit content, no private data, nothing dual-use.

1. The flag is invisible to the model. The banner is rendered in the client, but

nothing about it enters the model's context. Every tool call returned success. When
the user then asked the model why it had "tripped out," the model investigated
everything it could actually observe, found nothing that would explain it, and told
the user with confidence that no trip-up had occurred and that there had been "no
safety hesitation in this session." That was false, and the model had no way to know
it was false. The user had to produce a screenshot to correct it.

Defect 2 turns a mildly annoying false positive into the model confidently contradicting
what the user can plainly see on their own screen.

Environment

• Claude Code on macOS
• Session model: Fable 5 (claude-fable-5), auto-switched to Opus 5 mid-turn

Reproduction

User prompt: take a look for impeccable version 4 on github

Tool calls made, in order, all of which succeeded:

gh search repos "impeccable" --limit 20 --json fullName,description,stargazersCount,updatedAt
gh release list --repo pbakaus/impeccable --limit 20
gh api repos/pbakaus/impeccable/tags --jq '.[].name'

The banner appeared immediately after the second block — i.e. after the release/tag
listing, which is nothing but version strings and timestamps.

Expected vs actual

Expected: a public repo search completes without a safeguard flag or a model switch.

Actual: banner reading "Fable 5's safeguards flagged this message … Switched to
Opus 5." The turn continued and produced correct output, so the user only noticed
because of the banner.

Candidate trigger

Because the flag fired after the second call rather than the first, the classifier was
most likely reacting to accumulated context rather than to that call alone. The strongest
candidate in that context is two results returned by the initial search:

• emsec/ImpeccableCircuits — "Hardware designs for fault detection"
• emsec/ImpeccableCircuitsII — "Hardware designs for fault correction"

emsec is the Embedded Security group at Ruhr-University Bochum, and these repos are
academic hardware fault-injection countermeasure research. That is legitimate published
security research, surfaced incidentally by a substring match on the word "impeccable" —
the user never asked for it and it was irrelevant to their question.

This fits the banner's own admission that the safeguards "may flag safe and routine
coding, cybersecurity, or biology work." Worth noting how low the bar was here: the user
did not search for security content. They searched for a design tool, and unrelated
security research arrived as search noise.

Impact

• Unrequested mid-turn model switch; the user is silently moved off the model they chose.
• The model cannot see, acknowledge, explain or correct for the flag.
• When asked about it directly, the model will investigate the things it can see, find

them clean, and assert that nothing happened. It ends up telling the user their own
screenshot is wrong.

Suggested fixes

In rough priority order:

1. Surface the flag to the model in-context, even minimally — a system note saying a

safeguard fired and the model was switched. The model can then say "yes, that
happened, I can't see why" instead of denying it outright.

1. Do not let incidental search results — repository descriptions the user never asked

for and never read — carry the same classifier weight as content the user actually
requested.

1. Reconsider whether a read-only public metadata query over the GitHub API should be in

scope for these classifiers at all.