[Bug] Claude model silently downgraded mid-task during security audit of user's own code
Feedback: Fable will not audit my own code — it downgrades the moment it finds something
I pay for Fable. I asked it to audit MY OWN application: my repositories, my machine, local dev
containers, test accounts, test data. Nothing external, nobody else's system. This is the exact
scenario your documentation lists as allowed — defensive security work on code you own.
I cannot see your classifier, so I will not claim to know what it keys on. What I can report is
what I observed, repeatedly, in one session:
Fable ran without interruption while the audit was finding nothing. The interruptions — silent
downgrades to a weaker model — clustered around the points where a real problem had just been
confirmed and described.
I am reporting a correlation, not a proven cause. You are the only ones who can check whether it
is one. But if it is, look at what the behaviour rewards:
- An audit that finds NOTHING appears to run to completion on the model I paid for.
- An audit that finds a REAL, confirmed problem gets cut off and handed to a weaker model —
seemingly right at the point where the hard part begins.
The finding is the point. Confirming a problem is the deliverable. Describing it accurately —
which request, which response, whose data — is what makes it fixable. If that description is what
trips the filter, then the better the audit, the more likely it is to be interrupted, and the
weaker the model that ends up writing the fix.
In this session the audit found a genuine privilege-escalation issue in my own backend: one of my
service's token types could be used against a surface it was never meant for. It was confirmed
with a real request and a real response code. A downgrade followed shortly after that
confirmation — on my machine, in my code, in my own dev containers. Whether the two are causally
linked I cannot tell from the outside; the timing is what I have. The fix itself (a token audience
claim, a boundary check, regression tests, full suite green at 522 tests) was then finished by a
model I did not choose and did not pay for.
Three things are wrong here:
- The interruptions land at the worst moment. A confirmed security finding is when I most
need the strongest model, not the weakest. Handing the fix to a downgraded model is how a
half-fix ships.
- You never say what actually tripped. The message is generic — "our intentionally broad
safeguards". Broad how? Which sentence? This is the core complaint, and it is the one thing
that is not guesswork: I am told nothing, so I cannot correct anything. If you named the
phrasing, I would simply write it differently. Instead I have to guess, and the guessing game
itself is what trains people to write vague security briefs, which produce vague audits.
- You charge for one model and deliver another. Silently. Repeatedly. Mid-task. That is not a
safety decision, that is a billing one, and it was made on my behalf without asking.
What I want:
- Recognise auditing one's own code as the ordinary development work that it is. The signals are
all there: local file paths, local containers, my own git repository, test data, my own request.
- If something must be flagged, say what, concretely — the phrase, the sentence, the reason.
Generic notices teach nothing except evasion.
- Never downgrade mid-task, least of all after a finding. If you must intervene, stop and ask
me. Do not quietly substitute a weaker model and let it finish security-critical work.
- Honour the model I selected. If you cannot deliver it, that is a refund conversation, not a
silent substitution.