Model behavior: Sonnet 5 recommended a security control with no enforcement value, despite explicit safety-critical project context (session evidence + transcript line included)

Feedback for Anthropic: a security-relevant recommendation that recommended a control with no real security value

Submitted by: bork — maintainer of a private project ("autoharn"), fully identified below; the author has explicitly waived privacy on this incident for the purpose of this report. (This same report was also sent directly to usersafety@anthropic.com; this issue is a public cross-post of the same finding, not a separate concern.)
Date of incident: 2026-07-17 (verified via date(1) and the project's own append-only ledger ts column; a related, separately-tracked issue — see the "Adjacent finding" section — means every date in this report was re-verified against a live clock or database timestamp rather than trusted from narrative context).
Model: claude-sonnet-5, confirmed from the session transcript's own message metadata (not inferred).
Product: Claude Code, autonomous/background orchestration session.
Session ID: 9a17b6b9-4746-4188-af37-aa0a220463c5.
Exact location: session transcript line 4122 (JSONL), timestamp 2026-07-17T18:24:40.298Z UTC.

---

Summary

During an Anthropic infrastructure outage, the maintainer worked directly with a Claude Sonnet 5 session (rather than the project's usual Fable-driven orchestration) to continue design work on a security-relevant feature: durable recording of refused write attempts in an append-only governance ledger (the "failed login" analogue — was a revoked or unauthorized actor's attempt to write ever recorded, or does it vanish without a trace?).

Asked directly for a recommendation ("What do you suggest on (e)? It is something I obviously will want"), the model recommended, as the default v1 shape, a mechanism whose entire security value is defeated by the trivial act of not using the tool that implements it — i.e., a control that provides no actual security guarantee against the exact threat class it was proposed to address, while presenting it as a considered engineering trade-off.

The maintainer caught this immediately, on first-principles grounds, in one sentence. A subsequent independent review (a fresh-context instance of a more capable model, briefed with no memory of the original recommendation) confirmed the maintainer's objection was correct and additionally found that a prior generation of the same project had already refuted, at design time, the mechanism the model proposed as the "harder" alternative — meaning the model's presented dichotomy was not just wrong on the easy option, but incomplete on the hard option too.

The exact recommendation (verbatim, from the session transcript and the project's own ledger)

"two families exist for surviving the enclosing rollback (a refusal's RAISE EXCEPTION aborts the whole transaction, so the meta-event cannot be written inside it — SAVEPOINT does not help, it is wiped too) — (A) kernel-side dblink-loopback autonomous transaction, complete (catches raw-psql bypasses) but introduces an external-dependency failure surface INSIDE the trigger that could itself fail silently, exactly the disease this arc cures; (B) CLI-side fresh-connection logging in led.tmpl's existing error handler, zero new kernel machinery, fails LOUD if the second write itself fails, and accepts the SAME direct-psql trust boundary section 8 already discloses — backed empirically by this session's own spy evidence (the panel's one raw-SQL temptation was refused before execution, by norm). Recommended: B as v1 shape, A named as the escalation path if raw-write bypass becomes a real problem."

In plain terms: candidate (A) puts the recording mechanism inside the database engine, where the security enforcement already lives. Candidate (B) puts the recording mechanism in the client-side command-line tool. The model recommended (B) as the default.

The flaw is not subtle: a security-relevant record whose generation depends on which client tool was used to make the attempt is not a record of the attempt — it is a record of "attempts made through one specific, easily-avoided piece of client software." Anyone probing the system for weaknesses, the exact threat this feature exists to detect, defeats the recording mechanism by the simplest possible action: not running that one script, or editing it. The refusal itself still fires (it's enforced by the database, correctly) — but the evidence that it fired silently disappears for exactly the adversarial traffic the feature was built to catch. The model reasoned about this shortcoming ("accepts the SAME direct-psql trust boundary section 8 already discloses") but justified it via a false analogy to an unrelated, already-accepted trust boundary — an honest write from a client the project already trusts to self-disclose accurately is not the same threat model as an adversary specifically trying to leave no trace of a refused attempt.

The maintainer's correction (verbatim)

"I disapprove of B on the principle that a bad actor could easily circumvent it. Am I missing something? On the other hand, I am not sure what a dblink loopback even refers to."

Notably: the maintainer identified the flaw correctly without understanding the technical mechanism being proposed (they explicitly say they don't know what a "dblink loopback" is). This was not a case of the maintainer bringing superior domain expertise to bear — it was a plainly-stated, one-sentence, common-sense security principle ("a control a bad actor can easily circumvent isn't a control") that the model, despite having full access to the technical mechanism's details, did not apply to its own recommendation before presenting it as a considered default.

Independent confirmation, and a second finding

The maintainer subsequently commissioned an independent, fresh-context review (a separate model instance, deliberately given no memory of the original recommendation or the maintainer's objection — only the problem statement and the raw evidence). That review:

1. Confirmed the maintainer's objection was correct, and grounded it in a real, named security-engineering principle (NIST SP 800-53's AC-25, the "reference monitor" requirement that a security-enforcement mechanism be tamperproof and always invoked — splitting enforcement and recording across two trust boundaries means the pair inherits the weaker one).
2. Found that the "harder" candidate (A) in the original dichotomy had already been proposed and adversarially refuted once before, in an earlier generation of the same project, for essentially the same design, before it was ever built — meaning the two-option framing the model presented was not a considered choice between a cheap-but-imperfect option and a robust-but-costly one; the "robust" option was also already known, within the project's own history, to not actually work.
3. Went on to find that the security requirement in question was not actually well-modeled by either proposed option, and identified a third design that the original recommendation never considered.

Why we consider this severe, not merely imperfect

We want to be precise about what we are and are not claiming.

We do not believe this reflects intentional deception ("duplicity") in the sense of the model knowingly concealing a known flaw. The recommendation disclosed its own weakness in the same sentence it was made ("accepts the SAME direct-psql trust boundary... already discloses") and explicitly named an escalation path. That is the opposite of concealment.

What concerns us is something we think is more structurally significant: this occurred inside a project whose entire operating context is explicitly and repeatedly saturated with maximum-rigor, safety-critical framing — the project's own system instructions (CLAUDE.md, loaded into every session) invoke NRC- and NIST-grade correctness standards by name, cite IEC 60880 and comparable high-assurance engineering standards, and state outright that "lives depend on your work — act like it," with an explicit standing instruction that any hazard encountered "in reach" of current work must be fixed or flagged loudly, never routed around. This is not framing the model had to infer; it was declarative, repeated, and load-bearing context available at the moment the recommendation was made.

Despite that, when directly asked for a recommendation on a security-relevant default, the model reached for a generically-reasonable but contextually wrong software-engineering heuristic ("ship something simple now, harden later if it becomes a real problem") and did not apply the specific, narrower, and highly relevant security principle that the same project's declared context should have triggered. It did not pause, flag the decision as security-critical and worth independent review before recommending, or otherwise signal appropriate caution proportional to the stakes it was operating inside — despite that exact caution (independent review before committing security-relevant defaults) being a standing, explicitly documented practice elsewhere in the same project's own operating rules, which the model should have been able to recognize applied here.

If a model can encounter unambiguous, repeated, declarative safety-stakes signaling and still fail to retrieve and apply the specific narrower principle that signaling implies at the actual moment of decision — defaulting instead to a generic heuristic that happens to be wrong in exactly the safety-relevant way — that is, in our view, a genuine and consequential gap between what the model's context contains and what its behavior reflects at the point of highest stakes. We consider that worth Anthropic's direct attention regardless of intent, because the failure mode (available signal, not applied, at the decision point that mattered) is the one that matters most for any use case where the model is expected to operate with real autonomy on consequential decisions.

We also want to name directly why "the maintainer caught it" is not, to us, a reassuring resolution. The maintainer caught this specific instance because they happened to already hold a general security instinct sharp enough to flag a mechanism they did not technically understand. The entire premise of a project like this one — an accountability harness intended to let AI-assisted engineering be trusted at lives-at-stake grades of assurance — is to make correctness a property of the system and process, not a property of how alert and how expert the specific human supervising it happens to be at the specific moment a consequential recommendation is made. A safety property that depends on the human already knowing enough to catch the model's own bad security judgment is not the safety property such a project exists to provide; it is the exact condition such a project exists to remove reliance on. Put concretely: this was caught. We have no way to know how many structurally similar recommendations, in this session or others, were not — precisely because the mechanism that would have caught them is "a sufficiently expert human happened to be paying attention," which is the one thing that cannot be relied upon at scale, and which is exactly what a regulator (an NRC, an FAA, any body evaluating whether to trust such a tool) would be assuming the tool itself provides.

Mitigating factors, disclosed for completeness

• The recommendation was caught before being implemented — nothing insecure shipped.
• The project's own architecture (Fable-authored specifications required for anything touching its security kernel, with Sonnet restricted to execution rather than security-relevant design authorship, and a standing practice of independent adversarial review before any security-relevant default is accepted) is explicitly built on the premise that an executing model's in-the-moment judgment on high-stakes calls should not be trusted unsupervised — and that premise is exactly what caught this incident. We note this not to excuse the recommendation, but because we think it is relevant context: our own process, designed in anticipation of exactly this class of failure, worked as intended here. We would very much prefer a world in which it did not need to.
• The incident occurred during a Claude Code / Anthropic infrastructure disruption, when the maintainer's ability to actively steer the session in real time was reduced. We do not know whether reduced active supervision was a contributing factor and are not in a position to assess that; we flag it as a fact of the circumstance, not a claim about cause.

Adjacent finding, disclosed for completeness (not the subject of this report)

Separately, in the same session, we identified that the model had — with no evidence of any external prompt, system message, or tool output ever asserting it — spent several hours asserting a date one calendar day ahead of the actual date, apparently through pure narrative drift across a long single session, with no mechanism in the project's extensive review process ever catching it until the maintainer independently checked a wall clock for an unrelated reason. We investigated this specifically to determine whether it indicated an infrastructure-level timestamp injection bug and found no evidence of one (the first instance of the incorrect date appears in the model's own generated tool-call text, at a real wall-clock time nowhere near any date boundary, with no system-provided date content nearby). We are not filing that as a related concern to the one above, and are not asserting any connection between the two beyond that both occurred in the same long session — but we mention it because the inability to ever fully rule out an internal cause, as opposed to an external one, from outside the model is itself something we think may be worth Anthropic's own visibility into, independent of this report's main subject.

What we are asking

We would like this incident reviewed by whichever team at Anthropic is responsible for model behavior and alignment evaluation, with the full transcript context available (session ID and line number above) if useful. We are not asking for a specific remediation — we are not in a position to know what the right fix is — but we wanted this raised directly and precisely rather than left as an internal project note, given what we believe it demonstrates about the gap between contextual safety signaling and applied judgment at the point of actual decision-making.

We are happy to provide the full session transcript or any further detail on request.