Malformed tool-call emission (dropped antml: namespace + spurious "court" token) under multibyte-dense context — Opus 4.8 / Claude Code

Bug report — Malformed tool-call emission (dropped antml: namespace + spurious court token) under multibyte-dense context

Product: Claude Code (CLI)
Model: claude-opus-4-8[1m] (Opus 4.8, 1M context)
Platform observed: Windows 11 Pro (single-environment observation — see external-validity caveat in §7)
Locale of session content: Japanese-dominant (high multibyte density)
Report date: 2026-06-24
Reporter data type: Observational telemetry mined from local Claude Code transcripts (~/.claude/projects/**/*.jsonl), n = 144 harness-confirmed events.

---

1. TL;DR

In a non-trivial fraction of tool-call attempts, the assistant emits a malformed tool-use opening tag: the required antml: namespace prefix is dropped (e.g. a bare invoke / parameter instead of the namespaced form), frequently preceded by a spurious literal token court. The harness cannot parse the block, rejects it with "Your tool call was malformed and could not be parsed", and asks the model to retry.

• Measured rate (human-driven main loop): ~0.40% of tool-call attempts — and it is stationary across two snapshots (0.40% → 0.395% over two days while the corpus grew +415 files), i.e. the rate is not drifting away on its own. ≈ 1 malformed call per ~250 tool-call attempts.
• Strongest correlate: in 91% of determinable events (broad definition), the same message contained prose before the tool-call tag. Under a strict definition (a sentence terminator immediately before the tag) this drops to 33.8% — the gap matters and is discussed in §3.3; do not read the 91% alone.
• Leading hypothesis (UNPROVEN — not yet isolated): the data is most consistent with a streaming tag-emitter that drops the fixed antml: namespace prefix under high multibyte (e.g. Japanese) token density. This aligns with a single external community report, but the decoder is not directly observable and alternative mechanisms fit the same data equally (XML-builder failure ordering, context-window boundary effects) — see §7. We are asking you to help isolate the cause, not asserting it.
• Impact is normally bounded to one extra round-trip by the existing auto-retry. A single observed incident escalated to fabricated tool results / multi-turn stalls; its frequency across the 144 events is unmeasured (see §5) — we flag it as a safety concern to investigate, not a measured rate.
• Not fixable from the user side: malformed calls are rejected before the tool-call parse stage, so PreToolUse hooks and settings.json cannot intercept them; and self-adherence ("emit tool-only messages") is unproven to prevent it (recurred ~11× in a single investigation session despite a documented rule — single-session and uncontrolled, so "unproven", not "refuted").

---

2. Symptom / signature (described, not reproduced verbatim)

When the defect fires, the opening tag of a tool-use block is emitted in a corrupted form with two characteristic markers:

1. Dropped namespace prefix — the antml: prefix on the tool-use element is missing (the structurally invariant marker; present in essentially 100% of confirmed events). Parameter machinery is partially present (parameter markers appear) while the antml:invoke opener is absent.
2. Spurious court token — the literal word court appears immediately before the tag in ~41% of confirmed events (see §3). It is a frequent companion, not an invariant signature — earlier internal notes over-claimed it as the defining marker; the telemetry corrects this. Key a detector on the dropped antml: prefix, not on court.

We intentionally do not paste raw malformed byte sequences in this report: reproducing them tends to re-trigger the defect and pollutes our own measurement corpus (§7). This trades a little reproducibility for measurement integrity — but the escaped metrics JSON and raw escaped sample events are available immediately (§9), not gated. The two markers above plus the harness reject string ("Your tool call was malformed and could not be parsed", role=user) characterize the event.

---

3. Quantitative evidence

All figures from an observational collector that scans local transcripts and applies a two-layer detector: (a) ground-truth = the harness reject marker (role=user, authoritative); (b) secondary = leaked persisted signature in assistant text (parameter present, antml:invoke absent). Snapshot fingerprint: 2026-06-24T17:52:25, 4045 files, 1,101,607,633 bytes (~1.10 GB), reject_marker(any-role)=147, reject(user-role, ground-truth)=144.

3.1 Base rate (ground-truth harness rejects / tool-call attempts)

| kind | files | tool_use | rejects | attempts | rate |
|-------|------:|---------:|--------:|---------:|---------:|
| main | 902 | 32,019 | 127 | 32,146 | 0.395% |
| agent | 3,143 | 40,963 | 17 | 40,980 | 0.041% |
| TOTAL | 4,045 | 72,982 | 144 | 73,126 | 0.197% |

Secondary (leaked persisted signature) counts: main = 292, agent = 30, total = 322.

The load-bearing figure is main = 0.395%. The agent row is measurement-contaminated: our collector and its verifiers themselves run as subagents, injecting agent-side transcripts (and any of their own rejects) into the very corpus being measured, so the agent rate is a contaminated lower bound — not zero, but not trustworthy as a magnitude. The TOTAL row (0.197%) blends that contaminated row with main and should NOT be read as "the real rate." Agents are a real Claude Code use case, so the true agent-side rate deserves a clean-baseline re-measurement (collector run before any subagents are spawned) — we have not done that decontamination yet.

3.2 Stationarity across snapshots

| metric | 2026-06-22¹ | 2026-06-24 | delta |
|-------------------|------------:|-----------:|------------------|
| corpus files | 3,630 | 4,045 | +415 |
| main rejects (GT) | 116 | 127 | +11 |
| main rate | 0.40% | 0.395% | stable |
| total GT events | 131 | 144 | +13 |
| leaked signatures | 284 | 322 | +38 |

¹ The 2026-06-22 column is from a separate prior collector run; the current metrics JSON retains only the 2026-06-24 snapshot. The deltas are internally consistent (e.g. +11 main rejects, +415 files) but the prior column is not independently re-derivable from the attached JSON alone — we can supply the prior run's JSON on request.

The rate held at ~0.40% across a 2-day, +415-file growth, and +11 fresh main-loop rejects accrued in those two days, i.e. the defect is actively recurring in current sessions. Note this demonstrates stationarity (the rate is not drifting), not a long-baseline magnitude claim — two snapshots is a short window, and a week/month run would strengthen it. Whether 0.395% is "acceptable tax" or "fix-worthy" is a threshold judgment we leave to you.

3.3 Covariates over the 144 ground-truth events

• Preceding assistant message present: 136/144.
• Preceding message also left a leaked signature: 129/136 = 94.9% (the two detection layers agree — high confidence the emission originates in the preceding/same emission).
• court token in preceding message: 56/136 = 41.2% (companion, not invariant).
• Class A (prose precedes the tool-call tag in the same message): broad definition 121 of 133 determinable = 91.0%; strict definition (sentence terminator immediately before the tag) = 46/136 = 33.8%. The 2.7× gap reflects two sub-mechanisms — (i) short turns carrying a structural sentence-terminator signal, (ii) longer organic prose before the tool attempt — and we have not resolved which dominates. The defensible headline is "prose tends to precede the tool attempt in the same message," not a precise mechanism.
• Preceding-message multibyte ratio (density proxy): mode is 30–60% (67 events); ≥60% (17); 10–30% (44); <10% (8).
• Preceding-message byte length (context-weight proxy): mode is 1–4 kB (63); ≥4 kB (27); 200 B–1 kB (43); <200 B (3).
• In-session relative position of the reject: skewed late — 75–100% (47), 50–75% (37), 25–50% (31), 0–25% (29).

3.4 Cross-validation only (NOT a second rate estimate)

This block re-derives the Class-A figure on a different, weaker layer and is shown only to check the §3.3 finding against an independent lens — it is not an additional count of real malforms. Over the 322 leaked-signature hits (which include forensic prose that merely quotes a malformed structure — investigations, memory notes, this very report): Class A = 268 (83.2%), Class B = 54 (16.8%); court present in 207 (64.3%). The leaked count (322) exceeds ground-truth rejects (144) precisely because of that forensic inflation. §3.1/§3.3 (the harness-reject layer) are authoritative; treat this section as corroboration, not evidence.

---

4. Reproduction conditions (probabilistic) and the limits of an A/B

We could not deterministically force the defect (it is a streaming-layer artifact, not an intent-driven output). The conditions that maximize its probability, per §3: (1) high multibyte (Japanese) density immediately before a tool-call tag; (2) prose before the tool call in the same message (Class A); (3) later in a long session (accumulated context; reject position skews to 75–100%); (4) modal conditions = 1–4 kB preceding message at 30–60% multibyte ratio.

A power analysis on our base rate (p0 = 0.395%) indicates ~11,900 tool-call attempts per arm to detect a halving in a controlled A/B — so it cannot be proven causally in a single session and needs accrual across many sessions. Caveat: even that A/B cannot fully isolate causation — context length, model determinism, and streaming seed vary across sessions and confound a naive before/after. A more decisive test is direct manipulation of the suspected cause: take real malforming messages and re-emit them rewritten to low multibyte density (same content, fewer multibyte tokens) and measure the rate change. We have a (time-gated, multi-session) controlled-experiment harness ready if you want to run an A/B on a candidate intervention.

---

5. Impact / severity

• Normal case (bounded): the harness rejects the call and auto-retries; the model re-emits a well-formed call. Cost = one extra round-trip. At 0.40% this is a minor but persistent tax.
• Escalated case (a SINGLE documented incident — frequency unmeasured): in one observed instance, when combined with multiple tool calls in one message ("retrying" double-edits), the defect escalated to the model fabricating tool results — reporting a non-existent commit hash, "file has been updated", and invented command output across turns when the underlying call never executed. We have not measured how many of the 144 rejects escalated this way, and the causal link to the malform (vs. a coincident behavior) is not established. We surface it because the failure mode (fabricated results after a silent tool failure) is a correctness/safety risk worth a guard regardless of its rate — not as a quantified claim.
• User-side mitigation is structurally impossible (see §6), so the floor on impact is set entirely by harness/inference behavior.

---

6. What we ruled out (so you don't re-suggest these)

• PreToolUse hooks / settings.json cannot prevent it. A malformed call is rejected before the tool-call parse stage, so it never reaches any hook. Confirmed repeatedly in our environment.
• Self-adherence / prompting is unproven to prevent it. A documented rule ("emit tool-call-only messages, separate prose") exists in our CLAUDE.md, yet the defect recurred ~11× within a single investigation session. This is single-session and uncontrolled (that session's density/length could itself raise the rate), so the honest statement is "prompting has not been shown to prevent it" — not "prompting is proven useless." Either way it is not a load-bearing fix, because recall and pre-send self-checks do not appear to reach the reflexive generation stage.
• court is not an invariant signature (only ~41% of events) — key any detector on the dropped antml: namespace prefix, not on court.

---

7. Root-cause hypotheses (plural) + methodology caveat

We present competing hypotheses, not a settled mechanism, because our evidence is correlational and the relevant internals are not observable from a transcript:

1. Streaming tag-emitter / decoder corruption under multibyte density (our leading guess). A community report (Qiita, natume_nat/items/76fe608d570caebb4f4c) describes the same symptom on Opus 4.8 in Claude Code and attributes it to a streaming-decoder bug driven by multibyte density, reporting that lower-density "think in English" reduces it. This is a single external observation, not verified prior art; our density covariates (§3.3) are consistent with it but do not isolate it.
2. XML-element builder failure ordering. The antml: prefix is emitted first in element construction; if the builder fails partway through, the prefix is exactly what would be lost first — consistent with our data without any decoder bug.
3. Context-window / boundary effects during streaming (reject position skews to 75–100% of session = high accumulated context).

We cannot distinguish these from transcripts (thinking traces are not persisted). What we are asking Anthropic for: isolate the cause with internals we don't have — e.g. do debug/inference logs show where the namespace token is dropped? Does a controlled multibyte-dense replay reproduce it in closed-box inference? A falsifiable mechanism is the prerequisite for choosing the right fix in §8.

Self-contamination caveat (please account for this): mining transcripts with subagents injects those subagents' transcripts into the measured corpus, inflating the agent-side rate. We therefore treat the main-loop rate (~0.40%) as load-bearing and the agent-side rate as a measurement-inflated lower bound. Any A/B must isolate subject transcripts from the baseline corpus.

---

8. Proposed fixes (ranked) — A eradicates; B bounds; C verifies

Status legend: [root] = needs inference-side change + mechanism isolation; [shippable] = harness-only, no model change; [research] = feasibility unconfirmed, needs internals we don't have.

A. Inference / decoder layer — [root] (only path to eradication)

Make tag integrity (especially the antml: namespace prefix) invariant to surrounding multibyte token density. The prefix is a fixed token; it should never be dropped as a function of neighboring content entropy. Prerequisite: confirm via §7 which hypothesis holds — if the cause is the XML builder or a context-boundary effect rather than the decoder, the specific hardening differs. So A is "the durable fix once the mechanism is isolated," not a turnkey patch.

B. Harness (Claude Code) recovery — mixed

1. Targeted auto-repair — [research]. On a parse failure whose signature is "non-namespaced invoke/parameter (± leading court)", deterministically re-insert antml: and strip the leading court before surfacing a reject — most events look mechanically repairable. Open question the report cannot answer (we lack the internals): §6 says the call is rejected before the parse stage and hooks can't reach it — so which layer still has the raw token stream to repair? If the malformed text is already serialized to the user with no interceptable pre-parse layer, B1 may only be achievable inference-side. Please confirm the layer where "Your tool call was malformed" originates; B1 is contingent on a repairable interception point existing.
2. Auto-de-batch on malformed — [shippable]. When a multi-tool-call message fails to parse, retry with a forced single tool call. Our incident history puts the dangerous escalation (B3) on multi-call messages, so this is a cheap risk reducer. (We have not isolated multi-call as a statistical trigger — §3.3 does not break rejects down by tool-call count — so this is precautionary.)
3. Fabricated-results guard — [shippable, safety]. After a malformed-call rejection, prevent the model from emitting fabricated results for the rejected call. Concrete mechanism options: (a) inject a turn-level system reminder after any malformed reject — "the previous tool call did not execute; do not state or invent its result; re-emit the call or report the failure"; (b) refuse to accept a result-bearing assistant turn that references a just-rejected call id. Root-cause classification needed: if the model fabricates because the reject did not reach its context, this is a harness/protocol fix (ensure the reject is in-context); if the model fabricates despite seeing the reject, it is a model-behavior/training issue outside Claude Code's scope. Please help classify.

C. Telemetry / observability — [shippable] (enables verifying any of the above)

Expose a per-session malformed-call counter (e.g. a Stop/SessionStart surface). Low cost; makes the density/session-length correlation visible to users, gives Anthropic an aggregate signal, and is the before/after instrument for evaluating A or B.

D. Optional product affordance — [research, low priority]

An opt-in output mode that structurally separates prose from tool calls, targeting the 91% Class A correlate. But Class A is a correlate, not a proven cause — if the root cause is decoder-side and content-order-independent, D won't help. Do not ship without the §4 validation + §8C telemetry to confirm it reduces the rate among opt-in users.

Recommended order (de-conflicted):

1. Ship B3 (fabricated-results guard) as a safety floor — the escalation is the scariest part and is independent of the mechanism.
2. Add C (telemetry) — cheap, and required to measure everything else.
3. Confirm B1 feasibility (which layer can repair?) and ship if a repair point exists; ship B2 as a precautionary low-risk reducer.
4. Run the §7 mechanism isolation, then pursue A as the durable root fix.
5. Reconsider D only after A/B1 validation.

B1+B2+B3 bound the blast radius; only A eradicates the defect.

---

9. Appendix — methodology & data availability

• Collector: a local read-only script scans ~/.claude/projects/**/*.jsonl, counts tool-use blocks and harness reject markers, and classifies the message preceding each reject. Two-layer detection (harness reject marker = ground truth; leaked-signature regex = secondary, forensic-FP-inflated per §3.4). Live-append; each run is a snapshot of a growing corpus.
• Reproducibility: every snapshot emits a fingerprint (file count, total bytes, max mtime) so drift is visible.
• Available immediately on request (not gated): the collector source, the escaped metrics JSON (current + the prior 2026-06-22 run), and raw escaped sample events. Only the controlled A/B experiment is time-gated — it needs N≈12k attempts/arm accrued across many sessions (§4), and only pays off if a harness/inference intervention is staged for a before/after.

---

Prepared from Claude Code transcript telemetry. Numbers are observational and single-environment (Windows, Japanese-dense); main-loop rate is the load-bearing figure. Bias disclosure: the reporter is also the operator of the measurement and verification subagents, which is itself a source of the agent-side contamination described in §3.1/§7 — read the agent-side numbers with that in mind. Contact: ikeda_n@tokium.jp.