[BUG] Streaming input_json_delta accumulation truncates large MCP tool arguments (tail cut) — CLI 2.1.179

[BUG] Streaming input_json_delta accumulation truncates large MCP tool arguments (tail cut) — CLI 2.1.179

Likely the same root cause as #67765 and the same regression window as #66247 — filing separately because it adds an MCP-server-independent reproduction that those threads lack. Happy to have this merged/closed as a duplicate if maintainers prefer. The new contribution: a minimal control that empirically exonerates the server, transport, and SDK, isolating the fault to the Claude Code client's streamed-tool-argument accumulation, plus the misleading downstream error it produces.

Summary

On Claude Code 2.1.179, a large MCP tool argument (a JSON-string documents_json) intermittently reaches the MCP server truncated near the end, causing a server-side json.loads failure: Invalid documents_json: Expecting ',' delimiter "pointing near the end of the payload." The value immediately before the cut is complete (it is not an Unterminated string), i.e. the tail of the argument is missing. This matches the VH1-tokenizer string-token dropout described in #67765 and the post-2.1.165 tool_use parse-failure regression in #66247.

Environment

• Claude Code: 2.1.179 (within the #66247 regression window; #67765 reports v2.1.173+)
• Model: claude-opus-4-8
• OS: Windows 11
• Transport: MCP streamable-http (also reproduced historically; #67765 notes the bug is transport-agnostic)
• MCP server: a local FastMCP 3.1.0 / mcp 1.26.0 server (Embranch), but the server is provably not involved — see Control below.

Symptom (observed)

Calling an MCP write tool with a single large JSON-string argument (~1–4 KB+) intermittently fails with:

Invalid documents_json: Expecting ',' delimiter   (parse error near the end of the payload)

• Intermittent: the same logical write fails several times, then a re-attempt (or splitting it) succeeds.
• Size-correlated: larger string arguments fail disproportionately (consistent with #67765's ">4 KB" note).
• The error is a tail truncation, not a mid-string cut (Expecting ',' delimiter, not Unterminated string) — the argument is cut at/after a completed value, missing the closing structure.

What rules out the server, transport, and SDK (the new evidence)

A two-part, MCP-server-independent test (attached scripts mcp_payload_probe.py + probe_control.py):

1. Server + a spec-correct HTTP client are faithful. A minimal FastMCP server exposing one tool that echoes the received byte count, driven by FastMCP's own httpx client over a real uvicorn socket, round-trips payloads from 256 B to 64 KB intact (received_chars == sent, Content-Length matches), including the ~1 KB band.
2. Claude Code's own client also delivered some large requests intact — e.g. a 2053-byte request arrived whole — consistent with the failure being probabilistic (delta-boundary shear) rather than a hard size cap.

Direct source review of the server path (proxy relay + ASGI middleware + the vendored mcp streamable-http handler) confirms the request body is read in full and forwarded/parsed verbatim — there is no re-serialization or slicing of tool arguments. Therefore the truncation is introduced before the bytes leave the client, in the streamed-input_json_delta accumulation — exactly the VH1 string-token dropout in #67765.

Minimal reproduction

Deterministic (per #67765): configure any MCP server with a tool taking a string argument; have Claude begin a tool call with a >4 KB string value, press Escape mid-stream, then call the same tool again — arguments arrive empty/corrupt and persist for the session (per-tool cache).

Server-independent control (attached):

pip install fastmcp
python mcp_payload_probe.py --transport http --port 8765      # minimal echo server
python probe_control.py http://127.0.0.1:8765/mcp/            # baseline: every size OK
claude mcp add --transport http probe http://127.0.0.1:8765/mcp/
# then, in Claude Code, repeatedly: "call probe with a payload of N 'A' characters"
# increasing N past ~4 KB; compare received_chars to N. A shortfall on a size the
# control reported OK = client-side truncation reproduced.

API-level (no MCP): api_tooluse_truncation_repro.py streams one tool call with a large argument and flags an invalid accumulated input_json_delta at a non-max_tokens stop.

Impact

Silent corruption of MCP write tool arguments. In our case it corrupts knowledge/diary offload payloads; combined with the per-tool argument cache (#67765, #3296) a single shear poisons every subsequent call to that tool until session restart. The server-side error message (Expecting ',' delimiter) is misleading — it looks like malformed input or a size limit, sending users to debug the wrong layer.

Relation to existing issues

• #67765 — same root cause (streaming partial-JSON accumulator / VH1 token dropout). This report adds the server-independent control + the misleading downstream error.
• #66247 — same regression window (≥ 2.1.165); we observe it on 2.1.179.
• #19143 — related but distinct (API-layer premature-stop); closed. The active cause here is client-side accumulation, per the version correlation.

Suggested fix (echoing #67765)

• Emit provisional tokens for incomplete strings at a delta boundary instead of dropping them; continue accumulating across input_json_delta chunks until the tool input is complete.
• Do not cache tool parameters from aborted/errored content blocks.
• Surface a typed "incomplete tool arguments" error/retry rather than forwarding {} or a truncated body.

Workaround

Pin claude install 2.1.162 && export DISABLE_AUTOUPDATER=1 (#66247), or split large writes into small, individually-complete tool calls.