Remote WebSocket connections fail with non-deterministic data limit in sandbox

Remote WebSocket connections fail with non-deterministic data limit in sandbox

Environment

• Claude Code Version: 2.1.19
• OS: macOS 15.7.2 (Darwin 24.6.0)
• Platform: darwin/arm64

Summary

The Claude Code sandbox has a non-deterministic per-connection data limit for remote WebSocket (wss://) connections that causes failures when transferring large amounts of data over a single connection. The limit typically ranges from 25MB to 100MB, making it impossible to use tools like kubectl exec with rsync for transferring files larger than ~20MB.

Impact

This breaks critical development workflows including:

• kubectl exec with rsync: Fails when syncing files larger than ~20MB to Kubernetes pods
• kubectl exec with large stdin/stdout: Fails when piping data >20MB
• Any long-lived remote WebSocket connection: Unpredictably fails after 25-100MB of cumulative data transfer

Detailed Behavior

Observed Failure Patterns

1. Per-connection cumulative limit: The limit applies to the total data sent over a single WebSocket connection, not per-message
2. Non-deterministic: The exact failure point varies between runs
3. Bimodal distribution: Failures cluster around 25-30MB (60% of tests) or 80-100MB (40% of tests)
4. Does not affect local connections: Connections to localhost/127.0.0.1/LAN IPs work fine with 100MB+
5. Does not affect HTTP/1.1: Simple HTTP requests work fine with large payloads
6. Resets with new connections: Creating a new WebSocket connection resets the limit

Test Results

Test 1: kubectl exec stdin (inside sandbox)

dd if=/dev/zero bs=1048576 count=19 | kubectl exec -i pod -- cat > /dev/null  # ✓ Works
dd if=/dev/zero bs=1048576 count=20 | kubectl exec -i pod -- cat > /dev/null  # ✗ Fails

Error: error reading from error stream: read message: %!w(<nil>)

Test 2: rsync over kubectl exec (inside sandbox)

rsync -avz --rsync-path='rsync' -e 'kubectl exec -i' ./large-file.bin pod:/tmp/

Error: remote error: tls: bad record MAC
(Fails when transferring 210MB file)

Test 3: Remote WebSocket (PieSocket) - 10 runs with 5MB chunks over single connection:

• Failure points: 25MB, 30MB (×5), 80MB (×2), 85MB, 100MB
• Average: 52MB
• Range: 25-100MB

Test 4: Same test outside sandbox:

• All 10 runs succeeded at 100MB

Test 5: New connections vs reused connection (inside sandbox):

• Reused connection: Failed at 90MB
• 10 new connections (10MB each): All succeeded (100MB total)

Reproduction Steps

Minimal Reproduction

1. Create test script /tmp/test-ws-limit.py:

#!/usr/bin/env python3
import asyncio
import websockets

async def test():
    uri = "wss://demo.piesocket.com/v3/channel_123?api_key=VCXCEuvhGcBDP7XhiJJUDvR1e1D3eiVjgZ9VRiaV&notify_self"

    async with websockets.connect(uri, max_size=100*1024*1024) as ws:
        total = 0
        for i in range(20):  # Try to send 100MB total
            data = b'\x00' * (5 * 1024 * 1024)  # 5MB chunks
            await ws.send(data)
            total += 5
            print(f"Sent {total}MB")
            await asyncio.sleep(0.3)

asyncio.run(test())

1. Run inside Claude Code sandbox:

python3 /tmp/test-ws-limit.py

1. Run outside sandbox:

python3 /tmp/test-ws-limit.py

Expected: Both succeed at 100MB
Actual: Inside sandbox fails between 25-100MB with "no close frame received or sent"

Real-World Reproduction (kubectl + rsync)

Prerequisites:

• kubectl configured with access to a Kubernetes cluster
• Sandbox settings allowing the cluster domain (e.g., *.staging.dog)

Steps:

# Create a test file
dd if=/dev/zero of=/tmp/test-25mb.bin bs=1048576 count=25

# Try to sync it to a pod
rsync -avz /tmp/test-25mb.bin -e 'kubectl exec -i -n namespace' pod:/tmp/

# Observe failure with "tls: bad record MAC" error

Expected Behavior

Remote WebSocket connections should support arbitrarily large data transfers (or at least multi-GB transfers like local connections do), with consistent behavior across runs.

Actual Behavior

Remote WebSocket connections fail non-deterministically after 25-100MB of cumulative data transfer over a single connection.

Workarounds

1. For kubectl sync: Add ./dev/hotdog-sync-once:* to excludedCommands in .claude/settings.json to run outside sandbox
2. For other WebSocket usage: Create new connections periodically to reset the limit
3. For file transfers: Use HTTP/1.1 instead of WebSocket when possible

Additional Context

What is NOT the issue:

• ✅ TLS itself (local wss:// works fine with 100MB+)
• ✅ WebSocket protocol (works perfectly with local connections)
• ✅ Bidirectional streaming (tested and works locally)
• ✅ Network domain permissions (domains are properly allowed in settings)

What IS the issue:

• ❌ Combination of: Remote + WebSocket/HTTP2 + Large cumulative data (>25MB) over single connection

Sandbox Configuration

Our .claude/settings.json:

{
  "sandbox": {
    "enabled": true,
    "network": {
      "allowAllUnixSockets": true,
      "allowLocalBinding": true
    }
  },
  "permissions": {
    "allow": [
      "WebFetch(domain:*.staging.dog)"
    ]
  }
}

Related

This appears to be a limitation in the sandbox's network proxy layer, possibly related to buffer management or connection pooling for multiplexed protocols.