[FEATURE] Voice input for Remote Control (mobile is unusable without dictation)

Preflight Checklist

• [x] I have searched existing requests and this feature hasn't been requested yet
• [x] This is a single feature request (not multiple features)

Problem Statement

First off - Remote Control is incredible. Being able to pick up my phone and continue a local Claude Code session from anywhere is exactly the kind of thing that makes this tool special. Huge props to the team for shipping it.

That said, there's one thing that would take it from great to genuinely transformative: voice input. Right now, using Remote Control on a phone means typing everything, and typing detailed technical prompts on a phone keyboard is really slow. For my workflow, this is the one missing piece.

For context, I manage a fleet of embedded test devices across multiple workstations in a manufacturing/lab environment. I use Claude Code to SSH into machines, run diagnostics, check service health, tail logs, and investigate issues throughout the day. Remote Control lets me do all of this from the floor instead of walking back to my desk every time - which is amazing. But the input side is where the friction lives. A 30-second voice message could replace 5 minutes of thumb-typing a technical prompt.

The dream UX: Tap a mic button, put my phone in my pocket, walk around the floor, talk for a few minutes doing a brain dump of everything I need the agent to handle, tap stop. Hands-free, no time limits, no stress about whether the recording will make it through.

A Pattern That Works Well

I've been running a voice-to-agent pipeline in production that solves this, and I wanted to share the architecture in case it's helpful. It's pretty straightforward:

1. User records voice (any length - 30 seconds to 10 minutes, no artificial limits)
2. Audio gets auto-chunked - ffmpeg splits into segments (10-min chunks, respecting API size limits) with silence detection for clean split points
3. Each chunk hits a Whisper-based STT endpoint for transcription
4. Chunks get concatenated into a single coherent transcript
5. Transcript feeds into the agent as a normal text message

In my setup, I used Groq's hosted Whisper (whisper-large-v3-turbo) because their LPU hardware makes inference really fast - a 5-minute audio file comes back in under 2 seconds. Obviously I'm not suggesting Anthropic wire up Groq's endpoints - that wouldn't make sense. They're essentially hosting OpenAI's open-source Whisper model on their own inference hardware. But I mention it because the speed sets a nice bar for what the experience should feel like.

Anthropic has way more options here - train your own STT model, run an open-source Whisper variant on your infrastructure, use whatever internal transcription capabilities you already have. The specific model doesn't matter nearly as much as the architecture around it.

The chunking is the real hero here. It's not just about API size limits - it's a resilience thing. If a 5-minute recording tries to upload as one blob and the connection hiccups, you lose the whole thing. With chunking, each segment uploads and transcribes independently. If one chunk fails, retry just that chunk. The user never loses a long recording. That reliability is what makes people trust voice input enough to actually use it for real work.

One thing I'd flag: please don't use the browser's native Speech-to-Text API for this. Native STT (Web Speech API, iOS dictation) times out after ~30 seconds, struggles with technical terms, requires the screen to stay active, and quality varies a lot across devices. A real transcription model server-side is the way to go.

Per-User Voice Dictionary (Bonus Feature That Would Be Huge)

One thing that would really set this apart: a user-configurable voice dictionary for transcription correction.

Developers and engineers talk about things that no general-purpose STT model has seen before - custom hostnames, internal tool names, project codenames, CLI flags, file paths. When I say "deploy to kv-node-7" the transcription might come back as "deploy to cave node seven" or "KV note 7". When I reference a config path like /etc/myapp/staging.conf, the STT might interpret the slashes and dots as pauses or filler words.

What would help:

1. Custom word list per user - let users add domain-specific terms, hostnames, tool names, etc. The transcription engine uses these as hints to bias toward correct spellings. For example, if I add "kubectl", "etcd", "nginx", and "pg_dump" to my dictionary, the model knows to prefer those over "cube cuddle", "et CD", "engine X", and "PG dump".

1. Smart flagging for ambiguous terms - after transcription, highlight words that look like they might be file paths, hostnames, or technical terms (contains /, ., -, unusual capitalization patterns). Let the user tap a flagged word and pick from suggestions or manually correct it. Over time, these corrections feed back into the user's dictionary automatically.

1. Path/symbol awareness - when the STT detects patterns that sound like file paths or special characters (user says "slash etc slash config dot yaml"), offer to convert them to the actual symbols (/etc/config.yaml). A small toggle like "interpret as path" on flagged segments would go a long way.

This doesn't need to be perfect out of the gate - even a simple "here are my custom words" text file that biases the transcription would be a massive improvement over generic STT for technical workflows. Whisper's initial_prompt parameter already supports this kind of vocabulary hinting, so the infrastructure is there.

Proposed Solution

A tap-to-record mic button in the Remote Control UI:

1. Tap once to start, tap again to stop - hands-free recording. Phone can be in your pocket, screen can dim.
2. Records audio locally on the device (MediaRecorder API - works great across mobile browsers)
3. Progressive chunking during recording - stream chunks to the server as they're captured (e.g., every 60-90 seconds). A 10-minute recording starts transcribing at minute 1. If connection drops at minute 8, minutes 1-7 are already safe.
4. Server-side transcription via whatever model/infrastructure Anthropic prefers. Not browser native STT.
5. Per-chunk retry - if any chunk fails, retry it independently. Never lose the full recording over one network hiccup.
6. Transcript review before send - show the assembled transcript for review/edit before submitting. Flagged/ambiguous technical terms are highlighted for quick tap-to-correct.
7. User voice dictionary - per-user custom word list that biases transcription toward domain-specific terms. Corrections during review automatically update the dictionary.

Here's the core schema that's worked well for me:

record(audio_blob)
  -> chunk(blob, max_duration=90s, max_size=25MB)
  -> for each chunk: transcribe(chunk, user_dictionary) with retry(max=3)
  -> concatenate(transcripts)
  -> flag_ambiguous_terms(transcript, user_dictionary)
  -> populate_input(full_transcript)  // with highlighted terms for review

Happy to Contribute

I've already shipped this exact pattern in production and I know the failure modes pretty well. If Remote Control's architecture supports extensibility or plugin hooks, I'd genuinely love to help build this out and submit a PR. And if it needs to be built natively by the team, totally understand - just wanted to share what's worked and offer to pitch in however I can.

Why This Is Distinct From #14444

Issue #14444 requests native audio modality support in Claude Code broadly (terminal + editor). This is specifically about Remote Control on mobile devices, where:

1. No physical keyboard - voice input goes from nice-to-have to essential
2. The use case is inherently mobile/remote - voice is the natural input modality
3. This is a UI-level feature (speech to text in the input field) - no model-level audio changes needed
4. MediaRecorder API for audio capture is mature across all mobile browsers
5. Could ship independently of any broader audio modality support

Alternative Solutions

• Device-native dictation: iOS/Android keyboard dictation is designed for short messages. Technical terms get mangled, it times out after ~30 seconds, requires screen to stay active. Not practical for engineering workflows with long-form input.
• External transcription pipelines: What I'm currently running (OpenAI Whisper API as the backend). Works, but requires separate infrastructure and kind of defeats the purpose of a unified Remote Control experience.
• Local Whisper on device: Not practical on mobile.

Priority

High - for workflows that involve being away from a desk (lab, factory floor, server room), this is the difference between Remote Control being a convenience and being a daily driver.

Feature Category

UI improvements

Use Case Example

I'm walking around a lab. A test node needs debugging. I pull out my phone, open Remote Control, tap the mic button, put my phone back in my pocket, and talk while I walk:

"Hey, so node-33 on workstation-A is throwing intermittent API push failures. Can you SSH in and check if the core services are all running - database, DHCP, device manager. Then hop into node-33 and tail the logs for the last 15 minutes, filter for any upstream push errors. If they're 4xx it's probably config, if they're 5xx or timeouts it's network. Also check the DHCP lease list on the workstation to make sure all the test nodes still have active leases. And grab the last successful transaction timestamp from node-33 so we know when it stopped working."

That's about 45 seconds of talking vs 5+ minutes of typing on a phone. And with my voice dictionary loaded (node-33, workstation-A, and my custom tool names are all pre-registered), the transcript comes back clean on the first pass instead of needing manual correction.

Additional Context

• Related: #14444 (native audio modality), #1852 (VS Code Speech), #25954 (full voice interaction)
• MediaRecorder API works across all major mobile browsers including in background
• Claude mobile app already has mic permissions for the main chat - Remote Control sessions could inherit this
• The chunking + progressive upload pattern is battle-tested in production for 10+ minute recordings
• Whisper's initial_prompt parameter already supports vocabulary hinting, making user dictionaries straightforward to implement
• Happy to share more implementation details or contribute code if that would be helpful