Long prompts are slow to type on a phone. Speaking is faster, and a coding agent handles spoken-style instructions well. The catch is where the recording goes. A prompt can describe your code, your files and your plans, so it is worth asking who hears it.

The usual options

  • The keyboard’s own mic button. Free and works everywhere. Your phone maker handles the audio.
  • A speech service in the cloud. Accurate and quick, but your recording goes to a third party.
  • A speech model on your own computer. Nothing leaves your devices, at the cost of a one-time setup.

What Brocoded does

Tap the mic in the message box and speak. The phone records a short clip, encrypts it to your computer using the same end-to-end channel as everything else in a session, and sends it. Your computer turns it into text with a local speech model (Whisper) and sends the text back. You can edit it before you send it to the agent.

The relay in the middle only carries encrypted data, as it does for terminal output and approvals. No speech service is involved.

The one-time setup

The computer needs two small tools and a model file. Update Brocoded on the computer, install ffmpeg and whisper.cpp (on a Mac with Homebrew, brew install ffmpeg whisper-cpp), then run this in a terminal:

foreman voice setup

It checks what is installed, downloads the speech model and tells you what is missing. Until you do this, the mic button says so, and your phone keyboard’s own mic still works in the message box.

Trade-offs

  • It needs setup. A cloud service would not.
  • Speed depends on your computer. A short message is usually quick on a recent laptop. A long one takes longer.
  • Accuracy depends on the model size and on your accent and surroundings. Read the text before you send it.
  • It is new. We have not checked it on every phone yet.

Why not just use the cloud

Often you can, and for casual messages it is fine. The point of doing it on your own computer is that you do not have to decide. A prompt that mentions a client, a secret or an unreleased feature gets the same treatment as everything else Brocoded handles: it goes between your two devices and nowhere else.

We have not had the encryption design reviewed by an outside cryptographer yet, and we say so on the security page. Voice input uses the same channel as the rest of Brocoded and has the same status.