Bolt Assistant

A voice assistant for Bolt. Say "Hey Bolt" (or "Bolt", "Thunderbolt", "Hey Assistant" and a few others), ask something, and Bolt answers through your speakers. It uses the same server, memory and persona as Bolt-Pet, Bolt Console and the phone app.

Listening for "Hey Bolt" (or "Thunderbolt", "Hey Assistant"). Ctrl+C to quit.
> what time is it
< It's quarter past three.
* Yes?
> what's in my downloads folder
  file list: C:\Users\aaron\Downloads
< You've got three installers and a PDF called taxes.

Setup

cp .env.example .env     # BOLT_SERVER_URL, DESK_API_KEY, ELEVENLABS_VOICE_ID
run.bat                  # Windows (./run.sh on Linux/macOS); creates .venv on first run

run.bat --check tests the settings, server, microphone, speaker and speech-to-text. run.bat --devices lists audio devices. Set MIC_DEVICE / SPEAKER_DEVICE to use something other than the system default. run.bat --type lets you type instead of speak, and run.bat --say "hello" speaks one line.

How it works

Listening happens on your machine first. A voice-activity detector watches the mic's loudness. Silence, hum and clicks are never sent anywhere. When it hears something that sounds like talking, that stretch of audio is sent to your server's /desk/stt relay (Deepgram, as in Bolt-Pet) and transcribed as you speak.
Only transcripts that start with a wake phrase count. "Hey Bolt, what time is it" asks "what time is it". "I need to tighten the bolt" is ignored, because the phrase has to open what you say, with at most a filler word such as "um" or "okay" before it. Saying just "Hey Bolt" plays a chime, and whatever you say in the next 8 seconds is the request.
The answer is spoken through the server's /desk/tts, one sentence at a time as Bolt produces them. If the answer ends in a question, the assistant keeps listening for your reply without needing the wake phrase again.
While Bolt speaks, the mic is ignored, so his own voice through the speakers (which often says "Bolt") can't wake him.

Cost and privacy

The server's speech-to-text is what recognises the wake phrase, so everything said near the mic that sounds like speech is sent to your server and transcribed, not just things said to Bolt. The server counts this against your credits like a chat turn. The local detector keeps that to actual speech: silence is never sent, and bursts shorter than MIN_UTTERANCE_SEC are dropped. In a room with a TV or a lot of conversation, it will still transcribe a lot.

If that becomes a problem, the alternative is on-device wake-word detection, which is what Bolt-Pet does with openWakeWord and its thunderbolt.onnx model. Nothing leaves the machine until the wake word is heard, but it only recognises the phrases it was trained on.

What Bolt may do on this machine

There's no screen to approve things on, so ASSISTANT_COMMANDS decides:

Setting	What Bolt can do
`off`	Nothing on this computer
`read` (default)	List folders and read files ("what's in my Downloads?")
`all`	Everything, including shell commands and file changes, like Bolt-Pet. Anyone within earshot can ask for anything.

Anything that isn't allowed is refused, and Bolt is told to say he can't do it here.

Settings

See .env.example. The ones you're most likely to change:

VAD_RMS_THRESHOLD: raise it if background noise keeps getting transcribed, lower it if you have to raise your voice.
WAKE_WORDS: the phrases, comma-separated.
ANYWHERE_WAKE_WORDS: words that count anywhere in a sentence, not just at the start (default assistant: "what's the weather, assistant?"). Blank to turn off.
COMMAND_WAIT_SEC, FOLLOW_UP_SEC: how long it waits after "Hey Bolt", or after asking you something.

It has its own conversation on the server (pet-assistant-<computer>), which shows up in Bolt Console's Chats list.

Tests

.venv\Scripts\python -m pytest tests        # Windows; .venv/bin/python on Linux

No mic, speaker or server needed. The tests use synthetic audio and fakes for the network.

4.1 KiB Raw Blame History