Bolt Assistant: wake-phrase voice assistant for Bolt

2026-09-26 22:59:33 -06:00
commit 8bcd539b31
23 changed files with 2726 additions and 0 deletions
@@ -0,0 +1,88 @@
+# Bolt Assistant
+
+A voice assistant for Bolt. Say **"Hey Bolt"** (or "Bolt", "Thunderbolt", "Hey Assistant" and a few
+others), ask something, and Bolt answers through your speakers. It uses the same server, memory and
+persona as Bolt-Pet, Bolt Console and the phone app.
+
+```
+Listening for "Hey Bolt" (or "Thunderbolt", "Hey Assistant"). Ctrl+C to quit.
+> what time is it
+< It's quarter past three.
+* Yes?
+> what's in my downloads folder
+  file list: C:\Users\aaron\Downloads
+< You've got three installers and a PDF called taxes.
+```
+
+## Setup
+
+```bash
+cp .env.example .env     # BOLT_SERVER_URL, DESK_API_KEY, ELEVENLABS_VOICE_ID
+run.bat                  # Windows (./run.sh on Linux/macOS); creates .venv on first run
+```
+
+`run.bat --check` tests the settings, server, microphone, speaker and speech-to-text.
+`run.bat --devices` lists audio devices. Set `MIC_DEVICE` / `SPEAKER_DEVICE` to use something
+other than the system default. `run.bat --type` lets you type instead of speak, and
+`run.bat --say "hello"` speaks one line.
+
+## How it works
+
+1. **Listening happens on your machine first.** A voice-activity detector watches the mic's loudness.
+   Silence, hum and clicks are never sent anywhere. When it hears something that sounds like
+   talking, that stretch of audio is sent to your server's `/desk/stt` relay (Deepgram, as in
+   Bolt-Pet) and transcribed as you speak.
+2. **Only transcripts that start with a wake phrase count.** "Hey Bolt, what time is it" asks "what
+   time is it". "I need to tighten the bolt" is ignored, because the phrase has to open what you say,
+   with at most a filler word such as "um" or "okay" before it. Saying just "Hey Bolt" plays a chime,
+   and whatever you say in the next 8 seconds is the request.
+3. **The answer is spoken** through the server's `/desk/tts`, one sentence at a time as Bolt
+   produces them. If the answer ends in a question, the assistant keeps listening for your reply
+   without needing the wake phrase again.
+4. **While Bolt speaks, the mic is ignored**, so his own voice through the speakers (which often
+   says "Bolt") can't wake him.
+
+## Cost and privacy
+
+The server's speech-to-text is what recognises the wake phrase, so **everything said near the mic
+that sounds like speech is sent to your server and transcribed**, not just things said to Bolt. The
+server counts this against your credits like a chat turn. The local detector keeps that to actual
+speech: silence is never sent, and bursts shorter than `MIN_UTTERANCE_SEC` are dropped. In a room
+with a TV or a lot of conversation, it will still transcribe a lot.
+
+If that becomes a problem, the alternative is on-device wake-word detection, which is what Bolt-Pet
+does with openWakeWord and its `thunderbolt.onnx` model. Nothing leaves the machine until the wake
+word is heard, but it only recognises the phrases it was trained on.
+
+## What Bolt may do on this machine
+
+There's no screen to approve things on, so `ASSISTANT_COMMANDS` decides:
+
+| Setting | What Bolt can do |
+|---|---|
+| `off` | Nothing on this computer |
+| `read` (default) | List folders and read files ("what's in my Downloads?") |
+| `all` | Everything, including shell commands and file changes, like Bolt-Pet. Anyone within earshot can ask for anything. |
+
+Anything that isn't allowed is refused, and Bolt is told to say he can't do it here.
+
+## Settings
+
+See `.env.example`. The ones you're most likely to change:
+
+- `VAD_RMS_THRESHOLD`: raise it if background noise keeps getting transcribed, lower it if you
+  have to raise your voice.
+- `WAKE_WORDS`: the phrases, comma-separated.
+- `COMMAND_WAIT_SEC`, `FOLLOW_UP_SEC`: how long it waits after "Hey Bolt", or after asking you
+  something.
+
+It has its own conversation on the server (`pet-assistant-<computer>`), which shows up in Bolt
+Console's Chats list.
+
+## Tests
+
+```bash
+.venv\Scripts\python -m pytest tests        # Windows; .venv/bin/python on Linux
+```
+
+No mic, speaker or server needed. The tests use synthetic audio and fakes for the network.