89 lines
3.9 KiB
Markdown
89 lines
3.9 KiB
Markdown
# Bolt Assistant
|
|
|
|
A voice assistant for Bolt. Say **"Hey Bolt"** (or "Bolt", "Thunderbolt", "Hey Assistant" and a few
|
|
others), ask something, and Bolt answers through your speakers. It uses the same server, memory and
|
|
persona as Bolt-Pet, Bolt Console and the phone app.
|
|
|
|
```
|
|
Listening for "Hey Bolt" (or "Thunderbolt", "Hey Assistant"). Ctrl+C to quit.
|
|
> what time is it
|
|
< It's quarter past three.
|
|
* Yes?
|
|
> what's in my downloads folder
|
|
file list: C:\Users\aaron\Downloads
|
|
< You've got three installers and a PDF called taxes.
|
|
```
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
cp .env.example .env # BOLT_SERVER_URL, DESK_API_KEY, ELEVENLABS_VOICE_ID
|
|
run.bat # Windows (./run.sh on Linux/macOS); creates .venv on first run
|
|
```
|
|
|
|
`run.bat --check` tests the settings, server, microphone, speaker and speech-to-text.
|
|
`run.bat --devices` lists audio devices. Set `MIC_DEVICE` / `SPEAKER_DEVICE` to use something
|
|
other than the system default. `run.bat --type` lets you type instead of speak, and
|
|
`run.bat --say "hello"` speaks one line.
|
|
|
|
## How it works
|
|
|
|
1. **Listening happens on your machine first.** A voice-activity detector watches the mic's loudness.
|
|
Silence, hum and clicks are never sent anywhere. When it hears something that sounds like
|
|
talking, that stretch of audio is sent to your server's `/desk/stt` relay (Deepgram, as in
|
|
Bolt-Pet) and transcribed as you speak.
|
|
2. **Only transcripts that start with a wake phrase count.** "Hey Bolt, what time is it" asks "what
|
|
time is it". "I need to tighten the bolt" is ignored, because the phrase has to open what you say,
|
|
with at most a filler word such as "um" or "okay" before it. Saying just "Hey Bolt" plays a chime,
|
|
and whatever you say in the next 8 seconds is the request.
|
|
3. **The answer is spoken** through the server's `/desk/tts`, one sentence at a time as Bolt
|
|
produces them. If the answer ends in a question, the assistant keeps listening for your reply
|
|
without needing the wake phrase again.
|
|
4. **While Bolt speaks, the mic is ignored**, so his own voice through the speakers (which often
|
|
says "Bolt") can't wake him.
|
|
|
|
## Cost and privacy
|
|
|
|
The server's speech-to-text is what recognises the wake phrase, so **everything said near the mic
|
|
that sounds like speech is sent to your server and transcribed**, not just things said to Bolt. The
|
|
server counts this against your credits like a chat turn. The local detector keeps that to actual
|
|
speech: silence is never sent, and bursts shorter than `MIN_UTTERANCE_SEC` are dropped. In a room
|
|
with a TV or a lot of conversation, it will still transcribe a lot.
|
|
|
|
If that becomes a problem, the alternative is on-device wake-word detection, which is what Bolt-Pet
|
|
does with openWakeWord and its `thunderbolt.onnx` model. Nothing leaves the machine until the wake
|
|
word is heard, but it only recognises the phrases it was trained on.
|
|
|
|
## What Bolt may do on this machine
|
|
|
|
There's no screen to approve things on, so `ASSISTANT_COMMANDS` decides:
|
|
|
|
| Setting | What Bolt can do |
|
|
|---|---|
|
|
| `off` | Nothing on this computer |
|
|
| `read` (default) | List folders and read files ("what's in my Downloads?") |
|
|
| `all` | Everything, including shell commands and file changes, like Bolt-Pet. Anyone within earshot can ask for anything. |
|
|
|
|
Anything that isn't allowed is refused, and Bolt is told to say he can't do it here.
|
|
|
|
## Settings
|
|
|
|
See `.env.example`. The ones you're most likely to change:
|
|
|
|
- `VAD_RMS_THRESHOLD`: raise it if background noise keeps getting transcribed, lower it if you
|
|
have to raise your voice.
|
|
- `WAKE_WORDS`: the phrases, comma-separated.
|
|
- `COMMAND_WAIT_SEC`, `FOLLOW_UP_SEC`: how long it waits after "Hey Bolt", or after asking you
|
|
something.
|
|
|
|
It has its own conversation on the server (`pet-assistant-<computer>`), which shows up in Bolt
|
|
Console's Chats list.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
.venv\Scripts\python -m pytest tests # Windows; .venv/bin/python on Linux
|
|
```
|
|
|
|
No mic, speaker or server needed. The tests use synthetic audio and fakes for the network.
|