99 lines
4.8 KiB
Markdown
99 lines
4.8 KiB
Markdown
# Bolt Assistant
|
|
|
|
A voice assistant for Bolt. While the mic is open, just ask something — there's no wake word — and
|
|
Bolt answers through your speakers. If you'd rather activate it by voice, set
|
|
`REQUIRE_WAKE_WORD=true` and say **"Hey Bolt"** (or "Bolt", "Thunderbolt", "Hey Assistant" and a few
|
|
others) first. It uses the same server, memory and persona as Bolt-Pet, Bolt Console and the phone
|
|
app.
|
|
|
|
```
|
|
Listening - anything you say is a request. Ctrl+C to quit.
|
|
> what time is it
|
|
< It's quarter past three.
|
|
> what's in my downloads folder
|
|
file list: C:\Users\aaron\Downloads
|
|
< You've got three installers and a PDF called taxes.
|
|
```
|
|
|
|
## Setup
|
|
|
|
```bash
|
|
cp .env.example .env # BOLT_SERVER_URL, DESK_API_KEY, ELEVENLABS_VOICE_ID
|
|
run.bat # Windows (./run.sh on Linux/macOS); creates .venv on first run
|
|
```
|
|
|
|
`run.bat --check` tests the settings, server, microphone, speaker and speech-to-text.
|
|
`run.bat --devices` lists audio devices. Set `MIC_DEVICE` / `SPEAKER_DEVICE` to use something
|
|
other than the system default. `run.bat --type` lets you type instead of speak, and
|
|
`run.bat --say "hello"` speaks one line.
|
|
|
|
## How it works
|
|
|
|
1. **Listening happens on your machine first.** A voice-activity detector watches the mic's loudness.
|
|
Silence, hum and clicks are never sent anywhere. When it hears something that sounds like
|
|
talking, that stretch of audio is sent to your server's `/desk/stt` relay (Deepgram, as in
|
|
Bolt-Pet) and transcribed as you speak.
|
|
2. **Every transcript is a request.** With `REQUIRE_WAKE_WORD` off (the default) the mic being open
|
|
is the activation: whatever the detector hears is sent to Bolt. A wake phrase said anyway ("Hey
|
|
Bolt, what time is it") is stripped, and the rest is the request. With `REQUIRE_WAKE_WORD=true`,
|
|
only transcripts that start with a wake phrase count: "Hey Bolt, what time is it" asks "what time
|
|
is it", and "I need to tighten the bolt" is ignored because the phrase has to open what you say,
|
|
with at most a filler word such as "um" or "okay" before it. Saying just "Hey Bolt" plays a chime,
|
|
and whatever you say in the next 8 seconds is the request.
|
|
3. **The answer is spoken** through the server's `/desk/tts`, one sentence at a time as Bolt
|
|
produces them. If the answer ends in a question, the assistant keeps listening for your reply
|
|
without needing the wake phrase again.
|
|
4. **While Bolt speaks, the mic is ignored**, so his own voice through the speakers (which often
|
|
says "Bolt") can't wake him.
|
|
|
|
## Cost and privacy
|
|
|
|
The server's speech-to-text is what turns what you say into a request, so **everything said near the
|
|
mic that sounds like speech is sent to your server and transcribed** — and, with the wake word off,
|
|
answered. The server counts this against your credits like a chat turn. The local detector keeps
|
|
that to actual speech: silence is never sent, and bursts shorter than `MIN_UTTERANCE_SEC` are
|
|
dropped. In a room with a TV or a lot of conversation, it will still transcribe and answer a lot.
|
|
|
|
If that becomes a problem, set `REQUIRE_WAKE_WORD=true` so only speech that starts with the wake
|
|
phrase is sent on. The alternative is on-device wake-word detection, which is what Bolt-Pet does with
|
|
openWakeWord and its `thunderbolt.onnx` model. Nothing leaves the machine until the wake word is
|
|
heard, but it only recognises the phrases it was trained on.
|
|
|
|
## What Bolt may do on this machine
|
|
|
|
There's no screen to approve things on, so `ASSISTANT_COMMANDS` decides:
|
|
|
|
| Setting | What Bolt can do |
|
|
|---|---|
|
|
| `off` | Nothing on this computer |
|
|
| `read` (default) | List folders and read files ("what's in my Downloads?") |
|
|
| `all` | Everything, including shell commands and file changes, like Bolt-Pet. Anyone within earshot can ask for anything. |
|
|
|
|
Anything that isn't allowed is refused, and Bolt is told to say he can't do it here.
|
|
|
|
## Settings
|
|
|
|
See `.env.example`. The ones you're most likely to change:
|
|
|
|
- `REQUIRE_WAKE_WORD`: `false` (default) answers anything said near the open mic; `true` requires a
|
|
wake phrase before each request.
|
|
- `VAD_RMS_THRESHOLD`: raise it if background noise keeps getting transcribed, lower it if you
|
|
have to raise your voice.
|
|
- `WAKE_WORDS`: the phrases, comma-separated (needed to activate Bolt when `REQUIRE_WAKE_WORD=true`;
|
|
said anyway, they're still stripped from the request).
|
|
- `ANYWHERE_WAKE_WORDS`: words that count anywhere in a sentence, not just at the start
|
|
(default `assistant`: "what's the weather, assistant?"). Blank to turn off.
|
|
- `COMMAND_WAIT_SEC`, `FOLLOW_UP_SEC`: how long it waits after "Hey Bolt", or after asking you
|
|
something.
|
|
|
|
It has its own conversation on the server (`pet-assistant-<computer>`), which shows up in Bolt
|
|
Console's Chats list.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
.venv\Scripts\python -m pytest tests # Windows; .venv/bin/python on Linux
|
|
```
|
|
|
|
No mic, speaker or server needed. The tests use synthetic audio and fakes for the network.
|