Bolt-Assistant/README.md

# Bolt Assistant

A voice assistant for Bolt. While the mic is open, just ask something — there's no wake word — and
Bolt answers through your speakers. If you'd rather activate it by voice, set
`REQUIRE_WAKE_WORD=true` and say **"Hey Bolt"** (or "Bolt", "Thunderbolt", "Hey Assistant" and a few
others) first. It uses the same server, memory and persona as Bolt-Pet, Bolt Console and the phone
app.

```
Listening - anything you say is a request. Ctrl+C to quit.
> what time is it
< It's quarter past three.
> what's in my downloads folder
  file list: C:\Users\aaron\Downloads
< You've got three installers and a PDF called taxes.
```

## Setup

```bash
cp .env.example .env     # BOLT_SERVER_URL, DESK_API_KEY, ELEVENLABS_VOICE_ID
run.bat                  # Windows (./run.sh on Linux/macOS); creates .venv on first run
```

`run.bat --check` tests the settings, server, microphone, speaker and speech-to-text.
`run.bat --devices` lists audio devices. Set `MIC_DEVICE` / `SPEAKER_DEVICE` to use something
other than the system default. `run.bat --type` lets you type instead of speak, and
`run.bat --say "hello"` speaks one line.

## How it works

1. **Listening happens on your machine first.** A voice-activity detector watches the mic's loudness.
   Silence, hum and clicks are never sent anywhere. When it hears something that sounds like
   talking, that stretch of audio is sent to your server's `/desk/stt` relay (Deepgram, as in
   Bolt-Pet) and transcribed as you speak.
2. **Every transcript is a request.** With `REQUIRE_WAKE_WORD` off (the default) the mic being open
   is the activation: whatever the detector hears is sent to Bolt. A wake phrase said anyway ("Hey
   Bolt, what time is it") is stripped, and the rest is the request. With `REQUIRE_WAKE_WORD=true`,
   only transcripts that start with a wake phrase count: "Hey Bolt, what time is it" asks "what time
   is it", and "I need to tighten the bolt" is ignored because the phrase has to open what you say,
   with at most a filler word such as "um" or "okay" before it. Saying just "Hey Bolt" plays a chime,
   and whatever you say in the next 8 seconds is the request.
3. **The answer is spoken** through the server's `/desk/tts`, one sentence at a time as Bolt
   produces them. If the answer ends in a question, the assistant keeps listening for your reply
   without needing the wake phrase again.
4. **While Bolt speaks, the mic is ignored**, so his own voice through the speakers (which often
   says "Bolt") can't wake him.
5. **Reminders and delivered files ride a heartbeat.** Every `HEARTBEAT_SECONDS` (default 60) the
   assistant reports in to `/desk/report_status`, which is how Bolt's reminders and any files he
   sends to this client arrive — the same channel Bolt-Pet uses. Announcements are spoken when the
   mic is idle, and files download to `BOLT_DOWNLOAD_DIR` (default `~/Downloads/Bolt`), never
   overwriting what's already there. Set `HEARTBEAT_SECONDS=0` to turn it off.

## Cost and privacy

The server's speech-to-text is what turns what you say into a request, so **everything said near the
mic that sounds like speech is sent to your server and transcribed** — and, with the wake word off,
answered. The server counts this against your credits like a chat turn. The local detector keeps
that to actual speech: silence is never sent, and bursts shorter than `MIN_UTTERANCE_SEC` are
dropped. In a room with a TV or a lot of conversation, it will still transcribe and answer a lot.

If that becomes a problem, set `REQUIRE_WAKE_WORD=true` so only speech that starts with the wake
phrase is sent on. The alternative is on-device wake-word detection, which is what Bolt-Pet does with
openWakeWord and its `thunderbolt.onnx` model. Nothing leaves the machine until the wake word is
heard, but it only recognises the phrases it was trained on.

## What Bolt may do on this machine

There's no screen to approve things on, so `ASSISTANT_COMMANDS` decides:

| Setting | What Bolt can do |
|---|---|
| `off` | Nothing on this computer |
| `read` (default) | List folders and read files ("what's in my Downloads?") |
| `all` | Everything, including shell commands and file changes, like Bolt-Pet. Anyone within earshot can ask for anything. |

Anything that isn't allowed is refused, and Bolt is told to say he can't do it here.

## Settings

See `.env.example`. The ones you're most likely to change:

- `REQUIRE_WAKE_WORD`: `false` (default) answers anything said near the open mic; `true` requires a
  wake phrase before each request.
- `VAD_RMS_THRESHOLD`: raise it if background noise keeps getting transcribed, lower it if you
  have to raise your voice.
- `WAKE_WORDS`: the phrases, comma-separated (needed to activate Bolt when `REQUIRE_WAKE_WORD=true`;
  said anyway, they're still stripped from the request).
- `ANYWHERE_WAKE_WORDS`: words that count anywhere in a sentence, not just at the start
  (default `assistant`: "what's the weather, assistant?"). Blank to turn off.
- `COMMAND_WAIT_SEC`, `FOLLOW_UP_SEC`: how long it waits after "Hey Bolt", or after asking you
  something.
- `HEARTBEAT_SECONDS`: how often it checks in for reminders and delivered files (default 60;
  `0` turns the heartbeat off).
- `BOLT_DOWNLOAD_DIR`: where delivered files land (default `~/Downloads/Bolt`).

It has its own conversation on the server (`pet-assistant-<computer>`), which shows up in Bolt
Console's Chats list.

## Tests

```bash
.venv\Scripts\python -m pytest tests        # Windows; .venv/bin/python on Linux
```

No mic, speaker or server needed. The tests use synthetic audio and fakes for the network.