Refactor wake phrase handling: introduce REQUIRE_WAKE_WORD configuration, update documentation, and adjust assistant behavior for improved user experience.

.env.example

@@ -1,30 +0,0 @@
-# Bolt Assistant. Copy to .env and fill in the first three.
-BOLT_SERVER_URL=https://your-bolt-server.example.com
-DESK_API_KEY=
-# The voice replies are spoken in (an ElevenLabs voice id the server can use)
-ELEVENLABS_VOICE_ID=
-
-# Microphone / speaker: blank = system default, or an index / name from --devices
-#MIC_DEVICE=
-#SPEAKER_DEVICE=
-
-# Wake phrases, comma-separated (matched on the transcript, at the start of what's said)
-#WAKE_WORDS=hey bolt,bolt,thunderbolt,hey thunderbolt,ok bolt,okay bolt,hey assistant
-# Words that count anywhere in what's said; the whole sentence becomes the request
-#ANYWHERE_WAKE_WORDS=assistant
-
-# What Bolt may do on this machine when he relays a command:
-#   off | read (default: list and read files) | all (shell commands and file changes too)
-#ASSISTANT_COMMANDS=read
-
-# Local speech detection: nothing is sent to the server until someone seems to be talking.
-# Raise the threshold if background noise keeps getting transcribed.
-#VAD_RMS_THRESHOLD=500
-#VAD_SILENCE_END_SEC=0.8
-#MIN_UTTERANCE_SEC=0.35
-#MAX_UTTERANCE_SEC=12
-#COMMAND_WAIT_SEC=8
-#FOLLOW_UP_SEC=8
-#CHIMES=true
-#STREAMING_REPLIES=true
-#DESK_SESSION_ID=

README.md

@@ -1,14 +1,15 @@
 # Bolt Assistant
 
-A voice assistant for Bolt. Say **"Hey Bolt"** (or "Bolt", "Thunderbolt", "Hey Assistant" and a few
+A voice assistant for Bolt. While the mic is open, just ask something — there's no wake word — and
-others), ask something, and Bolt answers through your speakers. It uses the same server, memory and
+Bolt answers through your speakers. If you'd rather activate it by voice, set
-persona as Bolt-Pet, Bolt Console and the phone app.
+`REQUIRE_WAKE_WORD=true` and say **"Hey Bolt"** (or "Bolt", "Thunderbolt", "Hey Assistant" and a few
+others) first. It uses the same server, memory and persona as Bolt-Pet, Bolt Console and the phone
+app.
 
 ```
-Listening for "Hey Bolt" (or "Thunderbolt", "Hey Assistant"). Ctrl+C to quit.
+Listening - anything you say is a request. Ctrl+C to quit.
 > what time is it
 < It's quarter past three.
-* Yes?
 > what's in my downloads folder
   file list: C:\Users\aaron\Downloads
 < You've got three installers and a PDF called taxes.
@@ -32,8 +33,11 @@ other than the system default. `run.bat --type` lets you type instead of speak,
    Silence, hum and clicks are never sent anywhere. When it hears something that sounds like
    talking, that stretch of audio is sent to your server's `/desk/stt` relay (Deepgram, as in
    Bolt-Pet) and transcribed as you speak.
-2. **Only transcripts that start with a wake phrase count.** "Hey Bolt, what time is it" asks "what
+2. **Every transcript is a request.** With `REQUIRE_WAKE_WORD` off (the default) the mic being open
-   time is it". "I need to tighten the bolt" is ignored, because the phrase has to open what you say,
+   is the activation: whatever the detector hears is sent to Bolt. A wake phrase said anyway ("Hey
+   Bolt, what time is it") is stripped, and the rest is the request. With `REQUIRE_WAKE_WORD=true`,
+   only transcripts that start with a wake phrase count: "Hey Bolt, what time is it" asks "what time
+   is it", and "I need to tighten the bolt" is ignored because the phrase has to open what you say,
    with at most a filler word such as "um" or "okay" before it. Saying just "Hey Bolt" plays a chime,
    and whatever you say in the next 8 seconds is the request.
 3. **The answer is spoken** through the server's `/desk/tts`, one sentence at a time as Bolt
@@ -44,15 +48,16 @@ other than the system default. `run.bat --type` lets you type instead of speak,
 
 ## Cost and privacy
 
-The server's speech-to-text is what recognises the wake phrase, so **everything said near the mic
+The server's speech-to-text is what turns what you say into a request, so **everything said near the
-that sounds like speech is sent to your server and transcribed**, not just things said to Bolt. The
+mic that sounds like speech is sent to your server and transcribed** — and, with the wake word off,
-server counts this against your credits like a chat turn. The local detector keeps that to actual
+answered. The server counts this against your credits like a chat turn. The local detector keeps
-speech: silence is never sent, and bursts shorter than `MIN_UTTERANCE_SEC` are dropped. In a room
+that to actual speech: silence is never sent, and bursts shorter than `MIN_UTTERANCE_SEC` are
-with a TV or a lot of conversation, it will still transcribe a lot.
+dropped. In a room with a TV or a lot of conversation, it will still transcribe and answer a lot.
 
-If that becomes a problem, the alternative is on-device wake-word detection, which is what Bolt-Pet
+If that becomes a problem, set `REQUIRE_WAKE_WORD=true` so only speech that starts with the wake
-does with openWakeWord and its `thunderbolt.onnx` model. Nothing leaves the machine until the wake
+phrase is sent on. The alternative is on-device wake-word detection, which is what Bolt-Pet does with
-word is heard, but it only recognises the phrases it was trained on.
+openWakeWord and its `thunderbolt.onnx` model. Nothing leaves the machine until the wake word is
+heard, but it only recognises the phrases it was trained on.
 
 ## What Bolt may do on this machine
 
@@ -70,9 +75,12 @@ Anything that isn't allowed is refused, and Bolt is told to say he can't do it h
 
 See `.env.example`. The ones you're most likely to change:
 
+- `REQUIRE_WAKE_WORD`: `false` (default) answers anything said near the open mic; `true` requires a
+  wake phrase before each request.
 - `VAD_RMS_THRESHOLD`: raise it if background noise keeps getting transcribed, lower it if you
   have to raise your voice.
-- `WAKE_WORDS`: the phrases, comma-separated.
+- `WAKE_WORDS`: the phrases, comma-separated (needed to activate Bolt when `REQUIRE_WAKE_WORD=true`;
+  said anyway, they're still stripped from the request).
 - `ANYWHERE_WAKE_WORDS`: words that count anywhere in a sentence, not just at the start
   (default `assistant`: "what's the weather, assistant?"). Blank to turn off.
 - `COMMAND_WAIT_SEC`, `FOLLOW_UP_SEC`: how long it waits after "Hey Bolt", or after asking you

bolt_assistant/__main__.py

@@ -1,4 +1,5 @@
-"""Bolt Assistant: say "Hey Bolt", ask, hear the answer.
+"""Bolt Assistant: talk while the mic is open, hear the answer. (Setting
+REQUIRE_WAKE_WORD=true puts "Hey Bolt" back in front of each request.)
 
     python -m bolt_assistant                 listen on the default mic
     python -m bolt_assistant --verbose       also show what was heard but ignored
@@ -89,7 +90,8 @@ def main(argv=None) -> int:
         return 0
 
     if "--type" in argv:  # the whole pipeline minus the microphone
-        print("Type what you'd say (including the wake phrase). Ctrl+C to quit.")
+        hint = " (including the wake phrase)" if assistant.wake_required else ""
+        print(f"Type what you'd say{hint}. Ctrl+C to quit.")
         try:
             while True:
                 assistant.heard(input("say> "))

bolt_assistant/assistant.py

@@ -1,7 +1,10 @@
-"""The assistant: hear the wake phrase, take the request, answer out loud.
+"""The assistant: hear a request, answer out loud.
 
     idle      every utterance the local detector passes on is transcribed by
-              the server; only ones that open with a wake phrase count
+              the server and, by default, is itself the request — the mic
+              being open is the activation. With REQUIRE_WAKE_WORD on, only
+              ones that open with a wake phrase count instead
+                "what's the time"            -> ask "what's the time"
                 "Hey Bolt, what's the time"  -> ask "what's the time"
                 "Hey Bolt"                   -> chime, go to `command`
     command   the next thing said (within COMMAND_WAIT_SEC) is the request
@@ -88,6 +91,7 @@ class Assistant:
         clock: Callable[[], float] = time.monotonic,
         wake_words=None,
         anywhere_words=None,
+        wake_required: Optional[bool] = None,
         verbose: bool = False,
     ):
         self._transcription = transcription
@@ -98,6 +102,7 @@ class Assistant:
         self.tools = tools or Tools(log=log)
         self.wake_words = wake_words or config.WAKE_WORDS
         self.anywhere_words = config.ANYWHERE_WAKE_WORDS if anywhere_words is None else anywhere_words
+        self.wake_required = config.REQUIRE_WAKE_WORD if wake_required is None else wake_required
         self.segmenter = Segmenter(config.VAD_RMS_THRESHOLD, config.VAD_START_FRAMES,
                                    config.VAD_SILENCE_END_SEC, config.MIN_UTTERANCE_SEC,
                                    config.MAX_UTTERANCE_SEC)
@@ -132,6 +137,8 @@ class Assistant:
             self.handle_transcripts()
 
     def listening_banner(self) -> str:
+        if not self.wake_required:
+            return "Listening - anything you say is a request. Ctrl+C to quit."
         return 'Listening for "Hey Bolt" (or "Thunderbolt", "Hey Assistant"). Ctrl+C to quit.'
 
     def process(self, frame: bytes) -> None:
@@ -210,6 +217,12 @@ class Assistant:
         self._check_deadline()
         woke, rest = find_wake(text, self.wake_words, self.anywhere_words)
         if self.state == "idle":
+            if not self.wake_required and not woke:
+                # Always listening: the mic being open is the activation, so
+                # whatever it managed to hear is the request.
+                self._follow_ups = 0
+                self.ask(text)
+                return
             if not woke:
                 if self.verbose:
                     self.log(f"  (not for me) {text}")

bolt_assistant/config.py

@@ -56,11 +56,18 @@ MIC_DEVICE = _device(os.environ.get("MIC_DEVICE", ""))
 SPEAKER_DEVICE = _device(os.environ.get("SPEAKER_DEVICE", ""))
 
 # ── wake phrases ────────────────────────────────────────────────────────────
+# With REQUIRE_WAKE_WORD off (the default) the mic itself is the activation:
+# every utterance the local detector passes on is treated as a request, and
+# whatever is said near an open mic gets answered. Set it to true to go back
+# to needing a phrase before each request.
+REQUIRE_WAKE_WORD = _flag("REQUIRE_WAKE_WORD", "false")
+
 # Heard through the server's speech-to-text, so these are matched against
 # *text*. A phrase counts only near the start of what was said ("hey bolt,
 # what's the time"), not buried in a sentence ("tighten that bolt"). Speech
 # recognisers often write "Bolt" as "bolts"/"volt"/"bold"; those spellings
-# only count after "hey"/"ok", where they can't be ordinary words.
+# only count after "hey"/"ok", where they can't be ordinary words. Only used
+# when REQUIRE_WAKE_WORD is on (a phrase said anyway is still stripped).
 
 WAKE_WORDS = [w.strip().lower() for w in os.environ.get(
     "WAKE_WORDS",

tests/conftest.py

@@ -0,0 +1,14 @@
+"""Test runs use the stock wake phrases, not whatever a developer happens to
+have in the project's `.env`. The speech tests are written against the
+defaults; a local `WAKE_WORDS=...` that narrows them would fail the suite for
+a reason that isn't in the code. Environment variables set by the shell still
+win, and everything else in `.env` is left alone."""
+
+import os
+
+os.environ.setdefault("WAKE_WORDS", (
+    "hey bolt,bolt,thunderbolt,hey thunderbolt,ok bolt,okay bolt,yo bolt,hi bolt,"
+    "hey assistant,ok assistant,okay assistant,"
+    "hey bolts,hey volt,hey bold,hey boat,okay volt,hey thunder bolt,thunder bolt"
+))
+os.environ.setdefault("ANYWHERE_WAKE_WORDS", "assistant")

tests/test_assistant.py

@@ -177,15 +177,31 @@ def spoken(played):
 def _config(monkeypatch):
     monkeypatch.setattr(config, "STREAMING_REPLIES", False)
     monkeypatch.setattr(config, "VAD_RMS_THRESHOLD", 500)
+    monkeypatch.setattr(config, "REQUIRE_WAKE_WORD", False)
     FakeTranscription.script = []
 
 
-def test_ignores_speech_without_the_wake_phrase():
+def test_ignores_speech_without_the_wake_phrase_when_required():
-    a, asked, played, logs, _ = make([])
+    a, asked, played, logs, _ = make([], wake_required=True)
     say(a, "so anyway I was telling him about the bolt")
     assert asked == [] and played == []
 
 
+def test_speech_without_a_wake_phrase_is_a_request_by_default():
+    a, asked, played, logs, _ = make(["It's three o'clock."])
+    say(a, "what time is it?")
+    assert asked[0].startswith("what time is it?\n\n[Bolt Assistant: a voice-only assistant")
+    assert "It's three o'clock." in spoken(played)
+    assert a.state == "idle"
+
+
+def test_the_banner_says_which_mode_is_on():
+    a, _, _, _, _ = make([])
+    assert "anything you say" in a.listening_banner()
+    required, _, _, _, _ = make([], wake_required=True)
+    assert "Hey Bolt" in required.listening_banner()
+
+
 def test_wake_phrase_and_request_in_one_breath():
     a, asked, played, logs, _ = make(["It's three o'clock."])
     say(a, "Hey Bolt, what time is it?")
@@ -205,7 +221,7 @@ def test_bare_wake_phrase_chimes_then_takes_the_next_utterance():
 
 
 def test_the_command_window_closes():
-    a, asked, played, logs, clock = make([])
+    a, asked, played, logs, clock = make([], wake_required=True)
     say(a, "Thunderbolt")
     clock.t += config.COMMAND_WAIT_SEC + 5
     say(a, "so then we left")