TheMajesticNetwork/Bolt-Console
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity

Bolt Console: desktop app with chat list / new chat, dictation, file tools

Browse Source 
Add a chat list and server-minted new chats on top of the desk API's
GET/POST /desk/chats:

- server_client.list_chats() / create_chat(), same shape and error
  handling as list_outbox_files().
- Switching is config.set_session(id): a plain reassignment of the
  SESSION_ID every call reads, persisted to session.json. The window
  refuses to switch mid-turn rather than locking.
- gui/chat_list_window.py: a Chats dialog (Ctrl+K) with a QListWidget,
  populated on open, with Refresh and New Chat. Network calls run as
  QRunnables on the global QThreadPool and come back through Signals,
  never from a slot. The header New chat (Ctrl+N) uses the same path.
- Switching clears the local transcript; there is no endpoint for a
  past chat's messages. Bolt's memory is per user, so it carries over.
- Terminal: /chats, /chat <n|id>, and /new (server-minted id, local
  fallback).

This commit also carries the uncommitted work since 54030a7 that turned
this Bolt-Pet fork into Bolt Console: the pet, wake word, TTS and screen
features removed; a Claude Code-style terminal client; the PySide6
window with tool cards, approvals and diffs; mic dictation through the
server's /desk/stt relay; filectl/JSON repairs; and detection of
invented tool results.

Build output (build/, dist/, Release/, *.spec) is now ignored.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
themajesticmagician
2026-09-25 19:50:10 -06:00
co-authored by Claude Opus 5.5
parent 54030a7b31
commit 3977804ae3
144 changed files with 5477 additions and 14736 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.env.example
+34
View File
@@ -0,0 +1,34 @@
# Bolt Console — copy to .env and fill in the first two.
# Your Bolt server and its desk API key (required).
BOLT_SERVER_URL=https://your-bolt-server.example.com
DESK_API_KEY=
# Conversation id on the server. Default: pet-console-<hostname>. Keep the
# "pet-" prefix: the server offers the filectl file tool to pet sessions.
#DESK_SESSION_ID=
# ask = confirm shell commands, file writes and edits (reads never ask)
# auto = run everything without asking
#APPROVAL_MODE=ask
# Seconds before a relayed shell command is killed.
#COMMAND_TIMEOUT_SECONDS=120
# Print sentences as they stream in (falls back automatically if unsupported).
#STREAMING_REPLIES=true
# Tell Bolt the working directory and OS with each message.
#CONSOLE_CONTEXT=true
# Files Bolt sends you with deliver_files land here.
#RECEIVE_FILES=true
#DELIVERED_FILES_DIR=~/Downloads/Bolt
# Dictation (mic button / Ctrl+M): input device index or name (blank = default),
# and how long a pause ends a recording once you've said something.
#MIC_DEVICE=
#DICTATION_SILENCE_SECONDS=3
# ANSI colour (NO_COLOR in the environment also turns it off).
#COLOR=true
.gitignore
+6
View File
@@ -5,3 +5,9 @@ __pycache__/
.pytest_cache/
.claude
CLAUDE.md
# PyInstaller output (build_bolt_console.py) and packaged releases
build/
dist/
Release/
*.spec
CLAUDE.md
+99 -567
View File
@@ -4,587 +4,119 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## What this is
A desktop pet (PySide6 window) that is a voice/click UI on top of an external
Bolt server's desk API — same brain, memory, tools, and persona as that
server's Discord bot and Linux desk client. This repo has **no import
dependency** on the server repo; it's a standalone HTTP client configured via
its own `.env`.
A PySide6 desktop app (default) and a terminal REPL (`--cli`), both Claude Code-style, on top of an
external Bolt server's desk API. It uses the same
brain, memory, tools and persona as the server's Discord bot and its other desk clients. There is
**no import dependency** on the server repo: it's a standalone HTTP client configured through its
own `.env`. It was forked from Bolt-Pet with the desktop pet, wake word, STT/TTS, screen
context/OCR, notifications, auto-updater and self-restart removed.
Pipeline: `mic → openWakeWord ("thunderbolt", on-device) / push-to-talk /
click → record utterance → the server's own /desk/stt (same relay the
Android app uses — no local Deepgram account) → + active-window + screen-
layout context → POST /desk/converse → [server may relay a shell command to
run on this machine, or a `petctl` pseudo-command that moves/emotes the pet,
jumps it to another monitor, reads a screen's text back, or plays a
multi-voice scene instead] → reply (optionally tagged with a voice the
server picked for it) → the server's own /desk/tts, streaming (or offline
pyttsx3 fallback) → speakers`, with the pet sprite/speech bubble reflecting
state throughout, and playback interruptible by talking over it (barge-in).
Because a relayed command's output goes back up the tool-result relay before
the final reply, a `petctl read` mid-turn means Bolt can look at a monitor and
then talk about what's on it in the same answer.
Side channels that let the pet act between turns: the heartbeat (proactive
announcements), the desktop notification bridge, and autonomous wandering —
all suppressed while it's napping (quiet hours / fullscreen DND).
Pipeline: `typed line → + cwd/OS note → POST /desk/converse_stream (or /desk/converse) → [server
relays a command: filectl file op, or a shell command, after y/n approval] → /desk/tool_result →
… → reply printed`.
## Commands
```bash
# Setup + run (creates .venv and installs requirements.txt on first run)
./run.sh # macOS/Linux
run.bat # Windows
# Preflight: is this install actually going to work? (config, mic, keys, sprites…)
python -m bolt_pet --doctor # shallow: no network, no mic
python -m bolt_pet --doctor --deep # contacts the server and opens the microphone
# Run tests (no pytest config file — tests self-insert repo root via sys.path).
# QT_QPA_PLATFORM=offscreen avoids a QApplication segfault on headless/no-display hosts.
QT_QPA_PLATFORM=offscreen .venv/bin/pytest tests/
.venv/bin/pytest tests/test_state.py::test_happy_path_transitions # single test
# Redraw the pet's sprite frames (the committed PNGs are this script's output)
python scripts/generate_bolt_sprites.py # --out /tmp/x to preview first
# Convert a grid sprite sheet into the per-frame-PNG convention sprite.py expects
python scripts/slice_spritesheet.py path/to/sheet.png assets/sprites/idle --cols 6 --rows 1
run.bat / run-cli.bat / ./run.sh [--cli] # window / terminal; creates .venv; keeps the caller's cwd
python -m bolt_console --doctor [--deep]
python -m bolt_console -p "message" # one-shot
.venv/Scripts/python -m pytest tests # (.venv/bin on Linux/macOS)
```
There is no lint/build step configured beyond pytest. `cp .env.example .env`
and fill in `BOLT_SERVER_URL` / `DESK_API_KEY` before running — without them
the controller logs a missing-config message and exits its thread instead of
starting. That's also all STT and normal-reply TTS need now (both go through
the server); `ELEVENLABS_VOICE_ID` picks the voice, and `ELEVENLABS_API_KEY`
is only for the one feature with no server endpoint — multi-voice
`dialoguectl` scenes, see dialogue.py below.
Runtime dependencies: `requests`, plus `PySide6` for the window only. `bolt_console.gui` is imported
lazily, so `--cli` works without Qt. The local venv is Python 3.9, so no `match`, no
runtime `X | Y` unions, and no builtin generics outside annotations.
## Architecture
## Architecture (`bolt_console/`)
- **`config.py`** — loads `.env` from the project root (not via python-dotenv;
a small hand-rolled parser matching the server repo's `desk_client/bolt_desk.py`
convention) into module-level constants. Everything else reads config from
here, never `os.environ` directly.
- **`state.py`** — `PetStateMachine`, pure logic with no Qt/audio imports (kept
that way deliberately for cheap unit testing). Enforces a transition table;
notably `IDLE -> TALKING` is legal directly (no LISTENING/THINKING leg)
because the heartbeat can make the pet speak proactively/unprompted.
- **`controller.py`** — `PetController(QObject)`, the pipeline orchestrator.
Runs on a background `QThread` (wired in `ui/app.py`) so audio I/O/network
never blocks the Qt event loop; communicates with the UI only through Qt
signals (`state_changed`, `said`, `log`, `action`, `napping`), never touches
a `QWidget` directly. Also drives the periodic heartbeat
(`_maybe_heartbeat`, gated by `HEARTBEAT_INTERVAL_SECONDS`) which lets the
server push proactive spoken announcements between user turns, and on the
same tick re-evaluates nap state, checks for delivered files, and drains
queued desktop notifications. It owns the live wake-word threshold
(`wake_threshold()` is passed to `listen_for_wake_word` as a *callable* so
the tray slider takes effect mid-listen) and the conversation `history`.
- **`server_client.py`** — HTTP client for the desk API, dependency-free
beyond `requests` so it's easy to mock in tests. `converse()` loops relaying
server-issued shell commands (`run_local_command`, executed via
`subprocess.run(shell=True)` as the desktop user, 30s default timeout) via
`/desk/tool_result` until the server sends a final `reply` (capped at
`_MAX_RELAY_HOPS`). This is the same "full desktop control" trust model as
the server repo's other desk clients — commands only ever originate from
the user's own voice/click requests in their own session. A final reply is
returned as a `Reply(text, voice_id, voice_name)` rather than a bare string,
because the server can tag it with a voice — see "Voices" below.
`list_outbox_files`
/ `download_outbox_file` hit the same `/desk/files` and `/desk/files/<id>`
endpoints the server's `deliver_files` tool queues onto — see `file_delivery.py`.
- **`file_delivery.py`** — the filesystem half of receiving files the server
queues via its `deliver_files` tool (`ai/desk_api.py` in the main tmn-api
repo — "send me that report" during a conversation spools the matched
workspace files, zipping multiple into one, onto the session's outbox).
`controller._check_deliveries` lists `/desk/files` and downloads anything
queued — right after a conversation/notification turn (the common case)
and once per heartbeat tick for anything queued out-of-band — saving each
under `DELIVERED_FILES_DIR` (default `~/Downloads/Bolt`). Downloading a
file dequeues it server-side, so it's only ever handed out once; `save()`
never overwrites an existing download, suffixing `" (1)"`, `" (2)"`, ... on
a name collision. `sanitize_filename()` reduces a server-supplied name to
its bare filename (`Path(...).name`), which is defense-in-depth against a
delivered name that's secretly a path, since a per-user desk API key means
the name isn't always coming from someone as trusted as the owner. Toggle
off entirely with `RECEIVE_FILES=false`.
- **`audio/`** — `mic.py` (energy-based VAD utterance capture, ported from the
server repo's `bolt_desk.py`), `wake_word.py` (openWakeWord `thunderbolt.onnx`
detection + `NearMissLog` for threshold tuning — see below), `stt.py` +
`stt_stream.py` (the server's own `/desk/stt` websocket relay — no local
Deepgram account; see below), `tts.py` (the server's own `/desk/tts`,
streaming by default — `stream_pcm()` + `play_stream()` start playback on
the first chunk; `chunks_to_int16()` carries odd bytes across HTTP chunk
boundaries, without which everything after the first split sample plays as
static — falling back to whole-clip PCM then offline `pyttsx3`; every entry
point takes an optional `voice_id` overriding `ELEVENLABS_VOICE_ID` — which
model to synthesize with is the server's call now, not this client's;
see `synthesize_dialogue()` further down for the one path that's still
ElevenLabs-direct), `barge_in.py` (two detectors behind one
`reset()`/`check()` shape, chosen by `BARGE_IN_MODE` via `make_detector`:
**wake** (default) scores every frame with the same openWakeWord model the
idle listener uses, so only the wake phrase cuts playback; **energy** is the
original N-consecutive-loud-frames rule, threshold ~4x the VAD one because
the mic hears the pet's own voice. Wake mode shares `_default_model` with
the idle listener — the two never run concurrently — and `reset()`s it on
detection so the tail of one reply can't count toward the next). Each
accepts an injectable stream/model/protocol so tests don't need real audio
hardware or a display.
- **`pet_actions.py`** — `petctl` pseudo-commands (`petctl move top-left`,
`petctl emote wave`, `say`/`wander`/`nap`, the screen verbs
`jump`/`monitors`/`read`, and `voice reset`). The desk API has no "move the pet" payload type
and this repo can't change the server, so these ride the existing
shell-command relay: `controller._handle_command` parses them and they never
reach `subprocess`; anything else is a real shell command exactly as before.
Pure parsing; the UI half is `PetWindow.apply_action`. Note `jump`'s target
is *not* validated here — which monitors exist is a runtime fact this pure
module doesn't have, so the spec passes through to `monitors.resolve()`.
Query verbs (`monitors`, `read`) are answered in `_handle_command` rather
than by `pet_actions.describe()`, because their output *is* the point: it
goes back up the tool-result relay for Bolt to use in his reply — as is
`voice reset`, which reports what it dropped since the server can't see
which voice is in use. `voice` only ever resets: picking one is the
server's job (`speak_as`, which it already knows how to use), so a
`petctl voice <name>` attempt is an error pointing back at that marker.
- **`self_restart.py`** — `petctl self_restart`, the pet restarting itself so
Bolt can *see* a code change he just made instead of waiting for a human to
restart it. Three problems shape it, and all three are the interesting part.
(1) The restart can't happen inline: killing the process mid-turn would drop
the HTTP tool relay before the result was posted, leaving the server to wait
out its timeout on a turn that can never finish — so the command only
*arms* it (`controller._arm_self_restart`) and
`controller._maybe_self_restart` fires it after the reply is spoken, the
same "only between turns" rule the updater follows. (2) A broken edit must
not be fatal, so `preflight()` imports the package in a **subprocess**
before arming — this process holds the old modules, so an in-process import
would pass on a file that no longer parses — and a SyntaxError comes back as
the command's output, in the same turn, with the pet still running. (3) The
reason has to outlive the process, so it's written to
`~/.cache/bolt-pet/restart_context.json` (never inside the repo Bolt is
editing) and read on the way back up by `controller._report_self_restart`,
which posts it to the server as an ordinary turn — that's what makes
"restart and check the sprites load" finish as a spoken sentence rather than
a silence. `check_loop_guard` refuses after `SELF_RESTART_MAX` restarts in
`SELF_RESTART_WINDOW_SECONDS`, so an edit-restart-crash cycle stops itself.
Off switch: `SELF_RESTART=false`.
- **`dialogue.py`** — `dialoguectl` pseudo-commands: a multi-voice *scene*
through ElevenLabs' Text to Dialogue endpoint (`audio/tts.
synthesize_dialogue`), checked in `_handle_command` between petctl and
filectl. Same single-line-JSON wire format as filectl and for the same
reason (the server's `command` marker captures only up to the next
newline), and it accepts the ElevenLabs field names (`inputs`/`voice_id`)
as well as its own (`lines`/`voice`) because the model has read that API
and copying its shape is the obvious thing to try. Voices are *named*
(`DIALOGUE_VOICES` maps names to ids) rather than pasted as raw ids, and
`self` resolves to whatever voice the pet is speaking with right now —
including a `speak_as` pick — so Bolt sounds like himself in his own
scenes. The API's limits (10 distinct voices, ~2000 characters) are
enforced *before* the request so a mistake comes back up the tool-result
relay as a sentence Bolt can act on rather than an HTTP 422 he can't see.
Unlike the normal reply path there is no streaming variant, so a scene is
whole-clip: `controller._play_dialogue` plays it with the same bubble,
transcript and barge-in handling a spoken reply gets, and returns to
THINKING afterwards (not IDLE) because the server is still waiting on the
tool result — that leg is why `state.py` allows TALKING -> THINKING.
- **`file_ops.py`** — `filectl` pseudo-commands, checked in `_handle_command`
right after petctl and before falling through to a real shell command.
Executing arbitrary commands already worked via the shell relay
(`run_local_command` — see `server_client.py` below); what filectl adds is
a *reliable* way to do the read/write/edit/list slice of that, since
getting the model to hand-roll a shell heredoc for multi-line content full
of quotes/`$`/backticks is failure-prone — and `list` exists as its own op
(rather than relying on the model shelling out to `ls`/`dir`) because this
project is cross-platform and the model shouldn't have to guess which
listing command applies on Windows vs. Linux vs. macOS; one glob-based op
(`pattern`, default `*`; `recursive` for `rglob` instead of `glob`) covers
all three. Wire format is `filectl <json>` where `<json>` is a
**single-line** compact JSON object —
`{"op": "list"|"read"|"write"|"edit", "path": ..., ...}` — not a multi-line
marker block (an earlier design): the server relays this as the argument
to the ordinary `command` tool marker, and that marker's extractor
(`ai/agents/default.py` in the main repo) only captures up to the next
newline, so anything genuinely multi-line silently got truncated no matter
how the prompt worded it. JSON sidesteps that for free — `json.dumps`
already encodes embedded newlines as the two characters `\n`, not a real
line break, so multi-line file content still fits on the one physical line
the extractor sees. `edit` requires the old text to match exactly once —
same discipline as this project's own code-editing tool — and raises
rather than guessing if it's missing or ambiguous. This doesn't expand
what the server can do to this machine (a relayed shell command could
already overwrite anything the desktop user can write — see the security
notes below); it's a safer path to the same capability. Pure parsing
(`parse`) is separated from the filesystem I/O (`execute`), matching
pet_actions.py's parse/describe split.
- **`screen_context.py`** — active-window title (xprop/xdotool, Win32,
osascript) appended to each utterance via `context_for()`, plus
`is_fullscreen_active()` for do-not-disturb. Text only — the desk API takes
no images. Every probe is best-effort and returns None/False rather than
raising; the parsing is split into pure functions that are tested without a
display server.
- **`monitors.py`** — the screen layout, and resolving `petctl jump` targets
(a 1-based number, a name, `next`/`prev`/`primary`/`other`, or a direction
like `left`/`up` worked out from the actual geometry). Pure — no Qt, no
subprocess. The monitor list is *published by the UI*
(`PetWindow.publish_monitors` builds it from `QGuiApplication.screens()` and
emits it over a queued signal to `controller.set_monitors`), because the
controller and the window must agree on what "monitor 2" means: enumerating
with `xrandr` on one side and Qt's screen list on the other gives different
orderings on the same machine, and Bolt would announce one screen and land
on another. Qt is the single source of truth; `Monitor.index` is 0-based and
`.number` is the 1-based value used in every string a human or the model
sees. The controller resolves a jump to a concrete index *before* emitting
it, so the window can't re-resolve against a different list.
- **`screen_text.py`** — OCR, so Bolt can read what's on a monitor
(`petctl read [n|here|all]`). **Pull, not push**: nothing captures on its own
— the server has to ask, and the text goes back as that command's output.
That's deliberate; OCR of a 4K screen costs a second or two that would
otherwise be added to *every* utterance, and screen contents leaving the
machine should be a visible decision rather than a constant. Capture needs
`mss` (X11/Win32/macOS, **not** Wayland), recognition needs Tesseract or
RapidOCR; both are optional and soft-fail with a reason the way `hotkey.py`
does, and `read_monitor()` never raises because its return value is command
output. Engine selection takes injected probes so it's testable wherever.
- **`quiet.py`** — quiet-hours spec parsing (`23:00-08:00`, wraps midnight,
comma-separated). Napping suppresses *proactive* noise and wandering only;
wake word / click / push-to-talk still work.
- **`notifications.py`** — Linux/D-Bus notification bridge. Two latency/loss bugs
fixed 2026-08-02, both in `controller._maybe_heartbeat`: draining was wired to the
60s heartbeat interval rather than the ~1.2s wake tick, and a heartbeat that landed
mid-conversation stamped its own clock *before* checking — burning the slot and
waiting another full interval, repeatedly, which is how a notification could go
unspoken for five or ten minutes. Draining now runs on every tick while IDLE, and the
heartbeat clock only advances when the heartbeat actually runs. Separately, the rate
limit used to *drop* notifications inside its window (a second text a minute later was
silently lost); the filter still gates at queue time but the limit is gone — a burst
is **batched into one turn** instead, same single round trip, no lost messages, capped
by `_MAX_PENDING_NOTIFICATIONS`.
- **`notifications.py` internals** — tails
`dbus-monitor`, parses Notify calls (pure `iter_notifications()`), filters
and rate-limits them (`NotificationGate`), and the controller forwards
survivors through `converse()`. Off by default — each one is a round trip.
- **`sudo_askpass.py`** — makes server-relayed `sudo` usable from a process
with no terminal, by pointing sudo's `SUDO_ASKPASS` at a GUI helper and
rewriting bare `sudo` to `sudo -A` (`add_askpass_flag`, a conservative regex
that skips anything already carrying a flag and anything inside quotes).
Prefers a real askpass binary and falls back to generating a
zenity/kdialog wrapper in `~/.cache/bolt-pet/askpass.sh`. Resolution order
is injectable (`is_executable`/`which`) so it's testable on a machine with a
different set installed. See the security notes — the dialog is the boundary.
- **`updater.py`** — self-update from the Gitea releases API. Polls
`<UPDATE_REPO_API>/releases/latest` for a tag newer than
`bolt_pet.__version__` and moves the checkout to it with
`git fetch --tags` + `git checkout tags/<tag>`, so "downloading an update"
is just git and rolling back is one command. Three safety rules: a **dirty
working tree is skipped, never stashed** (silently discarding your
work-in-progress beats running an old version); everything after the
checkout — dependency install, then an **import smoke test in a
subprocess** (this process still has the old modules loaded, so importing
in-process would prove nothing) — is guarded, and any failure rolls back to
the exact ref that was live before, branch name or SHA; and the restart only
happens once the new code imports, so a broken release costs a log line
rather than a pet that won't start. Git goes through an injectable
`run(args) -> (code, output)` callable so apply/rollback is unit-tested
against a fake git; version comparison and release parsing are pure.
`controller._maybe_update` drives it from the wake-listener tick (so the pet
is IDLE and between turns by construction) and the actual `os.execv` happens
in `ui/app.py` *after* `app.exec()` returns — that ordering is what
guarantees the mic is released before the new process opens it.
- **`history.py`** — rolling transcript (`HISTORY_LIMIT` turns) behind the
tray's History window and click-to-copy on the bubble.
- **`hotkey.py`** — global push-to-talk via `pynput`; soft-fails with a logged
reason (Wayland, missing package, macOS permissions) since the wake word is
the primary trigger.
- **`audio/stt_stream.py`** / **`audio/stt.py`** — speech-to-text via the
server's `/desk/stt` websocket relay (audio up, Deepgram's JSON messages
down untouched — the same relay the Android app uses; no local Deepgram
account or API key). `stt_stream.py` is the opportunistic optimisation:
the one-shot path waits for the utterance to end, then sends the whole
clip and waits again; that second wait is dead time that grows with how
long you spoke, and streaming removes it — `record_utterance(on_frame=...)`
hands each captured frame to a `StreamingTranscriber` as it's captured, so
by the time the VAD decides you stopped the transcript is essentially
already there. `stt.py`'s `transcribe()` is the *guaranteed* fallback for
when that didn't produce anything: since there's no separate REST endpoint
server-side, it opens the exact same relay via `stt_stream.connect()` and
just feeds the whole buffered utterance in one go — deliberately
*unconditional*, not gated by `STT_STREAMING`/`available()` the way the
opportunistic path is, since there's nothing left to fall back to if that
connection fails. Three more deliberate limits on the streaming half:
`open()` returning **None is an ordinary outcome** (no websocket-client, no
network, streaming turned off) because the full audio is still buffered
and `controller._transcribe` just falls back to `stt.transcribe()`; the
**local VAD still decides when you stopped** rather than Deepgram's
endpointing, since barge-in, follow-up listening and the grace period are
all built on it and coupling them to the network is not a first-pass
change; and the socket is **per-utterance**, because holding one open
across an idle pet bills for silence and dies on the first blip.
Off: `STT_STREAMING=false`.
- **Streamed replies** — `server_client.converse_stream()` reads NDJSON from the desk
API's `/desk/converse_stream` and speaks each sentence as it arrives
(`controller._speak_stream_chunk`), so the wait is time-to-first-sentence instead of
the whole model call. `Reply.spoken` marks a reply whose sentences were already said:
the text is still carried, because the follow-up rule needs to see whether it ended on
a question, but it must not be read out again. A stream that fails *before* anything
was spoken falls back to `converse()` invisibly; one that fails after ends the turn
quietly rather than repeating the first half. Off: `STREAMING_REPLIES=false`.
- **`speech.py`** — everything the pet says, and the policy differences between
kinds of saying. There were four near-copies of this in `controller.py` (a
reply, a holding line, a streamed sentence, a dialogue scene), each repeating
the same dance — transition state, show the bubble, maybe record history,
reset barge-in, call TTS, reset barge-in *again*, resume state, decide
whether to keep the mic open — and they had already drifted apart: one forgot
to arm barge-in, another skipped the follow-up rule. Now the dance is
`Speaker.say()` and the differences are data on a frozen `Utterance`
(`record`, `resume`, `hold_talking`, `follow_up`, `interruptible`), built by
the four classmethods `reply`/`holding`/`stream_chunk`/`scene`. Notable
policies: a holding line is **not** recorded (it's filler; the transcript
should keep the answer) and **not** interruptible (cutting off "give me a
sec" strands the tool already running), and it resumes the state it
interrupted rather than dropping to IDLE, because the turn isn't over. A
streamed chunk stays TALKING so the sprite doesn't flicker between sentences.
Collaborators are injected, so all of this is tested without Qt, audio or a
real state machine. Two subtleties that are bugs waiting to happen: the
barge-in detector is read through a **callable**, not held (it's built after
the Speaker — it needs the mic stream — and swapped when the mode changes; two
copies drifting apart is invisible until the wake model starts hearing the
pet), and `last_detail` is captured *before* the post-playback reset, since
reading it after means every interruption reports zeroed counters.
`follow_up_decision()` is the mic-open rule as a pure function.
- **Lip-sync** — the mouth is driven by the audio, not a timer.
`audio/tts.level_of(frame)` reduces a PCM frame to 0..1 loudness on a **sqrt
curve** (speech sits well below peak most of the time, so a linear map leaves
the mouth barely open during normal talking), `envelope()` does the same for a
whole clip, and playback calls the `on_level` hook per chunk. That travels
`Speaker` → `controller.mouth` (a Signal) → `PetWindow.set_mouth`, and
`_mouth_frame()` indexes the talking frames directly — which works because
`generate_bolt_sprites.py` draws them as an **openness ramp** (closed first,
widest last) rather than an arbitrary loop. Levels going stale
(`_MOUTH_STALE_SECONDS`) hands control back to the ordinary animation, so
offline TTS — which has no envelope — degrades to the timed loop instead of
freezing the mouth mid-syllable.
- **`window_state.py`** — where the pet was left, so it starts there.
`~/.cache/bolt-pet/window.json`, atomic write on drag-end, every function
swallows its own errors (a corrupt state file must mean the default corner,
never a pet that won't start). `is_visible_on()` re-validates against the
*current* screen layout on load, because the common case for a stale position
is exactly the dangerous one: the pet was last on a monitor that is now
unplugged, and restoring it faithfully puts it somewhere unreachable. Takes
plain rectangles rather than importing Qt. Off: `PET_REMEMBER_POSITION=false`.
- **`doctor.py`** — `python -m bolt_pet --doctor`, a preflight, written after a
week of debugging things one command would have shown: a venv whose python
was a zero-byte file, an OCR engine never installed, a wrong proxy header.
Twelve independent checks, each reporting ok / warn (degraded but working) /
fail, and each saying **what to do about it** — `screen reading: warn` is
useless alone, `apt install tesseract-ocr` is the whole point. Nothing raises:
a doctor that crashes on a broken install is diagnosing the wrong patient, so
`run()` catches per-check and a failed check becomes a FAIL row rather than a
traceback. Shallow by default (no network, no mic) since it's the first thing
you reach for when the network is what's broken; `--deep` actually contacts
the server and opens the microphone.
- **`speech_text.py`** — sanitizes server replies before they're heard/shown.
`for_speech()` (called inside `tts.speak()`, so every path to the speakers is
covered) strips markdown, emoji, URLs and stray symbols the voice would read
literally ("asterisk asterisk"), turns bullet lists into full sentences, and
words a few symbols (`&` → "and"). `for_display()` is the looser version for
the speech bubble — markdown syntax gone, emoji kept. `is_question()` decides
whether a reply leaves the pet waiting on an answer: it tests the *spoken*
form (so a '?' inside a stripped code block or URL doesn't count) and only a
trailing one counts, since a question asked in passing isn't awaiting a
reply. `speech.follow_up_decision` uses it to keep listening without the
wake word, capped by `FOLLOW_UP_MAX_TURNS` so a server that ends every reply
with a question can't loop forever off mic noise. Pure string logic, no
Qt/audio imports.
- **`ui/`** — `app.py` wires `QApplication` + `PetWindow` + `PetTray` + the
history/tuner windows + the push-to-talk hotkey + the controller thread
together; `pet_window.py` is the frameless/translucent/always-on-top sprite
window + speech bubble (non-square frames are centered in the square
`PET_SIZE` window, see `paintEvent`), and also owns:
- **wandering** — a ~30fps timer walks the window toward a random on-screen
target every `PET_WANDER_INTERVAL_SECONDS` (randomized), suppressed
whenever the pet is non-IDLE, napping, dragged, or has a bubble up. A
commanded `petctl move` overrides all of that except the drag.
- **the walk cycle** — while actually travelling, `_animation_key()` swaps
the state animation for the side-view `walk/` frames (not a `PetState` —
see the sprites README). It is stepped by *distance travelled*
(`_WALK_PIXELS_PER_FRAME`), never by the animation timer, so the planted
paw tracks backwards at exactly the speed the window moves forwards;
`_advance_frame` deliberately no-ops while walking so the two can't
double-step it. The art is drawn facing right and `_oriented()` mirrors it
(cached per frame) when heading left. No `walk/` art → falls back to the
old coded bob rather than a placeholder blob.
- **emotes** — `emote_transform()` is pure maths (dx, dy, rotation, scale
from a 0..1 progress) kept out of `paintEvent` so the curves are unit
tested; every emote must return to the identity transform at progress 1.0
or the pet ends up permanently askew.
- **shaped input / click-through** — `PET_SHAPED_INPUT` masks the window to
the sprite's opaque pixels so the square window's transparent corners stop
eating clicks (mask rebuilt only when the frame changes, and pinned to the
resting position so a bob/spin doesn't thrash it); `PET_CLICK_THROUGH`
makes the pet ignore the mouse entirely.
- **edge snapping** (`PET_EDGE_SNAP`) after a drag or a stroll, and **nap
dimming** (`set_napping`).
`sprite.py` loads `assets/sprites/<state>/*.png` (filename-sorted, looping —
the art is *generated* by `scripts/generate_bolt_sprites.py`, a Pillow
drawing of Bolt as a shepherd pup; edit the script and re-run it rather than
the committed PNGs, see `assets/sprites/README.md`) and falls
back to a procedurally-drawn placeholder blob per state if a folder has no
frames. It also loads `EXTRA_ANIMATIONS` — currently just `walk/` — keyed by
name rather than by `PetState`, with `has()` reporting whether a key is
backed by real art so callers can decline a placeholder instead of trotting
a blob across the desktop; `tray.py` is the system tray menu (talk now / mute / nap / wander /
click-through / history / wake-word tuning / use-default-voice / quit) — the
pet window has no title bar or taskbar entry; `history_window.py` and
`wake_tuner.py` are the two dialogs it opens.
- **`config.py`**: a small hand-written `.env` parser feeding module constants. Nothing else reads
`os.environ`. `SESSION_ID` defaults to `pet-console-<host>`. The `pet-` prefix is deliberate: the
server advertises `filectl` to pet sessions, and `-console` keeps this a separate conversation
from a Bolt-Pet on the same machine. `new_session()` (New chat / `/new`) rotates to
`<base>-<timestamp>` and remembers it in `~/.cache/bolt-console/session.json`. `DESK_SESSION_ID`
in `.env` overrides the remembered id. Tests must monkeypatch `config.STATE_FILE`.
- **`server_client.py`**: the desk protocol. `_relay()` is the shared command → tool_result loop.
`converse_stream()` hands each `say` sentence over as it arrives and finishes a tool call through
`_relay`. It raises `ServerError` only if nothing was shown yet, which lets the caller fall back to
`converse()` invisibly. `Reply.streamed` means "already on screen, don't print again".
`run_local_command` runs in `os.getcwd()` with stdin closed and `CREATE_NO_WINDOW`, because under
pythonw every cmd.exe would otherwise flash a console.
- **`tools.py`**: `Tools`, the `on_command` callable. It routes `filectl` to `file_ops`, answers
`petctl`/`dialoguectl` with `PET_UNAVAILABLE` (the server may still emit them), and treats
anything else as a shell command. Approval: reads and lists never ask. Shell, write and edit ask
via `ui.confirm` unless `mode == "auto"`, and "always" switches the session to auto. A decline
returns `DECLINED` to the model rather than raising, so the turn still completes.
- **`file_ops.py`**: `filectl {json}` list/read/write/edit. It is a **single-line** JSON wire format
because the server's `command` marker only captures to the first newline. `edit` must match
exactly once. Parsing (`parse`) is separate from I/O (`execute`).
- **`relay_json.py`**: lenient JSON for model-typed payloads. It tries strict parsing first, then
named repairs (fences, smart quotes, invalid backslash escapes, stray quotes, trailing commas,
Python literals, single quotes), and accepts a repair only if the result parses. Repairs are
always reported back to the model, never silent. The backslash repair only touches *invalid*
escapes, so `C:\temp` still parses as a tab (that's ambiguous, so it's left alone).
- **`ui.py`**: `Terminal` handles plain ANSI rendering, the spinner thread, tool-call/result lines,
diffs, `confirm` and `read_message`. All output goes through `_emit` under the same lock the
spinner draws under. The spinner redraws by erasing the current line, so output is always written
as whole lines (see `assistant_chunk`). Glyphs fall back to ASCII on non-UTF-8 streams.
- **`console.py`**: `Session` (one turn, delivered-file download, slash commands, `!` local shell,
the loop) and `run()` (argv). Collaborators are injected so tests can drive it.
- **`gui/`**: the window. `bridge.Bridge` implements the same surface as `ui.Terminal`
(tool_call, tool_result, confirm, assistant_chunk, …) with each call emitting a Qt signal, so the
window reuses `console.Session` and `Tools` unchanged. `window.TurnRunner` runs `Session.send` on a
QThread. `Bridge.confirm` blocks the worker on an Event until a ToolCard's Allow/Deny calls
`answer()`, and `close()` releases it as "no". `window.MainWindow` reacts only to Bridge signals.
`widgets.py` holds the transcript pieces (ToolCard, AssistantMessage, Composer, …). `theme.py` holds
the colours and the one stylesheet (widgets set object names and dynamic properties for it, then
`repolish()`). `icons.py` draws every icon with QPainter, so there are no asset files.
`markdown_html` swaps Qt's fixed font for code (Courier New on Windows) with the app's mono font.
- **`dictation.py`**: the mic button. It streams 16 kHz int16 to the server's `/desk/stt`
websocket (Bolt-Pet's route: the server relays to Deepgram) and buffers every frame. If the live
socket never opened or died, `_fallback()` sends the whole buffer once from an empty transcript,
so a half-live session's words don't come back twice. A healthy socket is never re-sent, even
when it heard nothing, because that would pay twice. Stops on click, on `silence_seconds` after
speech, or on `max_seconds`. No Qt and no numpy (the array module does RMS), so it's identical
in the Linux single file. The mic and socket are injected. The window turns the callbacks into
`DictationSignals`, and the composer is read-only while a recording rewrites it. PySide6 6.10
needs `QTextCursor.MoveOperation.End`, not `cursor.End`.
- **Chats**: `server_client.list_chats()`/`create_chat()` wrap `GET`/`POST /desk/chats` (the server
mints `chat-…` ids; a chat is only listed once used). Switching is `config.set_session(id)`: a
plain reassignment of the `SESSION_ID` every call reads, persisted to `session.json`, and
deliberately not locked. The window refuses to switch mid-turn instead. `gui/chat_list_window.py`
runs list/create as `ChatTask`s on `QThreadPool.globalInstance()` and hands results back through
`ChatSignals`, never calling the network from a slot. The header New chat uses the same pool.
`Session.new_conversation()` (terminal `/new`) is blocking and falls back to a local id if the
server can't mint one. `chats.py` holds the Qt-free row helpers (the preview drops the
`[Bolt Console: …]` and `[on screen right now: …]` notes).
- **`file_delivery.py`**: saves files the server queues via `deliver_files`. The name is reduced to
its basename and never overwrites an existing file. `Session.check_deliveries` runs after every
turn.
- **`doctor.py`**: independent checks, each reporting ok/warn/fail plus what to do about it. A check
never raises.
### Voices (the server's `speak_as`)
## Testing conventions
Ask Bolt to talk like someone else, or in another language, and the *server*
does the picking: its desk-only `voice_search` marker browses the ElevenLabs
voice library, and `speak_as: <voice_id>` on the final reply tags that reply
with the chosen voice (adding a Voice Library pick to the ElevenLabs account
first, so the id is usable by the time it reaches us). Nothing about that is
this repo's to decide — all the client owes it is actually speaking in the
voice it was handed: `converse()` returns it on `Reply`, `_apply_voice()`
records it, and `_speak()` passes it to `tts.speak(voice_id=...)`.
`tests/test_gui.py` sets `QT_QPA_PLATFORM=offscreen` itself and drives real turns through
`MainWindow` and its worker thread against the loopback server (`pump_until` processes events).
Offscreen Qt has **no font database**, so check the look by rendering with the real platform plus
`WA_DontShowOnScreen` and `grab()`, not offscreen.
Two things are decided *here*, though, because the server can't:
Tests cover logic with injected fakes (`FakeUI`, `FakeShell`, a `Terminal` writing to `StringIO`
with a scripted `input_fn`). `tests/test_console.py` is the end-to-end file: a real threaded HTTP
server on a loopback port driven through the real `server_client`, `Tools` and `Terminal`. Add a
case there when a bug crosses module boundaries. Tests that embed a path in JSON must use
`json.dumps(str(path))`, because Windows backslashes are otherwise invalid escapes.
- **The voice sticks** (`VOICE_STICKY`, default on). The server tags one
reply and strips the marker before storing the turn, so it never sees the
id again — "keep talking like that" would send it searching for a voice all
over again, and it'd likely land on a different one. Holding the id
client-side is what makes the rest of the conversation stay in that voice.
An untagged reply therefore never *changes* the voice; only a new
`speak_as`, `VOICE_STICKY=false`, or a reset does.
- **There's a way back.** Since the server was never told Bolt's own voice
id, it can't ask for it back with `speak_as` — so reverting is local: the
tray's **Use default voice** entry (enabled only while a picked voice is
in use, kept in sync by the `voice_changed` signal), a restart, or
`petctl voice reset`, which is what lets Bolt honour "go back to your
normal voice" out loud. That last one needs the server's pet prompt block
(`ai/desk_api.py`, `pet_tools`) to mention the verb, or the model never
emits it — the desk API's prompt is where petctl is advertised.
## Fabricated tool results
### Wake-word detection
`audio/wake_word.py` uses a custom-trained openWakeWord model,
`thunderbolt.onnx` (ships in the project root), the same way the server
repo's `desk_client/bolt_desk.py` uses `bolt.onnx` for "hey bolt" — same
runtime (openWakeWord, ONNX inference framework), same per-frame
`predict()`/`reset()` loop. Every mic frame is scored; any class score at or
above `WAKE_WORD_THRESHOLD` (default `0.5`, in `.env`) counts as a
detection. Swap `WAKE_MODEL_FILE` to point at a differently-trained `.onnx`
model to change the wake phrase — everything downstream (STT, server call,
TTS) is unaffected.
**openwakeword's `Model.reset()` is not enough to forget a detection.** It
clears the *prediction* buffer only; the rolling audio window the classifier
actually scores lives in `model.preprocessor` (`raw_data_buffer` — 10s of raw
audio — plus `melspectrogram_buffer` and a ~120-frame `feature_buffer`) and
`AudioFeatures` has no reset method at all. So after a detection the wake
phrase is still in the window, and the next frame fed to the model re-fires on
it. Symptom when this bites: the pet cuts itself off a word into every reply,
because wake-mode barge-in resumes feeding the model and instantly matches the
"thunderbolt" that *started* the turn. `wake_word.hard_reset(model)` restores
the preprocessor to its as-constructed (silence) state and is what both
`listen_for_wake_word` and `WakeWordBargeIn.reset()` call — use it, not
`reset()`, anywhere a detection needs to be genuinely forgotten. The blank
state is cached on the preprocessor object (not in an `id()`-keyed dict —
CPython reuses ids after GC), since rebuilding it costs an ONNX pass over 10s
of silence.
The threshold is tunable at runtime: the tray's **Wake word tuning…** window
(`ui/wake_tuner.py`) shows the peak score seen and a rolling list of near
misses (frames within `WAKE_NEAR_MISS_MARGIN` *below* the threshold — i.e.
the times it nearly heard you), and its slider is read per frame because
`listen_for_wake_word` accepts a callable threshold. Set the threshold just
under the peak you can hit reliably, then persist it in `.env`.
### Testing conventions
`tests/` covers pure logic only (state machine, wake-word scoring loop, mic
VAD, HTTP client against mocks) — nothing there needs real audio hardware or
a display. Modules under test are written to accept fake streams/models/
`on_command` callables specifically to keep tests hardware-free; follow that
pattern (inject a `Protocol`-typed collaborator) rather than mocking at the
`sounddevice`/`openwakeword` import boundary when adding new testable logic.
`test_controller.py`, `test_controller_features.py`, `test_wander.py` and
`test_pet_window_features.py` need a `QApplication`, which segfaults without
a display unless run with `QT_QPA_PLATFORM=offscreen`.
Newer subsystems follow the same rule — the testable part is separated from
the part that needs hardware: dbus-monitor output is parsed by a pure
`iter_notifications(lines)`, xprop output by pure `parse_xprop_*` functions,
HTTP chunk reassembly by `chunks_to_int16`, emote motion by
`emote_transform`. Tests that touch the controller monkeypatch
`screen_context.context_for` / `is_fullscreen_active`, otherwise they shell
out to xprop on a headless box.
**`test_pipeline_smoke.py` is the exception, deliberately.** Unit tests inject
a fake at the seam they care about, and a run of production bugs — notifications
sitting unspoken for minutes, the pet saying things twice, `[laughing]` read
aloud, a device command arriving as prose — got through with every one of them
green, because each was an *interaction* between two individually-correct
units. So that file stands up a real threaded HTTP server on a loopback port,
speaks the desk protocol at it, and drives whole turns through the real
`server_client` (including NDJSON streaming), the real controller and the real
state machine, faking only the mic stream and the speakers. When a bug crosses
a module boundary, add the case there; when it lives inside one module, the
pure-function pattern above is still the cheaper test. It is also worth
mutation-checking a new case — break the source line it is meant to catch and
confirm it actually goes red.
The model sometimes writes tool output as prose instead of emitting a command. The tool marker
syntax in `console.annotate()` is load-bearing: vague wording ("use filectl") measurably caused
this against the live server, and spelling out `command: ...` fixed it. `console.fabricated()`
flags a reply shaped like tool output when `Tools.calls` didn't move, and the next message carries
`CORRECTION`. A polluted session keeps copying its own history, so the remedy is New chat.
## Security notes
The server can relay a shell command back to this machine to execute as the
desktop user (see `server_client.run_local_command`). This is intentional
("full desktop control" for things like "open firefox" or disk checks) and
matches the trust model of the server repo's other desk clients. Keep
`DESK_API_KEY` private and don't expose the desk API port to the open
internet.
`file_ops.py`'s `filectl` read/write/edit pseudo-commands ride that same
relay and are bound by the same trust model — no path is off-limits beyond
normal filesystem permissions for the desktop user, exactly like a relayed
`cat`/`sed`/`rm` already isn't. They don't grant the server anything a shell
command couldn't already do; they just make the read/write/edit path
reliable instead of relying on the model getting shell quoting right.
`sudo_askpass.py` widens that further, by design: with `SUDO_ASKPASS_PROMPT`
on (the default), a relayed bare `sudo` is rewritten to `sudo -A` and the
password is collected in a desktop dialog, so commands can escalate to root
instead of hanging on a tty the pet doesn't have. The dialog is the security
boundary — it's the only thing between the server deciding to run `sudo` and
it running, so the prompt is deliberately not suppressible per-command and
those commands get their own longer timeout (`SUDO_COMMAND_TIMEOUT_SECONDS`)
rather than being made non-interactive. Set `SUDO_ASKPASS_PROMPT=false` to
take the capability away entirely; sudo commands then fail. Note that
`sudo -n` / `sudo -A` / `sudo -u …` in a relayed command are never rewritten,
so an explicit non-interactive sudo stays non-interactive.
**Don't run the pet as root.** It needs no privileges of its own, PortAudio
can't reach the user's PipeWire socket from a root session (raw ALSA devices
reject the 16 kHz capture rate — `paInvalidSampleRate`), and every relayed
command would run unconstrained.
Several features widen what leaves this machine, all switchable in `.env`:
`SCREEN_CONTEXT` appends the focused window's *title* to each utterance
(titles often contain file paths, document names, or subject lines),
`MONITOR_CONTEXT` appends the screen layout (sizes and names only — no
contents), and `NOTIFICATION_BRIDGE` (off by default) forwards matching
desktop notifications to the server. None of those send screenshots or
notification contents you haven't matched with `NOTIFICATION_FILTER`.
`SCREEN_TEXT` is the biggest of them: `petctl read` OCRs a whole monitor and
sends the recognised text to the server — everything visible, not just the
focused window. Two things keep it honest. It's **pull-only**: no capture
happens unless the server explicitly asks, so it can't leak in the background
the way a per-turn annotation would, and each read is logged. And it is
strictly *not* a new capability — the shell relay could already run a
screenshot tool and pipe it through OCR — it just makes a thing the trust
model already allowed reliable, bounded (`SCREEN_TEXT_MAX_CHARS`) and
visible. It is nonetheless far easier to reach for than the shell route, so
if that trade isn't one you want, `SCREEN_TEXT=false` removes it and
`petctl read` starts reporting that it's disabled. Capture is `mss`-based and
therefore silently unavailable on Wayland.
Approved shell commands, writes and edits run as the user with no sandbox. `auto` mode removes the
prompt. That is the same "full desktop control" trust model as the server's other desk clients,
which is why `ask` is the default here. Keep `DESK_API_KEY` private.
README.md
+101 -229
View File
@@ -1,250 +1,122 @@
# Bolt Desktop Pet
# Bolt Console
A little animated pet that lives on your desktop and is just a face on top
of your Bolt server — same brain, memory, tools, and persona as Discord
chat and the Linux desk client. It talks to `ai/desk_api.py` on the server
exactly the way `desk_client/bolt_desk.py` does — including speech: STT and
TTS are the server's own `/desk/stt` and `/desk/tts`, the same endpoints the
Android app uses, so there's no separate Deepgram or ElevenLabs account to
set up for the pet to talk. Playback is cross-platform (no `mpv`/`ffplay`/
`espeak-ng` subprocess calls — pure `sounddevice`).
A desktop app for Bolt, in the style of Claude Code (with a terminal version too). You type, Bolt
answers, and when he needs to look at or change something on this machine you see each step as it
happens: reading files, listing folders, editing, writing, running commands. Changes wait for your
**Allow** on the step itself, and edits show a diff first.
It talks to the same Bolt server as Bolt-Pet, the Discord bot and the phone app, so it has the same
brain, memory, tools and persona. What it drops is the desktop pet around it: no sprite, no wake
word, no microphone, no speech, no screen reading.
The window is PySide6 with its own dark theme. Header: working folder (click to change) and an
**Ask first / Auto** switch. Below that: the conversation, with each tool call as a card that you can
click to expand, and a composer at the bottom (Enter sends, Shift+Enter adds a new line, Ctrl+L
clears, Ctrl+O picks a folder).
**Chats** (Ctrl+K) lists every conversation you've had with Bolt, from any device (console, pet,
phone), newest first. Open one to continue it. **New chat** (Ctrl+N, or the button in the Chats
dialog) asks the server for a fresh chat: new conversation history, but Bolt still remembers you,
because his memory belongs to you, not to a chat. Switching clears the window's transcript, since
the server has no endpoint for an old chat's messages, although Bolt still has them. The current
chat is remembered across restarts. The terminal version has `/chats`, `/chat <n>` and `/new`.
**Dictation:** the mic button next to Send (or Ctrl+M) records you and types what you say into the box
as you talk. Click again or pause for 3 seconds to stop, and Esc cancels. The text isn't sent until you
press Enter, so you can fix it first. It uses the same speech-to-text as Bolt-Pet: the Bolt server's
`/desk/stt` relay to Deepgram, so you don't need a Deepgram account. The server does count it against
your credits like a chat turn. `MIC_DEVICE` in `.env` picks the microphone.
The terminal version (`run-cli.bat` / `./run.sh --cli`) looks like this:
```
mic → wake-phrase spotter ("thunderbolt") / hotkey / click → record utterance
→ server-hosted STT (+ the focused window's title, for "what's this
error?") → POST /desk/converse on your Bolt server → [server may relay
a shell command back to run on THIS machine, or a `petctl` command
that moves or emotes the pet] → reply → server-hosted streaming TTS
→ speakers
→ shown in a speech bubble + the pet's sprite state (idle/listening/
thinking/talking) updates the whole time
> what's taking up space in this repo?
● List(C:\work\site, recursive)
⎿ assets/
assets/hero.mp4 48211332B
... +212 lines
● Shell(du -sh assets/* | sort -h | tail -5)
Run this command? [y]es / [n]o / [a]lways this session y
⎿ 48M assets/hero.mp4
...
● Most of it is one file: assets/hero.mp4 at 48 MB. ...
```
Nothing is sent to the server until the wake phrase fires, you press the
push-to-talk hotkey, or you click the pet — plus, if you turn them on, the
heartbeat and the desktop-notification bridge.
## Why a separate project instead of living in the tmn-api repo
This runs on your desktop machine, not the server — same relationship as
`desk_client/` (Linux) or the Android app, both of which are just clients of
the desk API over HTTP. It has no import dependency on the server repo at
all, so it can be copied anywhere and configured with its own `.env`.
## Setup
1. Copy this whole `bolt-pet/` folder to the machine you want the pet to run
on (if that isn't already this machine).
2. `cp .env.example .env` and fill in:
- `BOLT_SERVER_URL` + `DESK_API_KEY` — same as `desk_client/.env` on the
server side. Use the server's master `DESK_API_KEY`, or mint yourself a
personal one via the desk-only `api_key_generate` marker (see the main
repo's `CLAUDE.md` → "Per-user API keys"). That's it for STT — no
Deepgram account needed, it goes through the server's own `/desk/stt`.
- `ELEVENLABS_VOICE_ID` for TTS (optional — falls back to offline TTS via
`pyttsx3` if omitted or if the server call fails). No local ElevenLabs
API key needed for this either; `ELEVENLABS_API_KEY` is only for the
multi-voice `dialoguectl` scenes further down, the one feature the
server has no endpoint for.
3. Run it:
- macOS/Linux: `./run.sh`
- Windows: `run.bat`
```bash
cp .env.example .env # set BOLT_SERVER_URL and DESK_API_KEY
run.bat # Windows: the window
run-cli.bat # Windows: the terminal version
./run.sh [--cli] # macOS/Linux
```
Both scripts create a local `.venv` and install `requirements.txt` on
first run. On Linux you'll also need system packages for audio:
`sudo apt install libportaudio2 espeak-ng`.
The pet appears near the bottom-right of your screen. It wanders off on its
own now and then; drag it anywhere and it tucks itself flush against a nearby
screen edge. That position isn't saved across restarts (see Known
limitations).
## Talking to it
- Say **"thunderbolt"** — detected fully on-device by a custom-trained
openWakeWord model (`thunderbolt.onnx`, ships in the project root), the
same way the server repo's `desk_client/bolt_desk.py` detects "hey bolt"
with `bolt.onnx`. Matches the `DEFAULT_WAKE_WORD` already used for Bolt's
Discord voice channels, so it's the same word everywhere.
- Or press **Ctrl+Alt+Space** (`PUSH_TO_TALK_HOTKEY`) from anywhere — useful
in a noisy room where the wake word misfires. Needs `pynput` and a session
that allows global key hooks; most Wayland sessions don't, in which case it
logs why at startup and everything else still works.
- Or just **click the pet** once (a drag doesn't count as a click).
- **Talk over it** to cut a long answer short — the mic stays live while it
speaks, and barging in starts your next turn immediately (`BARGE_IN`).
- Right-click the tray icon for **Talk now**, **Mute mic**, **Nap**,
**Wander around**, **Click through the pet**, **History…**, **Wake word
tuning…**, **Use default voice** and **Quit** — the pet window itself has
no title bar or taskbar entry.
- **Click the speech bubble** to copy what it just said; the tray's
**History…** window keeps the last `HISTORY_LIMIT` turns.
## What it does on its own
- **Wanders** the desktop while idle (`PET_WANDER`), stands still while
listening/thinking/talking or while a bubble is up.
- **Moves and emotes on command.** Bolt can relay `petctl move top-left`,
`petctl emote wave|hop|spin|nod|shake`, `petctl say ...`, `petctl wander
on|off`, `petctl nap on|off`, `petctl voice reset`, and `dialoguectl` for a
multi-voice scene. These are intercepted here and never reach a shell.
- **Naps** during `QUIET_HOURS` (e.g. `23:00-08:00`) or while a fullscreen
app is focused (`DND_ON_FULLSCREEN`) — it dims, stops wandering, and makes
no proactive noise. It still answers when you speak to it.
- **Reacts to desktop notifications** if you turn on `NOTIFICATION_BRIDGE`
(Linux/D-Bus) and set a `NOTIFICATION_FILTER` regex — matching
notifications get forwarded to the server, so it can tell you the deploy
went green. Off by default: each one costs a round trip.
## Speaking in another voice
Ask for a different voice — "use a clearer voice", "talk like a pirate", "say
that in Japanese" — and Bolt searches the ElevenLabs voice library on the
server, picks one, and tags his reply with it (`speak_as`); the pet is what
actually speaks in it. A Voice Library pick is added to your ElevenLabs
account automatically the first time it's used. The server also synthesizes
with one fixed, multilingual-capable model for every request now (not a
flash/multilingual switch per reply) — nothing to configure on this side,
and non-English text or a picked voice no longer needs special-casing here.
The new voice **stays on** for the rest of the conversation, because the
server tags a single reply and doesn't remember which voice it chose — so
"keep talking like that" would otherwise send it hunting for a voice again.
To get his own voice back: ask him ("use your normal voice" — he relays
`petctl voice reset`), use **Use default voice** in the tray menu (greyed
out unless a picked voice is active), or restart the pet. Set
`VOICE_STICKY=false` in `.env` if you'd rather each pick lasted exactly one
reply.
## Multi-voice dialogue
Ask for a scene — "do the argument between the two of them", "read that back
as a radio play" — and Bolt can relay a `dialoguectl` command that the pet
renders through ElevenLabs' Text to Dialogue endpoint: several voices in one
take, with delivery tags the v3 model acts on (`[cheerfully]`, `[whispering]`,
`[stuttering]`). One request per scene, so the voices actually react to each
other instead of sounding like clips glued together.
Name the cast in `.env` (`DIALOGUE_VOICES=narrator:9BWts…,villain:IKne3…`);
the name `self` always means whatever voice the pet is currently using, so
Bolt sounds like himself in his own scenes — including after a `speak_as`
switch. Scenes show up in the speech bubble with the tags stripped, count as
normal speech for the transcript, and can be talked over like any other reply.
`DIALOGUE=false` turns the whole thing off on this device.
## Wake-word detection
`bolt_pet/audio/wake_word.py` feeds every mic frame into `thunderbolt.onnx`
via the openWakeWord runtime (ONNX inference) and treats any class score at
or above `WAKE_WORD_THRESHOLD` (default `0.5`) as a detection — the exact
same per-frame `predict()`/`reset()` pattern as `desk_client/bolt_desk.py`'s
main loop. Point `WAKE_MODEL_FILE` in `.env` at a different `.onnx` model to
change the wake phrase later without touching any other code.
If it keeps ignoring you (or firing at the TV), open **Wake word tuning…**
from the tray: it shows the peak score while you talk and a rolling list of
near misses — frames that scored just under the threshold — and the slider
takes effect immediately, mid-listen. Set the threshold just below the peak
you can hit reliably, then write it into `.env` as `WAKE_WORD_THRESHOLD`.
## Something not working?
The launchers create `.venv` on first run. Dependencies: `requests`, plus `PySide6` for the window
(the terminal version doesn't import Qt). Bolt works in the
directory you **launch from**, so you can put the launcher on your PATH and start it inside whatever
project you want him to work on.
```bash
python -m bolt_pet --doctor
run.bat --auto # don't ask before commands, writes and edits
run-cli.bat -p "summarise README.md" # one message, print the reply, exit
run-cli.bat --doctor [--deep] # check the install (--deep contacts the server)
```
Checks the things that make the pet look broken in ways that don't point at
themselves — missing server config (the controller exits its thread at startup,
so the pet appears alive and simply never answers), no input device, no OCR
engine behind `petctl read`, a wake model that isn't where `.env` says, a
silence timeout long enough to feel like lag. Each line says what to do about
it, not just what's wrong. It doesn't touch the network or open the microphone
unless you add `--deep`, so it's safe to run when the network is the suspect.
## Using the terminal version
## Little things
| Input | What it does |
|---|---|
| any text | sent to Bolt |
| `!<command>` | runs a shell command yourself. Nothing is sent to Bolt |
| `/cd <dir>`, `/pwd` | change or show the working directory |
| `/ask`, `/auto` | turn approval prompts on or off for this session |
| `/status` | server, session, approval mode, working directory |
| `/files` | download anything Bolt has queued for you |
| `/new` | start a fresh chat (same as New chat in the window) |
| `/chats`, `/chat <n>` | list your chats from every device; switch to number n (or a session id) |
| `/clear` | clear the screen. Bolt's memory lives on the server and is kept |
| `/help`, `/exit` | |
| line ending in `\` | keep typing on the next line |
The pet **remembers where you left it** — drag it somewhere deliberate and
that's where it starts next time. If that position is on a monitor you've since
unplugged it goes back to the default corner rather than restoring itself
somewhere off-screen. `PET_REMEMBER_POSITION=false` to always start in the
corner.
Ctrl+C stops the spinner and gives the prompt back. The server may still finish that turn on its
side. At an approval prompt, Ctrl+C means no.
Its **mouth moves with the actual audio** rather than flapping on a timer: the
PCM going to the speakers is reduced to a loudness per frame and that picks the
talking sprite, so the pet shuts up when the voice pauses. Offline `pyttsx3`
playback has no waveform to follow, so it falls back to the timed loop.
## How Bolt reaches this machine
## Project layout
The desk API has one tool channel: the server sends back a `command` string, the client runs it and
posts the output to `/desk/tool_result`, and this repeats until a final reply. The console decides
what each string means (`bolt_console/tools.py`):
```
bolt_pet/
config.py .env loading (same pattern as desk_client/bolt_desk.py)
state.py PetState enum + a small transition-checked state machine
server_client.py /desk/converse, /desk/tool_result, /desk/report_status
controller.py the pipeline: wake word -> STT -> server -> TTS, on a QThread
speech.py what the pet says and how each kind of saying behaves
speech_text.py strips markdown/emoji/URLs so the voice never says "asterisk"
pet_actions.py petctl move/emote/say/wander/nap parsing
screen_context.py active-window title + fullscreen detection
quiet.py quiet-hours schedule
notifications.py desktop notification bridge (Linux/D-Bus)
history.py rolling conversation transcript
hotkey.py global push-to-talk (pynput, optional)
window_state.py remembers where you left the pet
doctor.py `--doctor` preflight: is this install going to work?
audio/
mic.py input stream + energy-based VAD utterance capture
wake_word.py openWakeWord thunderbolt.onnx detection (see above)
stt.py server /desk/stt relay (one-shot: whole utterance at once)
stt_stream.py server /desk/stt live websocket — transcribes while you speak
tts.py server /desk/tts streaming PCM, offline pyttsx3 fallback,
plus the loudness envelope that drives the mouth
barge_in.py "you started talking" detector, to cut playback short
ui/
app.py wires QApplication + window + tray + controller thread together
pet_window.py frameless/translucent/always-on-top sprite window + speech bubble
sprite.py frame animation loader (see assets/sprites/README.md)
tray.py system tray menu
history_window.py conversation scrollback (copyable)
wake_tuner.py live wake-word threshold + near-miss log
assets/sprites/ Kenney robot-pack art (CC0) — see assets/sprites/README.md
scripts/
slice_spritesheet.py cuts a grid sprite sheet into the per-frame convention
tests/ pure-logic unit tests (state machine, wake-phrase
matching, HTTP client against mocks) plus one
end-to-end smoke test that drives whole turns against
a real local HTTP server — nothing here needs real
audio hardware or a display
- **`filectl {json}`**: list, read, write or edit a file (`file_ops.py`). This is what makes file
editing reliable, because the model never has to hand-quote a shell heredoc. Edits must match
exactly once. Common JSON slips, such as a raw `C:\Users\...` path, are repaired and the model is
told about the repair.
- **`petctl …` / `dialoguectl …`**: the pet's body and voice. These are not run. Bolt is told
there's no pet here.
- **anything else**: a real shell command, run in the working directory, with a
`COMMAND_TIMEOUT_SECONDS` limit (default 120s).
Reads and listings never ask. Shell commands, writes and edits ask first in `ask` mode (the
default). Edits show a diff before you answer.
## Security
Anything you approve runs as your user, with your permissions. In `auto` mode that means anything
the server asks for. Keep `DESK_API_KEY` private and don't expose the desk API to the open internet.
Each message includes your working directory and OS name (`CONSOLE_CONTEXT=false` to stop that).
## Development
```bash
QT_QPA_PLATFORM=offscreen .venv/bin/pytest tests/ # Windows: .venv\Scripts\pytest tests
```
## Security notes
`tests/test_gui.py` drives turns through the real window and its worker thread against the same
loopback server.
Same as `desk_client/bolt_desk.py`: the server can relay a shell command
back to this machine ("full desktop control" — "open firefox", "how full is
my disk", etc.), which this client executes as your desktop user with a
30-second timeout (`COMMAND_TIMEOUT_SECONDS`). That's the same trust model
as the Linux desk client and the Android app — commands only ever originate
from your own voice/click requests in your own session. Keep `DESK_API_KEY`
private; don't expose the desk API port to the open internet.
`tests/test_console.py` runs whole turns against a real HTTP server on a loopback port. Only the
keyboard is faked. Everything else takes its collaborators as arguments, so no test needs a real
terminal or server.
`petctl` commands (move/emote/say/wander/nap) are handled inside the pet and
never reach a shell, so that channel can't run anything.
Two features widen what leaves this machine, both off-switchable in `.env`:
`SCREEN_CONTEXT=true` (default) appends the focused window's *title* to what
you say — titles often contain file paths, document names or email subjects —
and `NOTIFICATION_BRIDGE=false` (default) can forward matching desktop
notifications. No screenshots or images are ever sent.
## Known limitations / not-yet-done
- Wandering is a straight walk to a random point — no Shimeji-style physics,
wall-climbing or falling.
- Push-to-talk and the notification bridge are platform-limited: the hotkey
needs a session that allows global key hooks (most Wayland setups don't),
and the notification bridge is Linux/D-Bus only.
- Barge-in listens through the same mic that hears the pet's own voice. It
wants headphones or a decent gap between speaker and mic; if playback
interrupts itself, raise `BARGE_IN_RMS_THRESHOLD` or set `BARGE_IN=false`.
- Screen context is the window *title* only — the desk API takes text, so
there's no screenshot understanding.
This project started as a copy of Bolt-Pet. See that repo for the voice/desktop-pet version.
bolt_console/__init__.py
+3
View File
@@ -0,0 +1,3 @@
"""Bolt Console — a terminal client for Bolt's desk API."""
__version__ = "0.1.0"
bolt_console/__main__.py
+32
View File
@@ -0,0 +1,32 @@
"""Entry point.
python -m bolt_console the window
python -m bolt_console --cli the terminal REPL
python -m bolt_console -p MESSAGE one message in the terminal, then exit
python -m bolt_console --doctor [--deep]
(--auto works with the window and the terminal)
"""
import sys
def main(argv=None):
args = list(sys.argv[1:] if argv is None else argv)
if "--doctor" in args:
from bolt_console.doctor import main as doctor
return doctor(args)
if "--cli" in args or "-p" in args or "-h" in args or "--help" in args:
from bolt_console.console import run
return run([a for a in args if a != "--cli"])
from bolt_console.gui.window import run
return run(args)
if __name__ == "__main__":
sys.exit(main())
bolt_console/chats.py
+53
View File
@@ -0,0 +1,53 @@
"""Chat-list helpers with no Qt, shared by the Chats dialog and the
terminal's /chats: turning what GET /desk/chats returns into readable rows."""
from __future__ import annotations
import re
from datetime import datetime, timezone
from typing import Optional
# Context notes clients append to what you typed. They're in the stored last
# message, but they aren't what you said, so they stay out of the preview.
# The preview may be cut off mid-note, so an unclosed one runs to the end.
_CONTEXT_NOTES = re.compile(r"\s*\[(?:Bolt Console:|on screen right now:)[^\]]*(?:\]|$)", re.IGNORECASE)
def clean_preview(text: str, limit: int = 110) -> str:
text = " ".join(_CONTEXT_NOTES.sub("", str(text or "")).split())
return text if len(text) <= limit else text[: limit - 1].rstrip() + "…"
def relative_time(iso: str, now: Optional[datetime] = None) -> str:
"""'just now', '12 min ago', '3 h ago', 'yesterday', '4 days ago', or a
date. Unparseable input comes back as it was."""
try:
then = datetime.fromisoformat(str(iso).replace("Z", "+00:00"))
except (TypeError, ValueError):
return str(iso or "")
if then.tzinfo is None:
then = then.replace(tzinfo=timezone.utc)
now = now or datetime.now(timezone.utc)
seconds = (now - then).total_seconds()
if seconds < 60:
return "just now"
if seconds < 3600:
return f"{int(seconds // 60)} min ago"
if seconds < 86400:
return f"{int(seconds // 3600)} h ago"
days = int(seconds // 86400)
if days == 1:
return "yesterday"
if days < 7:
return f"{days} days ago"
return then.astimezone().strftime("%d %b %Y").lstrip("0")
def local_time(iso: str) -> str:
try:
then = datetime.fromisoformat(str(iso).replace("Z", "+00:00"))
if then.tzinfo is None:
then = then.replace(tzinfo=timezone.utc)
return then.astimezone().strftime("%A %d %B %Y, %H:%M")
except (TypeError, ValueError):
return str(iso or "")
bolt_console/config.py
+162
View File
@@ -0,0 +1,162 @@
"""Environment configuration for Bolt Console.
Same lightweight ".env next to the script" pattern as desk_client/bolt_desk.py
in the main tmn-api repo, so this project can be copied anywhere (it does not
import anything from that repo) and configured the same way.
"""
from __future__ import annotations
import os
import platform
import sys
from pathlib import Path
if getattr(sys, "frozen", False):
# PyInstaller build: the .env sits next to the .exe, not inside the bundle.
HERE = Path(sys.executable).resolve().parent
else:
HERE = Path(__file__).resolve().parent.parent # project root (one above bolt_console/)
def _load_env() -> None:
env_path = HERE / ".env"
if not env_path.exists():
return
for line in env_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
os.environ.setdefault(key.strip(), value.strip())
_load_env()
def _flag(name: str, default: str) -> bool:
return os.environ.get(name, default).lower() in ("1", "true", "yes", "on")
def _node_name() -> str:
try:
return platform.node() or "desktop"
except Exception:
return "desktop"
# ── server / identity ───────────────────────────────────────────────────────
# The session id keeps the "pet-" prefix on purpose: the desk API advertises
# the `filectl` read/write/edit tool to pet sessions, and this console is
# built around that tool. "-console" keeps it a separate conversation from a
# Bolt-Pet running on the same machine.
SERVER_URL = os.environ.get("BOLT_SERVER_URL", "").rstrip("/")
API_KEY = os.environ.get("DESK_API_KEY", "")
#
# "New chat" (GUI) / /new (terminal) rotates to a fresh id, remembered in
# STATE_FILE so a restart carries on the same conversation. DESK_SESSION_ID in
# .env pins the session and takes precedence over the remembered one.
STATE_FILE = Path.home() / ".cache" / "bolt-console" / "session.json"
_BASE_SESSION = "pet-console-" + _node_name()
def _remembered_session() -> str:
try:
import json
value = json.loads(STATE_FILE.read_text(encoding="utf-8")).get("session_id")
return str(value) if value else ""
except Exception:
return "" # missing or corrupt state means the default, never a crash
SESSION_ID = os.environ.get("DESK_SESSION_ID") or _remembered_session() or _BASE_SESSION
def set_session(session_id: str) -> str:
"""Make *session_id* the current conversation and remember it for the
next start. A plain reassignment of the module attribute every
server_client call reads; not locked against an in-flight request, which
is fine for a rare, deliberate user action (the window blocks switching
while a turn is running anyway)."""
import json
global SESSION_ID
SESSION_ID = session_id
try:
STATE_FILE.parent.mkdir(parents=True, exist_ok=True)
STATE_FILE.write_text(json.dumps({"session_id": SESSION_ID}), encoding="utf-8")
except OSError:
pass # switched for this run; it just won't survive a restart
return SESSION_ID
def new_session() -> str:
"""A locally made fresh id — the fallback when the server can't mint one
(POST /desk/chats is the normal route; see Session.new_conversation)."""
import time
return set_session(f"{_BASE_SESSION}-{time.strftime('%Y%m%d-%H%M%S')}")
def session_pinned() -> bool:
return bool(os.environ.get("DESK_SESSION_ID"))
# ── relayed commands ────────────────────────────────────────────────────────
COMMAND_TIMEOUT_SECONDS = int(os.environ.get("COMMAND_TIMEOUT_SECONDS", "120"))
# What happens when Bolt wants to change something on this machine:
# ask — shell commands and file writes/edits wait for y/n at the prompt
# (reads and directory listings never ask).
# auto — everything runs without asking, like the pet did.
# Toggle for the current session with /auto and /ask.
APPROVAL_MODE = os.environ.get("APPROVAL_MODE", "ask").strip().lower()
# ── replies ─────────────────────────────────────────────────────────────────
# Print each sentence as the server produces it instead of waiting out the
# whole model call. Falls back to the ordinary request/response path when the
# stream fails before anything was printed.
STREAMING_REPLIES = _flag("STREAMING_REPLIES", "true")
# Append a one-line note (working directory, OS) to each message, so "what's
# in this folder?" has a referent and Bolt knows there's no pet body here.
CONSOLE_CONTEXT = _flag("CONSOLE_CONTEXT", "true")
# ANSI colour. Also off when NO_COLOR is set or stdout isn't a terminal.
COLOR = _flag("COLOR", "true") and "NO_COLOR" not in os.environ
# ── file delivery ───────────────────────────────────────────────────────────
# The server's deliver_files tool (ai/desk_api.py in the main tmn-api repo)
# queues files on this session — e.g. "send me that report" — for the client
# to fetch via GET /desk/files. Downloading a file dequeues it server-side, so
# each one lands here exactly once.
RECEIVE_FILES = _flag("RECEIVE_FILES", "true")
DELIVERED_FILES_DIR = Path(
os.environ.get("DELIVERED_FILES_DIR") or str(Path.home() / "Downloads" / "Bolt")
).expanduser()
# ── dictation (the mic button) ──────────────────────────────────────────────
# Speech goes to the server's /desk/stt relay, like Bolt-Pet. MIC_DEVICE is an
# input index or name substring (blank = system default); a recording stops
# itself after DICTATION_SILENCE_SECONDS of quiet once you've said something.
MIC_DEVICE = os.environ.get("MIC_DEVICE", "")
DICTATION_SILENCE_SECONDS = float(os.environ.get("DICTATION_SILENCE_SECONDS", "3"))
def is_configured() -> bool:
return bool(SERVER_URL and API_KEY)
def missing_config() -> list:
missing = []
if not SERVER_URL:
missing.append("BOLT_SERVER_URL")
if not API_KEY:
missing.append("DESK_API_KEY")
return missing
bolt_console/console.py
+398
View File
@@ -0,0 +1,398 @@
"""The REPL: read a line, send it to Bolt, show the tool calls and the reply.
`Session` holds everything a turn needs and takes its collaborators as
arguments (the terminal, the tool dispatcher, the two converse functions), so
tests drive whole turns without a real server or a real terminal.
"""
from __future__ import annotations
import os
import platform
import re
import subprocess
import sys
from pathlib import Path
from typing import Callable, Optional
from . import __version__, config, file_delivery, server_client
from .chats import clean_preview, relative_time
from .tools import Tools
from .ui import Terminal, enable_windows_ansi
HELP = """\
Commands
/help this list
/exit, /quit leave (Ctrl+D / Ctrl+Z also works)
/clear clear the screen (Bolt's memory lives on the server and is kept)
/cd <dir> change the working directory commands and relative paths use
/pwd show the working directory
/ask ask before shell commands, writes and edits (default)
/auto run everything without asking
/status server, session and settings
/files download anything Bolt has queued for you now
/new start a fresh chat (new history; Bolt still remembers you)
/chats list your chats with Bolt, from every device
/chat <n|id> switch to chat number n from /chats, or a session id
!<command> run a shell command yourself, without sending it to Bolt
End a line with \\ to keep typing on the next one."""
def _os_label() -> str:
try:
return f"{platform.system()} {platform.release()}".strip() or "unknown OS"
except Exception:
return "unknown OS"
def annotate(text: str, cwd: str, os_label: str) -> str:
"""The context note appended to each message: where "here" is, and that
this client is text-only."""
# The exact marker syntax matters. Measured against the live server
# (2026-09-25): "use filectl and shell commands" made the model write
# `filectl: {...}` or a <tool_call> block as prose and then invent the
# output, on both /desk/converse and /desk/converse_stream; spelling out
# `command: ...` got a real relayed command on both.
return (
f"{text}\n\n[Bolt Console: a text terminal on {os_label}; working directory {cwd}. "
"To act on this machine, use your command tool: "
'`command: filectl {"op": "list", "path": "..."}` for files (ops: list, read, write, edit), '
"or `command: <shell command>` for anything else. The console runs it and sends back "
"the real output. Never write a command's output yourself. "
"There is no pet body, voice or screen here.]"
)
# What a reply looks like when it narrates tool output instead of calling a
# tool: the model copies the shape of real results it has seen earlier in the
# conversation ("→ Command result: [filectl] wrote 1200 chars to …"). Seen
# live: three turns in a row of "wrote the files" with no command ever sent.
_FAKE_RESULT = re.compile(
r"(?:→|->)\s*(?:command|deliver files|tool)\s+result\s*:"
r"|\[filectl\]\s+(?:wrote|edited)\b"
r"|\[exit \d+\]",
re.IGNORECASE,
)
FABRICATION_WARNING = (
"Bolt described running commands, but none reached this app, so nothing was run "
"or written. The results in that reply are made up. Bolt will be told on your next message."
)
CORRECTION = (
"[Bolt Console: correction. Your previous reply described command results, but no "
"command reached this machine, so nothing ran and no files were written. Never write "
"results yourself; emit the actual command and wait for its real output.]"
)
def fabricated(text: str, commands_run: int) -> bool:
"""A reply that reads like tool output when no tool ran this turn."""
return commands_run == 0 and bool(_FAKE_RESULT.search(text or ""))
class Session:
def __init__(
self,
ui: Terminal,
tools: Optional[Tools] = None,
converse: Callable = server_client.converse,
converse_stream: Optional[Callable] = server_client.converse_stream,
list_files: Callable = server_client.list_outbox_files,
download_file: Callable = server_client.download_outbox_file,
create_chat: Callable = server_client.create_chat,
list_chats: Callable = server_client.list_chats,
):
self.ui = ui
self.tools = tools or Tools(ui, mode=config.APPROVAL_MODE)
self._converse = converse
self._converse_stream = converse_stream if config.STREAMING_REPLIES else None
self._list_files = list_files
self._download_file = download_file
self._create_chat = create_chat
self._list_chats = list_chats
self._last_chats = [] # what /chats showed, so /chat 3 means that row
self._said = [] # sentences shown this turn
self._correct_next = False # tell Bolt his last reply was fiction
# ── one turn ─────────────────────────────────────────────────────────
def send(self, text: str) -> None:
message = annotate(text, os.getcwd(), _os_label()) if config.CONSOLE_CONTEXT else text
if self._correct_next:
message = f"{CORRECTION}\n\n{message}"
self._correct_next = False
self._said = []
calls_before = self.tools.calls
self.ui.start_spinner()
try:
reply = self._ask(message)
except server_client.ServerError as exc:
self.ui.stop_spinner()
self.ui.error(f"Error: {exc}")
return
except KeyboardInterrupt:
self.ui.stop_spinner()
self.ui.warn("Interrupted. (The server may still finish that turn on its side.)")
return
self.ui.stop_spinner()
if reply.streamed:
self.ui.end_reply()
else:
self.ui.assistant(reply.text)
shown = " ".join(self._said + [reply.text])
if fabricated(shown, self.tools.calls - calls_before):
self.ui.warn(FABRICATION_WARNING)
self._correct_next = True
self.check_deliveries()
def _ask(self, message: str) -> server_client.Reply:
if self._converse_stream is not None:
try:
return self._converse_stream(message, on_say=self._say, on_command=self.tools)
except server_client.ServerError:
pass # nothing was shown yet; the plain path is invisible to the user
return self._converse(message, on_command=self.tools, on_say=self._holding)
def _say(self, sentence: str) -> None:
self._said.append(sentence)
self.ui.assistant_chunk(sentence)
def _holding(self, line: str) -> None:
self._said.append(line)
self.ui.holding(line)
# ── delivered files ──────────────────────────────────────────────────
def check_deliveries(self, announce_empty: bool = False) -> None:
if not config.RECEIVE_FILES:
if announce_empty:
self.ui.info("File delivery is off (RECEIVE_FILES=false).")
return
try:
queued = self._list_files()
except server_client.ServerError as exc:
self.ui.warn(f"Couldn't check for delivered files: {exc}")
return
if not queued and announce_empty:
self.ui.info("Nothing queued.")
for entry in queued:
file_id, name = str(entry.get("id") or ""), str(entry.get("name") or "")
if not file_id:
continue
try:
data = self._download_file(file_id)
path = file_delivery.save(config.DELIVERED_FILES_DIR, name, data)
except (server_client.ServerError, OSError) as exc:
self.ui.warn(f"Couldn't save delivered file {name!r}: {exc}")
continue
self.ui.line(self.ui.style(f"Received {name} -> {path}", "green"))
# ── input dispatch ───────────────────────────────────────────────────
def handle(self, line: str) -> bool:
"""Act on one line of input. Returns False when the user asked to leave."""
text = line.strip()
if not text:
return True
if text.startswith("!"):
self._local_shell(text[1:].strip())
return True
if text.startswith("/"):
return self._slash(text)
self.ui.line()
self.send(text)
return True
def _local_shell(self, command: str) -> None:
if not command:
return
# The user's own command: live output, their terminal, nothing captured
# and nothing sent to Bolt.
try:
code = subprocess.run(command, shell=True, cwd=os.getcwd()).returncode
except KeyboardInterrupt:
self.ui.warn("Interrupted.")
return
if code:
self.ui.info(f"[exit {code}]")
def _slash(self, text: str) -> bool:
name, _, arg = text[1:].partition(" ")
name, arg = name.lower(), arg.strip()
if name in ("exit", "quit", "q"):
return False
if name == "help":
self.ui.line(HELP)
elif name == "clear":
self.ui.line("\033[2J\033[H" if self.ui.color else "\n" * 3)
elif name == "cd":
self._cd(arg)
elif name == "pwd":
self.ui.line(os.getcwd())
elif name in ("auto", "ask"):
self.tools.mode = name
self.ui.info(
"Auto mode: commands, writes and edits run without asking."
if name == "auto" else "Ask mode: commands, writes and edits need your OK."
)
elif name == "status":
self.status()
elif name == "new":
self.ui.info(self.new_conversation())
elif name == "chats":
self.show_chats()
elif name == "chat":
self.pick_chat(arg)
elif name == "files":
self.check_deliveries(announce_empty=True)
else:
self.ui.warn(f"Unknown command /{name} - try /help.")
return True
def switch_to(self, session_id: str) -> str:
"""Make *session_id* the current chat (new or picked from the list).
Server-side history follows the id; Bolt's memory is per user, so it
carries over either way. Returns a one-line note for the UI."""
config.set_session(session_id)
self._correct_next = False # a correction belongs to the chat it was about
note = f"Chat: {session_id}"
if config.session_pinned():
note += " (DESK_SESSION_ID in .env pins a different one after a restart)"
return note
def show_chats(self) -> None:
try:
self._last_chats = self._list_chats()
except server_client.ServerError as exc:
self.ui.error(f"Error: {exc}")
return
if not self._last_chats:
self.ui.info("No chats yet.")
return
for number, chat in enumerate(self._last_chats, 1):
session_id = str(chat.get("session_id") or "")
mark = self.ui.style("*", "green") if session_id == config.SESSION_ID else " "
label = str(chat.get("label") or session_id)
when = relative_time(chat.get("last_active", ""))
self.ui.line(f"{mark}{number:>3} {self.ui.style(label, 'bold')} {self.ui.style(when, 'dim')}")
preview = clean_preview(chat.get("last_message", ""), 90)
if preview:
self.ui.line(" " + self.ui.style(preview, "dim"))
self.ui.info("/chat <number> to switch, /new for a fresh one")
def pick_chat(self, arg: str) -> None:
arg = arg.strip()
if not arg:
self.ui.warn("Usage: /chat <number from /chats> or /chat <session id>")
return
if arg.isdigit():
index = int(arg) - 1
if not 0 <= index < len(self._last_chats):
self.ui.warn("No such number; run /chats first.")
return
chat = self._last_chats[index]
session_id, label = str(chat.get("session_id")), str(chat.get("label") or "")
else:
session_id, label = arg, arg
self.ui.info(f"{self.switch_to(session_id)} ({label})")
def new_conversation(self) -> str:
"""A fresh chat: empty history, so nothing said earlier (including
invented tool results) is there to be copied. The server mints the id
(POST /desk/chats); if it can't, a local id still gives a clean slate.
Blocking — the window runs create_chat on a worker and calls
switch_to() itself instead."""
try:
return "New chat. " + self.switch_to(self._create_chat())
except server_client.ServerError as exc:
note = self.switch_to(config.new_session())
return f"New chat (made locally; the server couldn't: {exc}). {note}"
def _cd(self, arg: str) -> None:
target = Path(arg or str(Path.home())).expanduser()
try:
os.chdir(target)
except OSError as exc:
self.ui.error(f"cd: {exc}")
return
self.ui.info(os.getcwd())
def status(self) -> None:
rows = [
("server", config.SERVER_URL or "(not set)"),
("session", config.SESSION_ID),
("approval", self.tools.mode),
("cwd", os.getcwd()),
("streaming", "on" if self._converse_stream else "off"),
("deliveries", str(config.DELIVERED_FILES_DIR) if config.RECEIVE_FILES else "off"),
]
for key, value in rows:
self.ui.line(f" {self.ui.style(key.ljust(11), 'dim')}{value}")
# ── the loop ─────────────────────────────────────────────────────────
def banner(self) -> None:
ui = self.ui
ui.line(ui.style(f"{ui.glyphs.bullet} Bolt Console", "bold", "cyan") + ui.style(f" v{__version__}", "dim"))
ui.info(f" {config.SERVER_URL} | session {config.SESSION_ID} | approval: {self.tools.mode}")
ui.info(f" cwd: {os.getcwd()}")
ui.info(" /help for commands, /exit to leave")
ui.line()
def loop(self) -> int:
self.banner()
while True:
try:
line = self.ui.read_message()
except EOFError:
self.ui.line()
return 0
except KeyboardInterrupt:
# Ctrl+C at an empty prompt clears the line, like a shell.
self.ui.line()
continue
if not self.handle(line):
return 0
def _usage() -> str:
return (
"usage: python -m bolt_console [--auto] [-p MESSAGE] [--doctor [--deep]]\n"
" -p MESSAGE send one message, print the reply, exit\n"
" --auto don't ask before commands, writes and edits\n"
" --doctor check the install (--deep also contacts the server)"
)
def run(argv: Optional[list] = None) -> int:
argv = list(sys.argv[1:] if argv is None else argv)
if "-h" in argv or "--help" in argv:
print(_usage())
return 0
try:
sys.stdout.reconfigure(errors="replace")
except Exception:
pass
enable_windows_ansi()
ui = Terminal(color=config.COLOR)
if not config.is_configured():
ui.error("Missing " + ", ".join(config.missing_config()) + ". Copy .env.example to .env and fill them in.")
return 2
mode = "auto" if "--auto" in argv else config.APPROVAL_MODE
session = Session(ui, tools=Tools(ui, mode=mode))
if "-p" in argv:
index = argv.index("-p")
message = " ".join(argv[index + 1:]).strip()
if not message:
ui.error("-p needs a message")
return 2
session.send(message)
return 0
return session.loop()
bolt_console/dictation.py
+339
View File
@@ -0,0 +1,339 @@
"""Dictation: speak into the mic, get text in the composer.
Same route as Bolt-Pet: 16 kHz mono PCM streamed over a websocket to the Bolt
server's own `/desk/stt` relay, which forwards it to Deepgram and sends
Deepgram's JSON back untouched. No Deepgram account or key here — auth is the
desk API key — though the server does meter it against the same credits as a
chat turn.
One `Dictation` is one recording:
d = Dictation(on_update, on_level, on_done)
d.start() # mic opens, audio streams up, on_update(final, interim) as words arrive
d.stop() # or silence / the length cap stops it; on_done(text, error) follows
d.cancel() # stop and throw the result away
Design points:
- **Live, with a safety net.** Audio goes up as it's captured so the words
appear while you talk. Every frame is also buffered, and if the socket
never opened or died, stop() opens a fresh one and sends the whole buffer —
Bolt-Pet's one-shot fallback.
- **You decide when it's done**, with help: a click stops it, and so do
`silence_seconds` of quiet after you've said something, or `max_seconds`.
- **No Qt, no numpy.** Callbacks run on background threads; the window turns
them into signals. Audio is raw int16 bytes, and loudness is computed with
the array module. That keeps this identical in the single-file Linux app.
- **Everything with hardware or network is injected** (`connect`,
`open_stream`), so the whole flow is tested without a mic or a server.
"""
from __future__ import annotations
import array
import json
import math
import queue
import sys
import threading
import time
import urllib.parse
from typing import Callable, List, Optional
from . import config
SAMPLE_RATE = 16000
BLOCK_FRAMES = 1280 # 80 ms, same chunking as Bolt-Pet
SPEECH_RMS = 500 # int16 RMS above this counts as talking
def unavailable_reason() -> Optional[str]:
"""None if dictation can work here, otherwise a sentence saying why —
shown as the mic button's tooltip instead of a crash."""
if not config.is_configured():
return "Set BOLT_SERVER_URL and DESK_API_KEY to use dictation"
try:
import websocket # noqa: F401 (websocket-client)
except Exception:
return "Dictation needs websocket-client: pip install websocket-client"
try:
import sounddevice # noqa: F401
except OSError:
return ("Dictation needs the PortAudio library (e.g. sudo apt install libportaudio2)"
if sys.platform.startswith("linux") else "PortAudio isn't available on this system")
except Exception:
return "Dictation needs sounddevice: pip install sounddevice"
return None
def mic_device():
"""MIC_DEVICE from .env: an index (as saved by Bolt-Pet's mic picker) or a
name substring; blank means the system default."""
raw = str(getattr(config, "MIC_DEVICE", "") or "").strip()
if not raw:
return None
return int(raw) if raw.lstrip("-").isdigit() else raw
def connect(sample_rate: int = SAMPLE_RATE):
"""Open the server's /desk/stt websocket for one recording."""
import websocket
query = urllib.parse.urlencode({
"session_id": config.SESSION_ID, "encoding": "linear16", "sample_rate": sample_rate,
})
url = config.SERVER_URL.replace("http", "ws", 1) + "/desk/stt?" + query
return websocket.create_connection(url, header=[f"X-Desk-Api-Key: {config.API_KEY}"], timeout=10)
def open_stream(callback: Callable[[bytes], None], sample_rate: int = SAMPLE_RATE):
"""Start the default (or MIC_DEVICE) microphone, handing raw int16 mono
bytes to *callback* from PortAudio's thread. Returns something with
stop() and close()."""
import sounddevice as sd
stream = sd.RawInputStream(
samplerate=sample_rate, channels=1, dtype="int16", blocksize=BLOCK_FRAMES,
device=mic_device(), callback=lambda data, _frames, _time, _status: callback(bytes(data)),
)
stream.start()
return stream
def rms(chunk: bytes) -> float:
samples = array.array("h")
samples.frombytes(chunk[: len(chunk) - len(chunk) % 2])
if not samples:
return 0.0
return math.sqrt(sum(s * s for s in samples) / len(samples))
def parse_result(message) -> tuple:
"""(text, is_final) from a Deepgram results frame; ("", False) for
anything else. Never raises — it runs on the reader thread."""
try:
if isinstance(message, bytes):
message = message.decode("utf-8", "ignore")
data = json.loads(message)
except (TypeError, ValueError):
return "", False
if not isinstance(data, dict) or data.get("type") not in (None, "Results"):
return "", False
alternatives = (data.get("channel") or {}).get("alternatives") or []
if not alternatives:
return "", False
text = str((alternatives[0] or {}).get("transcript") or "").strip()
return text, bool(data.get("is_final") or data.get("speech_final"))
class Dictation:
def __init__(
self,
on_update: Callable[[str, str], None],
on_level: Callable[[float], None],
on_done: Callable[[str, Optional[str]], None],
*,
connect_fn: Callable = None,
open_stream_fn: Callable = None,
silence_seconds: float = 3.0,
max_seconds: float = 300.0,
sample_rate: int = SAMPLE_RATE,
on_stopping: Callable[[], None] = None,
):
self._on_update, self._on_level, self._on_done = on_update, on_level, on_done
# Recording over (click, silence or cap), transcript not back yet.
self._on_stopping = on_stopping or (lambda: None)
self._connect = connect_fn or connect
self._open_stream = open_stream_fn or open_stream
self._silence_seconds = silence_seconds
self._max_seconds = max_seconds
self._rate = sample_rate
self._frames: "queue.Queue[bytes]" = queue.Queue()
self._buffer: List[bytes] = []
self._finals: List[str] = []
self._interim = ""
self._lock = threading.Lock()
self._stop = threading.Event()
self._cancelled = False
self._socket = None
self._socket_ok = False
self._stream = None
self._reader = None
self._pump = None
# ── control (any thread) ─────────────────────────────────────────────
def start(self) -> None:
"""Open the mic (raises if that fails — nothing to record without it)
and try the live socket (failure there is fine: the fallback has the
audio)."""
try:
self._socket = self._connect(self._rate)
self._socket_ok = True
self._reader = threading.Thread(target=self._read_loop, name="stt-reader", daemon=True)
self._reader.start()
except Exception:
self._socket, self._socket_ok = None, False
try:
self._stream = self._open_stream(self._frames.put, self._rate)
except Exception:
self._close_socket()
raise
self._pump = threading.Thread(target=self._pump_loop, name="stt-pump", daemon=True)
self._pump.start()
def stop(self) -> None:
self._stop.set()
def cancel(self) -> None:
self._cancelled = True
self._stop.set()
@property
def live(self) -> bool:
"""Whether words are arriving as you speak (vs. only after stop)."""
return self._socket_ok
# ── the pump: mic frames -> socket, buffer, level, auto-stop ─────────
def _pump_loop(self) -> None:
started = time.monotonic()
heard_speech, last_speech = False, started
while not self._stop.is_set():
try:
chunk = self._frames.get(timeout=0.1)
except queue.Empty:
chunk = None
now = time.monotonic()
if chunk:
self._buffer.append(chunk)
self._send(chunk)
loudness = rms(chunk)
self._safe(self._on_level, min(1.0, math.sqrt(loudness / 8000.0)))
if loudness >= SPEECH_RMS:
heard_speech, last_speech = True, now
if heard_speech and now - last_speech >= self._silence_seconds:
break
if now - started >= self._max_seconds:
break
self._finish()
def _send(self, chunk: bytes) -> None:
if not self._socket_ok:
return
try:
self._socket.send_binary(chunk)
except Exception:
self._socket_ok = False # the fallback will send the buffer instead
def _finish(self) -> None:
self._stop.set()
self._safe(self._on_stopping)
for step in ("stop", "close"):
try:
getattr(self._stream, step)()
except Exception:
pass
self._safe(self._on_level, 0.0)
if self._cancelled:
self._close_socket()
self._safe(self._on_done, "", None)
return
# Whatever was still queued when we stopped belongs to the recording.
while True:
try:
chunk = self._frames.get_nowait()
except queue.Empty:
break
self._buffer.append(chunk)
self._send(chunk)
# Live socket healthy to the end: its transcript is the answer, even
# an empty one (silence) — re-sending would pay to transcribe it twice.
error = None
if self._socket_ok:
self._flush_socket()
else:
error = self._fallback()
text = self._text()
if not text and error is None:
error = "Didn't catch anything — try again a little closer to the mic."
self._safe(self._on_done, text, error)
def _flush_socket(self) -> None:
"""Deepgram flushes its last results after CloseStream; wait for them,
bounded so a hung socket can't hold the text hostage."""
try:
self._socket.send(json.dumps({"type": "CloseStream"}))
except Exception:
self._socket_ok = False
if self._reader is not None:
self._reader.join(timeout=4)
self._close_socket()
def _fallback(self) -> Optional[str]:
"""The live socket never worked or died part-way: one fresh connection,
the whole buffer, from an empty transcript (a half-live session's words
would otherwise come back twice)."""
self._close_socket()
if self._reader is not None:
self._reader.join(timeout=1)
with self._lock:
self._finals, self._interim = [], ""
if not self._buffer:
return None
try:
self._socket = self._connect(self._rate)
except Exception as exc:
return f"Couldn't reach the transcription server: {exc}"
self._socket_ok = True
self._reader = threading.Thread(target=self._read_loop, name="stt-reader", daemon=True)
self._reader.start()
audio = b"".join(self._buffer)
for offset in range(0, len(audio), BLOCK_FRAMES * 2 * 8):
self._send(audio[offset:offset + BLOCK_FRAMES * 2 * 8])
self._flush_socket()
return None
def _close_socket(self) -> None:
socket, self._socket = self._socket, None
if socket is not None:
try:
socket.close()
except Exception:
pass
# ── the reader: transcripts back ─────────────────────────────────────
def _read_loop(self) -> None:
socket = self._socket
while socket is not None:
try:
message = socket.recv()
except Exception:
break
if not message:
break
text, is_final = parse_result(message)
with self._lock:
if is_final:
if text:
self._finals.append(text)
self._interim = ""
elif text:
self._interim = text
finals, interim = " ".join(self._finals), self._interim
if text:
self._safe(self._on_update, finals, interim)
def _text(self) -> str:
with self._lock:
# An interim that never became final is still better than nothing.
return " ".join(self._finals + ([self._interim] if self._interim else [])).strip()
@staticmethod
def _safe(callback, *args) -> None:
try:
callback(*args)
except Exception:
pass
bolt_console/doctor.py
+163
View File
@@ -0,0 +1,163 @@
"""`python -m bolt_console --doctor` — is this install actually going to work?
Carried over from Bolt-Pet, where a missing port publish, a wrong reverse-proxy
header and a venv whose python was a zero-byte file all presented as "it's
being weird" and each took a conversation to find.
Each check is independent and reports one of three things — ok, a warning
(works, but degraded), or a failure (this will not do what you expect) — and
says *what to do about it* rather than just what is wrong. Nothing here raises:
a doctor that crashes on a broken install is diagnosing the wrong patient.
Deliberately does not touch the network by default — it's the first thing you
reach for when the network is what's broken. `--deep` contacts the server.
"""
from __future__ import annotations
import sys
from dataclasses import dataclass
from typing import Callable, Optional
from . import config
OK, WARN, FAIL = "ok", "warn", "fail"
_MARKS = {OK: " ok ", WARN: " warn ", FAIL: " FAIL "}
@dataclass
class Check:
name: str
status: str
detail: str = ""
fix: str = ""
def line(self) -> str:
text = f"[{_MARKS[self.status]}] {self.name:22} {self.detail}"
if self.fix and self.status != OK:
text += f"\n{'':32}→ {self.fix}"
return text
# ── the checks ──────────────────────────────────────────────────────────────
def check_config() -> Check:
missing = config.missing_config()
if missing:
return Check("server config", FAIL, f"missing {', '.join(missing)}",
"set them in .env — without these the console exits at startup")
return Check("server config", OK, f"{config.SERVER_URL} as {config.SESSION_ID}")
def check_server(deep: bool = False) -> Check:
if not config.SERVER_URL:
return Check("server", FAIL, "no BOLT_SERVER_URL",
"cp .env.example .env and set BOLT_SERVER_URL to your Bolt server")
if not deep:
return Check("server", OK, f"{config.SERVER_URL} (not contacted; --deep to try)")
try:
from . import server_client
health = server_client.check_health(timeout=8)
return Check("server", OK, f"reachable — {health}")
except Exception as exc:
return Check("server", FAIL, f"unreachable: {exc}",
"check the URL, the key, and that the container is up")
def check_streaming_endpoint(deep: bool = False) -> Check:
"""The streamed-reply endpoint is newer than some deployed servers."""
if not config.STREAMING_REPLIES:
return Check("streamed replies", WARN, "disabled (STREAMING_REPLIES=false)")
if not deep:
return Check("streamed replies", OK, "enabled (endpoint not probed)")
try:
import requests
response = requests.post(
f"{config.SERVER_URL}/desk/converse_stream",
json={"session_id": config.SESSION_ID, "text": ""},
headers={"X-Desk-Api-Key": config.API_KEY}, timeout=8, stream=True,
)
if response.status_code == 404:
return Check("streamed replies", WARN, "server has no /desk/converse_stream",
"update the server, or set STREAMING_REPLIES=false to skip the probe")
return Check("streamed replies", OK, f"endpoint answered {response.status_code}")
except Exception as exc:
return Check("streamed replies", WARN, f"probe failed: {exc}")
def check_python() -> Check:
version = ".".join(str(n) for n in sys.version_info[:3])
if sys.version_info < (3, 9):
return Check("python", FAIL, version, "Python 3.9 or newer is required")
return Check("python", OK, f"{version} at {sys.executable}")
def check_approval() -> Check:
mode = config.APPROVAL_MODE
if mode == "auto":
return Check("approval", WARN, "auto — Bolt's commands, writes and edits run without asking",
"APPROVAL_MODE=ask to confirm each one (or /ask for one session)")
if mode != "ask":
return Check("approval", WARN, f"unknown APPROVAL_MODE={mode!r}, treated as ask",
"use ask or auto")
return Check("approval", OK, "ask before commands, writes and edits")
def check_deliveries() -> Check:
if not config.RECEIVE_FILES:
return Check("file delivery", WARN, "disabled (RECEIVE_FILES=false)")
directory = config.DELIVERED_FILES_DIR
probe = directory
while not probe.exists() and probe.parent != probe:
probe = probe.parent
import os
if not os.access(str(probe), os.W_OK):
return Check("file delivery", FAIL, f"can't write under {probe}",
"set DELIVERED_FILES_DIR to a folder you can write to")
return Check("file delivery", OK, str(directory))
CHECKS = (
("python", check_python),
("config", check_config),
("server", check_server),
("stream", check_streaming_endpoint),
("approval", check_approval),
("deliveries", check_deliveries),
)
def run(deep: bool = False) -> list:
results = []
for _name, check in CHECKS:
try:
try:
results.append(check(deep))
except TypeError:
results.append(check())
except Exception as exc: # a broken check must not hide the others
results.append(Check(_name, FAIL, f"the check itself failed: {exc}"))
return results
def main(argv: Optional[list] = None) -> int:
argv = list(argv if argv is not None else sys.argv[1:])
deep = "--deep" in argv
print(f"Bolt Console preflight{' (deep: contacting the server)' if deep else ''}\n")
results = run(deep=deep)
for check in results:
print(check.line())
failures = [c for c in results if c.status == FAIL]
warnings = [c for c in results if c.status == WARN]
print()
if failures:
print(f"{len(failures)} problem(s) will stop this working. Fix those first.")
elif warnings:
print(f"Ready. {len(warnings)} thing(s) degraded but working.")
else:
print("Everything checks out.")
return 1 if failures else 0
bolt_pet/file_delivery.py → bolt_console/file_delivery.py
View File
bolt_pet/file_ops.py → bolt_console/file_ops.py
+5 -8
View File
@@ -1,6 +1,5 @@
"""Local file read/edit/write, intercepted from the server-relayed command
channel the same way pet_actions.py intercepts petctl (see server_client.
run_local_command / controller._handle_command). Whatever text the server's
channel before it reaches a shell (see tools.Tools.handle). Whatever text the server's
"command" tool sends is just a string this repo is free to interpret before
it ever reaches subprocess — a `filectl` pseudo-command is one such
interpretation, giving the model a way to read/write/edit files on this
@@ -12,7 +11,7 @@ run_local_command/subprocess.run does) — filectl doesn't add that ability,
it only makes the read/write/edit slice of it reliable. It also doesn't
expand what the server can already do to this machine: a relayed shell
command could already overwrite any file the desktop user can write (see the
security notes in CLAUDE.md) — filectl is a safer *path* to the same
security notes in the README) — filectl is a safer *path* to the same
capability, not a new capability.
Wire format: `filectl <json>`, where <json> is a single-line, compact JSON
@@ -42,8 +41,7 @@ _system_context for the exact instruction the model is given, including the
"keep it one line" requirement.
Pure parsing (parse) is separated from the filesystem I/O (execute) so the
syntax is unit-testable without touching disk, matching pet_actions.py's
parse/describe split.
syntax is unit-testable without touching disk.
"""
from __future__ import annotations
@@ -73,7 +71,7 @@ HELP = (
class FileOpError(Exception):
"""Bad filectl syntax or a filesystem error — reported back to the
server as command output, exactly like ActionError in pet_actions.py."""
server as command output."""
def is_file_command(command: str) -> bool:
@@ -167,8 +165,7 @@ def _line_number(value, label: str) -> Optional[int]:
def describe(action: dict) -> str:
"""Text handed back to the server before execute() runs — mirrors
pet_actions.describe, used only for the log line."""
"""A one-line summary of the action, for logs."""
kind = action.get("action")
if kind == "help":
return "[filectl] help"
bolt_console/gui/__init__.py
+2
View File
@@ -0,0 +1,2 @@
"""PySide6 front end. Imported only when the window is wanted, so the
terminal client works without Qt installed."""
bolt_console/gui/bridge.py
+106
View File
@@ -0,0 +1,106 @@
"""The seam between a turn running on a worker thread and the window.
`Session` and `Tools` were written against a terminal-shaped UI (tool_call,
tool_result, confirm, assistant_chunk, …). `Bridge` is that same shape, but
every call becomes a Qt signal — delivered to the window on the GUI thread
through a queued connection — so the whole turn logic is shared, unchanged,
between the terminal and the window.
The one call that has to *return* something is confirm(): the worker blocks on
an Event until the user clicks Allow/Deny in the window, which calls answer()
from the GUI thread. close() releases a pending wait so the app can quit mid
approval without hanging the worker.
"""
from __future__ import annotations
import threading
from PySide6.QtCore import QObject, Signal
class Bridge(QObject):
busy = Signal(bool)
chunk = Signal(str) # one streamed sentence
reply_ended = Signal() # the streamed block is complete
reply = Signal(str) # a whole (non-streamed) reply
holding_line = Signal(str) # "give me a sec" while a tool runs
tool_started = Signal(int, str, str) # id, name, argument
tool_output = Signal(int, str, bool) # id, text, is_error
tool_diff = Signal(int, str, str) # id, old, new
approval_needed = Signal(int, str) # id, question
notice = Signal(str, str) # level (info/warn/error), text
def __init__(self, parent=None):
super().__init__(parent)
self._tool_id = 0
self._answer = "no"
self._answered = threading.Event()
self._closed = False
# ── the terminal-shaped surface Session and Tools call ───────────────
def start_spinner(self, label: str = "Thinking") -> None:
self.busy.emit(True)
def stop_spinner(self) -> None:
self.busy.emit(False)
def assistant_chunk(self, sentence: str) -> None:
if sentence.strip():
self.chunk.emit(sentence.strip())
def end_reply(self) -> None:
self.reply_ended.emit()
def assistant(self, text: str) -> None:
if text.strip():
self.reply.emit(text.strip())
def holding(self, text: str) -> None:
self.holding_line.emit(text.strip())
def tool_call(self, name: str, argument: str) -> None:
self._tool_id += 1
self.tool_started.emit(self._tool_id, name, argument)
def tool_result(self, text: str, error: bool = False) -> None:
self.tool_output.emit(self._tool_id, text, error)
def diff(self, old: str, new: str) -> None:
self.tool_diff.emit(self._tool_id, old, new)
def confirm(self, question: str) -> str:
if self._closed:
return "no"
self._answered.clear()
self._answer = "no"
self.approval_needed.emit(self._tool_id, question)
self._answered.wait()
return self._answer
def info(self, text: str) -> None:
self.notice.emit("info", text)
def warn(self, text: str) -> None:
self.notice.emit("warn", text)
def error(self, text: str) -> None:
self.notice.emit("error", text)
def line(self, text: str = "") -> None:
if text.strip():
self.notice.emit("info", text.strip())
def style(self, text: str, *_names: str) -> str:
return text
# ── called from the GUI thread ───────────────────────────────────────
def answer(self, value: str) -> None:
self._answer = value if value in ("yes", "no", "always") else "no"
self._answered.set()
def close(self) -> None:
self._closed = True
self.answer("no")
bolt_console/gui/chat_list_window.py
+235
View File
@@ -0,0 +1,235 @@
"""The Chats dialog: every conversation this user has with Bolt, across the
console, the pet and the phone, newest first, and a New Chat button.
Network calls never run on the UI thread. Each one is a `ChatTask` on the
global QThreadPool that calls server_client synchronously and hands the
result back through `ChatSignals`, the same signal-based hand-off the turn
worker uses. The list is fetched when the dialog opens and on Refresh; there
is no polling.
Picking or creating a chat doesn't switch anything here. The dialog emits
`chosen(session_id, label)` and the window does the switch, because the
window owns the transcript that has to be cleared along with it.
"""
from __future__ import annotations
from typing import Callable, Optional
from PySide6.QtCore import QObject, QRunnable, QSize, Qt, QThreadPool, Signal
from PySide6.QtWidgets import (
QDialog, QHBoxLayout, QLabel, QListWidget, QListWidgetItem, QPushButton,
QVBoxLayout, QWidget,
)
from .. import config, server_client
from ..chats import clean_preview, local_time, relative_time
from . import icons, theme
# ── off-thread calls ────────────────────────────────────────────────────────
class ChatSignals(QObject):
loaded = Signal(list) # chats, newest first
created = Signal(str) # the new session_id
failed = Signal(str, str) # operation ("list" / "create"), message
class ChatTask(QRunnable):
"""Runs one server_client call on the thread pool and reports back."""
def __init__(self, operation: str, call: Callable, signals: ChatSignals):
super().__init__()
self.operation, self.call, self.signals = operation, call, signals
def run(self) -> None:
try:
result = self.call()
except Exception as exc: # ServerError, or anything unexpected
self.signals.failed.emit(self.operation, str(exc))
return
if self.operation == "list":
self.signals.loaded.emit(list(result))
else:
self.signals.created.emit(str(result))
def run_in_pool(operation: str, call: Callable, signals: ChatSignals) -> None:
QThreadPool.globalInstance().start(ChatTask(operation, call, signals))
# ── the dialog ──────────────────────────────────────────────────────────────
class ChatRow(QWidget):
"""Label and time on the first line, the last message under it."""
def __init__(self, chat: dict, current: bool, parent=None):
super().__init__(parent)
outer = QVBoxLayout(self)
outer.setContentsMargins(14, 9, 14, 9)
outer.setSpacing(2)
top = QHBoxLayout()
top.setSpacing(8)
title = QLabel(str(chat.get("label") or chat.get("session_id") or "Untitled"))
title.setObjectName("ChatTitle")
top.addWidget(title, 1)
if current:
badge = QLabel("Current")
badge.setObjectName("Pill")
badge.setProperty("kind", "waiting")
top.addWidget(badge)
when = QLabel(relative_time(chat.get("last_active", "")))
when.setObjectName("ChatTime")
top.addWidget(when)
outer.addLayout(top)
preview = QLabel(clean_preview(chat.get("last_message", "")) or "—")
preview.setObjectName("ChatPreview")
outer.addWidget(preview)
for widget in (title, when, preview):
widget.setAttribute(Qt.WA_TransparentForMouseEvents)
class ChatListDialog(QDialog):
"""`chosen(session_id, label)` fires for a picked chat and for a newly
created one (label "New chat"); the window does the actual switch."""
chosen = Signal(str, str)
def __init__(self, parent=None, list_chats: Callable = None, create_chat: Callable = None):
super().__init__(parent)
self.setObjectName("ChatDialog")
self.setWindowTitle("Chats")
self.setModal(True)
self.resize(560, 580)
self._list_chats = list_chats or server_client.list_chats
self._create_chat = create_chat or server_client.create_chat
self.signals = ChatSignals()
self.signals.loaded.connect(self._populate)
self.signals.created.connect(self._created)
self.signals.failed.connect(self._failed)
col = QVBoxLayout(self)
col.setContentsMargins(18, 18, 18, 16)
col.setSpacing(12)
head = QHBoxLayout()
head.setSpacing(10)
head.addWidget(icons.IconLabel("bolt", theme.ACCENT_INK, 24, tile=theme.ACCENT))
heading = QLabel("Chats")
heading.setObjectName("Wordmark")
head.addWidget(heading)
head.addStretch(1)
self.status = QLabel("")
self.status.setObjectName("Hint")
head.addWidget(self.status)
col.addLayout(head)
sub = QLabel("Every conversation with Bolt, from any device. A new chat starts with "
"fresh history; Bolt still remembers you.")
sub.setObjectName("HeroSub")
sub.setWordWrap(True)
col.addWidget(sub)
self.list = QListWidget()
self.list.setObjectName("ChatList")
self.list.setUniformItemSizes(False)
self.list.setVerticalScrollMode(QListWidget.ScrollPerPixel)
self.list.itemActivated.connect(self._open_item) # Enter / double-click
self.list.currentItemChanged.connect(lambda *_: self._sync_buttons())
col.addWidget(self.list, 1)
buttons = QHBoxLayout()
buttons.setSpacing(8)
self.refresh_button = QPushButton("Refresh")
self.refresh_button.setObjectName("Ghost")
self.refresh_button.clicked.connect(self.refresh)
buttons.addWidget(self.refresh_button)
buttons.addStretch(1)
self.new_button = QPushButton("New Chat")
self.new_button.setObjectName("Secondary")
self.new_button.clicked.connect(self.new_chat)
buttons.addWidget(self.new_button)
self.open_button = QPushButton("Open")
self.open_button.setObjectName("Primary")
self.open_button.clicked.connect(lambda: self._open_item(self.list.currentItem()))
buttons.addWidget(self.open_button)
col.addLayout(buttons)
for button in (self.refresh_button, self.new_button, self.open_button):
button.setCursor(Qt.PointingHandCursor)
self._busy = False
self._sync_buttons()
# ── loading ──────────────────────────────────────────────────────────
def showEvent(self, event) -> None:
super().showEvent(event)
theme.dark_title_bar(self) # same caption colour as the main window on Windows
self.refresh()
def refresh(self) -> None:
if self._busy:
return
self._set_busy(True, "Loading…")
run_in_pool("list", self._list_chats, self.signals)
def _populate(self, chats: list) -> None:
self._set_busy(False, "")
self.list.clear()
current = None
for chat in chats:
session_id = str(chat.get("session_id") or "")
if not session_id:
continue
is_current = session_id == config.SESSION_ID
item = QListWidgetItem()
item.setData(Qt.UserRole, session_id)
item.setData(Qt.UserRole + 1, str(chat.get("label") or session_id))
item.setToolTip(f"{session_id}\nLast active {local_time(chat.get('last_active', ''))}")
row = ChatRow(chat, is_current)
item.setSizeHint(QSize(0, row.sizeHint().height()))
self.list.addItem(item)
self.list.setItemWidget(item, row)
if is_current:
current = item
if self.list.count() == 0:
self.status.setText("No chats yet")
self.list.setCurrentItem(current or self.list.item(0))
self._sync_buttons()
def _failed(self, operation: str, message: str) -> None:
self._set_busy(False, "")
what = "load chats" if operation == "list" else "start a new chat"
self.status.setText(f"Couldn't {what}")
self.status.setToolTip(message)
self.status.setStyleSheet(f"color: {theme.RED};")
# ── actions ──────────────────────────────────────────────────────────
def new_chat(self) -> None:
if self._busy:
return
self._set_busy(True, "Starting a new chat…")
run_in_pool("create", self._create_chat, self.signals)
def _created(self, session_id: str) -> None:
# A new chat isn't listed until it's used, so there's nothing to show:
# hand it to the window and close.
self._set_busy(False, "")
self.chosen.emit(session_id, "New chat")
self.accept()
def _open_item(self, item: Optional[QListWidgetItem]) -> None:
if item is None or self._busy:
return
self.chosen.emit(item.data(Qt.UserRole), item.data(Qt.UserRole + 1))
self.accept()
def _set_busy(self, busy: bool, status: str) -> None:
self._busy = busy
self.status.setText(status)
self.status.setToolTip("")
self.status.setStyleSheet("")
self._sync_buttons()
def _sync_buttons(self) -> None:
self.refresh_button.setEnabled(not self._busy)
self.new_button.setEnabled(not self._busy)
self.open_button.setEnabled(not self._busy and self.list.currentItem() is not None)
bolt_console/gui/icons.py
+219
View File
@@ -0,0 +1,219 @@
"""Icons drawn with QPainter rather than shipped as files or borrowed from an
emoji font — crisp at any DPI, recolourable, and no assets to lose.
`paint(painter, name, rect, colour)` draws into a rect; `icon(name, colour)`
wraps that as a QIcon; `IconLabel` is a tiny widget that shows one.
"""
from __future__ import annotations
from PySide6.QtCore import QBuffer, QByteArray, QIODevice, QPointF, QRectF, QSize, Qt
from PySide6.QtGui import QColor, QIcon, QImage, QPainter, QPainterPath, QPen, QPixmap
from PySide6.QtWidgets import QWidget
def _pen(colour: QColor, width: float) -> QPen:
pen = QPen(colour, width)
pen.setCapStyle(Qt.RoundCap)
pen.setJoinStyle(Qt.RoundJoin)
return pen
def paint(p: QPainter, name: str, rect: QRectF, colour) -> None:
colour = QColor(colour)
p.save()
p.setRenderHint(QPainter.Antialiasing)
# Everything below is drawn on a 24x24 grid, then scaled into *rect*.
side = min(rect.width(), rect.height())
p.translate(rect.center().x() - side / 2, rect.center().y() - side / 2)
p.scale(side / 24.0, side / 24.0)
stroke = _pen(colour, 1.9)
if name == "bolt":
path = QPainterPath()
for i, (x, y) in enumerate([(13.5, 2), (4.5, 13.5), (11, 13.5), (9.5, 22),
(19.5, 9.5), (13, 9.5), (13.5, 2)]):
path.moveTo(x, y) if i == 0 else path.lineTo(x, y)
p.setPen(Qt.NoPen)
p.setBrush(colour)
p.drawPath(path)
elif name == "send":
p.setPen(_pen(colour, 2.4))
p.drawLine(QPointF(12, 19), QPointF(12, 5.5))
p.drawLine(QPointF(6.5, 11), QPointF(12, 5.5))
p.drawLine(QPointF(17.5, 11), QPointF(12, 5.5))
elif name == "folder":
p.setPen(stroke)
path = QPainterPath()
path.moveTo(3, 7)
path.lineTo(3, 18.5)
path.quadTo(3, 20, 4.5, 20)
path.lineTo(19.5, 20)
path.quadTo(21, 20, 21, 18.5)
path.lineTo(21, 9.5)
path.quadTo(21, 8, 19.5, 8)
path.lineTo(11.5, 8)
path.lineTo(9.5, 5.5)
path.lineTo(4.5, 5.5)
path.quadTo(3, 5.5, 3, 7)
p.drawPath(path)
elif name == "shell":
p.setPen(stroke)
p.drawRoundedRect(QRectF(2.5, 4, 19, 16), 3, 3)
p.drawLine(QPointF(7, 10), QPointF(10, 12.5))
p.drawLine(QPointF(10, 12.5), QPointF(7, 15))
p.drawLine(QPointF(12.5, 15.5), QPointF(17, 15.5))
elif name in ("read", "write", "edit"):
p.setPen(stroke)
page = QPainterPath()
page.moveTo(14, 2.5)
page.lineTo(6, 2.5)
page.quadTo(4.5, 2.5, 4.5, 4)
page.lineTo(4.5, 20)
page.quadTo(4.5, 21.5, 6, 21.5)
page.lineTo(18, 21.5)
page.quadTo(19.5, 21.5, 19.5, 20)
page.lineTo(19.5, 8)
page.closeSubpath()
p.drawPath(page)
p.drawLine(QPointF(14, 2.5), QPointF(14, 8))
p.drawLine(QPointF(14, 8), QPointF(19.5, 8))
if name == "read":
for y in (12, 15, 18):
p.drawLine(QPointF(8, y), QPointF(16, y))
elif name == "write":
p.drawLine(QPointF(12, 11.5), QPointF(12, 18))
p.drawLine(QPointF(8.75, 14.75), QPointF(15.25, 14.75))
else:
p.drawLine(QPointF(8, 18), QPointF(15.5, 10.5))
p.drawLine(QPointF(8, 18), QPointF(8, 15.5))
p.drawLine(QPointF(8, 18), QPointF(10.5, 18))
elif name == "list":
p.setPen(stroke)
for y in (6.5, 12, 17.5):
p.drawPoint(QPointF(4.5, y))
p.drawLine(QPointF(8.5, y), QPointF(20, y))
elif name == "blocked":
p.setPen(stroke)
p.drawEllipse(QRectF(3.5, 3.5, 17, 17))
p.drawLine(QPointF(7, 17), QPointF(17, 7))
elif name == "chevron":
p.setPen(stroke)
p.drawLine(QPointF(9, 6), QPointF(15, 12))
p.drawLine(QPointF(15, 12), QPointF(9, 18))
elif name == "chats":
p.setPen(stroke)
bubble = QPainterPath()
bubble.addRoundedRect(QRectF(3, 4, 18, 13), 4, 4)
p.drawPath(bubble)
p.drawLine(QPointF(8, 17), QPointF(7, 21))
p.drawLine(QPointF(7, 21), QPointF(12, 17))
for x in (8, 12, 16):
p.drawPoint(QPointF(x, 10.5))
elif name == "mic":
p.setPen(stroke)
p.drawRoundedRect(QRectF(9, 3, 6, 11.5), 3, 3)
arc = QPainterPath()
arc.moveTo(5.5, 11)
arc.cubicTo(5.5, 19, 18.5, 19, 18.5, 11)
p.drawPath(arc)
p.drawLine(QPointF(12, 17), QPointF(12, 21))
elif name == "stop":
p.setPen(Qt.NoPen)
p.setBrush(colour)
p.drawRoundedRect(QRectF(7, 7, 10, 10), 2, 2)
elif name == "plus":
p.setPen(_pen(colour, 2.1))
p.drawLine(QPointF(12, 5), QPointF(12, 19))
p.drawLine(QPointF(5, 12), QPointF(19, 12))
elif name == "file":
paint(p, "read", QRectF(0, 0, 24, 24), colour)
p.restore()
def icon(name: str, colour, size: int = 32) -> QIcon:
pixmap = QPixmap(size, size)
pixmap.fill(Qt.transparent)
painter = QPainter(pixmap)
paint(painter, name, QRectF(0, 0, size, size), colour)
painter.end()
return QIcon(pixmap)
APP_ICON_SIZES = (16, 24, 32, 48, 64, 128, 256)
def app_icon_image(size: int) -> QImage:
"""An amber rounded square with a dark bolt, at *size* pixels. The one
drawing behind both the window icon and the .exe's icon (write_ico)."""
image = QImage(size, size, QImage.Format_ARGB32_Premultiplied)
image.fill(Qt.transparent)
p = QPainter(image)
p.setRenderHint(QPainter.Antialiasing)
p.setPen(Qt.NoPen)
p.setBrush(QColor("#f5b83d"))
p.drawRoundedRect(QRectF(0, 0, size, size), size * 0.26, size * 0.26)
inset = size * 0.18
paint(p, "bolt", QRectF(inset, inset, size - 2 * inset, size - 2 * inset), "#1a1305")
p.end()
return image
def app_icon() -> QIcon:
"""The window/taskbar icon."""
result = QIcon()
for size in APP_ICON_SIZES:
result.addPixmap(QPixmap.fromImage(app_icon_image(size)))
return result
def write_ico(path) -> None:
"""Save the app icon as a multi-size Windows .ico (PNG-compressed entries,
which Windows Vista+ reads at every size), for the frozen .exe."""
import struct
blobs = []
for size in APP_ICON_SIZES:
data = QByteArray()
buffer = QBuffer(data)
buffer.open(QIODevice.WriteOnly)
app_icon_image(size).save(buffer, "PNG")
buffer.close()
blobs.append((size, bytes(data)))
header = struct.pack("<HHH", 0, 1, len(blobs)) # reserved, type 1 = icon, count
offset = len(header) + 16 * len(blobs)
entries, payload = b"", b""
for size, blob in blobs:
edge = 0 if size >= 256 else size # 0 means 256 in the ICO format
entries += struct.pack("<BBBBHHII", edge, edge, 0, 0, 1, 32, len(blob), offset + len(payload))
payload += blob
with open(path, "wb") as handle:
handle.write(header + entries + payload)
class IconLabel(QWidget):
"""A fixed-size painted icon, optionally on a rounded tile."""
def __init__(self, name: str, colour, size: int = 18, tile=None, parent=None):
super().__init__(parent)
self._name, self._colour, self._tile = name, QColor(colour), tile
self.setFixedSize(QSize(size, size))
def set_icon(self, name: str, colour=None) -> None:
self._name = name
if colour is not None:
self._colour = QColor(colour)
self.update()
def paintEvent(self, _event) -> None:
p = QPainter(self)
p.setRenderHint(QPainter.Antialiasing)
rect = QRectF(self.rect())
if self._tile is not None:
p.setPen(Qt.NoPen)
p.setBrush(QColor(self._tile))
p.drawRoundedRect(rect, rect.width() * 0.28, rect.width() * 0.28)
pad = rect.width() * 0.22
rect = rect.adjusted(pad, pad, -pad, -pad)
paint(p, self._name, rect, self._colour)
bolt_console/gui/theme.py
+196
View File
@@ -0,0 +1,196 @@
"""Colours, fonts and the application stylesheet.
One dark theme, tuned by hand: near-black charcoal surfaces a step apart, a
single warm accent (the bolt's amber), and colour used only where it carries
meaning — green for done, red for failed, amber for "needs you".
"""
from __future__ import annotations
import sys
from PySide6.QtGui import QColor, QFont, QFontDatabase, QPalette
BG = "#0e1014" # window
SURFACE = "#15181e" # cards, composer
SURFACE_HI = "#1b1f27" # hover / user bubble
BORDER = "#252a34"
BORDER_HI = "#343b48"
TEXT = "#e7e9ee"
TEXT_DIM = "#9aa1ae"
TEXT_FAINT = "#6b7280"
ACCENT = "#f5b83d" # the bolt
ACCENT_HI = "#ffc95c"
ACCENT_INK = "#1a1305" # text on accent
GREEN = "#4ec98a"
RED = "#f06a6a"
BLUE = "#7aa7ff"
CODE = "#e8c98f" # inline code in replies
DIFF_ADD_BG = "#12291d"
DIFF_DEL_BG = "#301719"
RADIUS = 12
COLUMN_WIDTH = 820 # the transcript's reading column
def _first_available(families, fallback: str) -> str:
installed = set(QFontDatabase.families())
for family in families:
if family in installed:
return family
return fallback
def ui_font_family() -> str:
return _first_available(
["Segoe UI Variable Text", "Inter", "SF Pro Text", "Segoe UI", "Cantarell", "Ubuntu"],
QFont().defaultFamily(),
)
def mono_font_family() -> str:
return _first_available(
["Cascadia Code", "Cascadia Mono", "JetBrains Mono", "SF Mono", "Consolas",
"DejaVu Sans Mono", "Menlo"],
"monospace",
)
def mono_font(size: float = 9.5) -> QFont:
font = QFont(mono_font_family())
font.setPointSizeF(size)
font.setStyleHint(QFont.Monospace)
return font
def palette() -> QPalette:
pal = QPalette()
for role, colour in (
(QPalette.Window, BG), (QPalette.Base, SURFACE), (QPalette.AlternateBase, SURFACE_HI),
(QPalette.Text, TEXT), (QPalette.WindowText, TEXT), (QPalette.ButtonText, TEXT),
(QPalette.Button, SURFACE), (QPalette.Highlight, "#3a3222"),
(QPalette.HighlightedText, TEXT), (QPalette.Link, BLUE),
(QPalette.PlaceholderText, TEXT_FAINT), (QPalette.ToolTipBase, SURFACE_HI),
(QPalette.ToolTipText, TEXT),
):
pal.setColor(role, QColor(colour))
return pal
def stylesheet() -> str:
mono = mono_font_family()
return f"""
QWidget {{ color: {TEXT}; font-size: 10.5pt; }}
QMainWindow, #Root {{ background: {BG}; }}
QToolTip {{ background: {SURFACE_HI}; color: {TEXT}; border: 1px solid {BORDER_HI};
border-radius: 6px; padding: 5px 8px; }}
/* header */
#Header {{ background: {BG}; border-bottom: 1px solid {BORDER}; }}
#Wordmark {{ font-size: 12pt; font-weight: 600; letter-spacing: 0.3px; }}
#WordmarkSub {{ color: {TEXT_FAINT}; font-size: 9pt; }}
#Chip {{ background: {SURFACE}; border: 1px solid {BORDER}; border-radius: 15px;
padding: 5px 12px 5px 30px; color: {TEXT_DIM}; text-align: left; }}
#Chip:hover {{ border-color: {BORDER_HI}; color: {TEXT}; background: {SURFACE_HI}; }}
#Chip:disabled {{ color: {TEXT_FAINT}; }}
#Segment {{ background: {SURFACE}; border: 1px solid {BORDER}; border-radius: 15px; }}
#SegmentButton {{ background: transparent; border: none; border-radius: 12px;
padding: 4px 14px; color: {TEXT_DIM}; font-weight: 500; }}
#SegmentButton:hover {{ color: {TEXT}; }}
#SegmentButton:checked {{ background: {SURFACE_HI}; color: {TEXT}; }}
#SegmentButton[danger="true"]:checked {{ background: #3a2c12; color: {ACCENT_HI}; }}
/* transcript */
QScrollArea {{ background: transparent; border: none; }}
#Transcript {{ background: {BG}; }}
QScrollBar:vertical {{ background: transparent; width: 10px; margin: 4px 2px; }}
QScrollBar::handle:vertical {{ background: {BORDER}; border-radius: 3px; min-height: 40px; }}
QScrollBar::handle:vertical:hover {{ background: {BORDER_HI}; }}
QScrollBar::add-line, QScrollBar::sub-line, QScrollBar::add-page, QScrollBar::sub-page {{
background: none; height: 0; }}
QScrollBar:horizontal {{ height: 0; }}
#UserBubble {{ background: {SURFACE_HI}; border: 1px solid {BORDER}; border-radius: 16px; }}
#AssistantText {{ color: {TEXT}; }}
#Holding {{ color: {TEXT_FAINT}; font-style: italic; }}
#Notice {{ color: {TEXT_DIM}; font-size: 9.5pt; }}
#NoticeError {{ color: {RED}; }}
#ToolCard {{ background: {SURFACE}; border: 1px solid {BORDER}; border-radius: {RADIUS}px; }}
#ToolCard[state="waiting"] {{ border-color: #5a4418; }}
#ToolHeader {{ background: transparent; border: none; border-radius: {RADIUS}px; }}
#ToolHeader:hover {{ background: {SURFACE_HI}; }}
#ToolName {{ font-weight: 600; }}
#ToolArg {{ color: {TEXT_DIM}; font-family: "{mono}"; font-size: 9.5pt; }}
#ToolOutput {{ background: transparent; color: {TEXT_DIM}; border: none; border-top: 1px solid {BORDER};
font-family: "{mono}"; font-size: 9.2pt; padding: 10px 14px; }}
#Pill {{ border-radius: 9px; padding: 1px 9px; font-size: 8.5pt; font-weight: 600; }}
#Pill[kind="running"] {{ background: #1c2433; color: {BLUE}; }}
#Pill[kind="ok"] {{ background: #13271d; color: {GREEN}; }}
#Pill[kind="error"] {{ background: #2e1618; color: {RED}; }}
#Pill[kind="waiting"] {{ background: #33280f; color: {ACCENT}; }}
#Pill[kind="declined"] {{ background: #22252c; color: {TEXT_DIM}; }}
#ApprovalBar {{ border-top: 1px solid {BORDER}; background: transparent; }}
#ApprovalQuestion {{ color: {TEXT}; }}
#Primary {{ background: {ACCENT}; color: {ACCENT_INK}; border: none; border-radius: 8px;
padding: 6px 16px; font-weight: 600; }}
#Primary:hover {{ background: {ACCENT_HI}; }}
#Secondary {{ background: {SURFACE_HI}; color: {TEXT}; border: 1px solid {BORDER_HI};
border-radius: 8px; padding: 6px 14px; }}
#Secondary:hover {{ border-color: {TEXT_FAINT}; }}
#Ghost {{ background: transparent; color: {TEXT_DIM}; border: none; padding: 6px 10px; }}
#Ghost:hover {{ color: {TEXT}; }}
/* empty state */
#Hero {{ font-size: 19pt; font-weight: 600; }}
#HeroSub {{ color: {TEXT_DIM}; }}
#Suggestion {{ background: {SURFACE}; border: 1px solid {BORDER}; border-radius: 12px;
padding: 12px 14px; color: {TEXT_DIM}; text-align: left; }}
#Suggestion:hover {{ border-color: {BORDER_HI}; color: {TEXT}; background: {SURFACE_HI}; }}
/* composer */
#Composer {{ background: {SURFACE}; border: 1px solid {BORDER}; border-radius: 18px; }}
#Composer[focused="true"] {{ border-color: {BORDER_HI}; }}
#ComposerEdit {{ background: transparent; border: none; padding: 0; selection-background-color: #3a3222; }}
#Hint {{ color: {TEXT_FAINT}; font-size: 8.8pt; }}
/* chats dialog */
QDialog#ChatDialog {{ background: {BG}; }}
#ChatList {{ background: {SURFACE}; border: 1px solid {BORDER}; border-radius: {RADIUS}px;
padding: 4px; outline: none; }}
#ChatList::item {{ border-radius: 9px; margin: 1px 0; }}
#ChatList::item:hover {{ background: {SURFACE_HI}; }}
#ChatList::item:selected {{ background: #262017; border: 1px solid #4a3a18; }}
#ChatTitle {{ font-weight: 600; }}
#ChatTime {{ color: {TEXT_FAINT}; font-size: 9pt; }}
#ChatPreview {{ color: {TEXT_DIM}; font-size: 9.5pt; }}
"""
def dark_title_bar(window) -> None:
"""Windows 11: dark caption, painted the same colour as the window so the
title bar and the app read as one surface. Silently does nothing on other
platforms or older Windows."""
if sys.platform != "win32":
return
try:
import ctypes
from ctypes import wintypes
hwnd = wintypes.HWND(int(window.winId()))
dwm = ctypes.windll.dwmapi
def attr(key: int, value: int) -> None:
data = ctypes.c_int(value)
dwm.DwmSetWindowAttribute(hwnd, key, ctypes.byref(data), ctypes.sizeof(data))
attr(20, 1) # DWMWA_USE_IMMERSIVE_DARK_MODE
colour = BG.lstrip("#")
r, g, b = int(colour[0:2], 16), int(colour[2:4], 16), int(colour[4:6], 16)
attr(35, (b << 16) | (g << 8) | r) # DWMWA_CAPTION_COLOR (COLORREF is 0x00BBGGRR)
border = BORDER.lstrip("#")
r, g, b = int(border[0:2], 16), int(border[2:4], 16), int(border[4:6], 16)
attr(34, (b << 16) | (g << 8) | r) # DWMWA_BORDER_COLOR
except Exception:
pass
bolt_console/gui/widgets.py
+699
View File
@@ -0,0 +1,699 @@
"""The pieces the window is built from: messages, tool cards, the composer,
the header controls and the empty state. Styling lives in theme.stylesheet();
these set object names and properties for it to hook onto."""
from __future__ import annotations
import html
import math
import re
from PySide6.QtCore import QEvent, QRectF, QSize, Qt, QTimer, Signal
from PySide6.QtGui import QColor, QFontDatabase, QPainter, QTextCursor, QTextDocument, QTextOption
from PySide6.QtWidgets import (
QButtonGroup, QFrame, QHBoxLayout, QLabel, QPlainTextEdit, QPushButton,
QSizePolicy, QVBoxLayout, QWidget,
)
from . import icons, theme
# Tool name (as tools.py reports it) -> icon.
TOOL_ICONS = {
"Shell": "shell", "Read": "read", "List": "list", "Write": "write",
"Edit": "edit", "Pet": "blocked", "File": "file",
}
COLLAPSED_LINES = 6
def _label(text: str = "", name: str = "", wrap: bool = True, selectable: bool = False) -> QLabel:
label = QLabel(text)
if name:
label.setObjectName(name)
label.setWordWrap(wrap)
if selectable:
label.setTextInteractionFlags(Qt.TextSelectableByMouse | Qt.LinksAccessibleByMouse)
label.setCursor(Qt.IBeamCursor)
return label
def repolish(widget: QWidget) -> None:
"""Re-apply the stylesheet after a dynamic property changed."""
widget.style().unpolish(widget)
widget.style().polish(widget)
widget.update()
def markdown_html(text: str, font) -> str:
"""Markdown -> the HTML QLabel renders, with code in the app's mono font
and a warm tint instead of Qt's generic 'monospace' (usually Courier)."""
doc = QTextDocument()
doc.setDefaultFont(font)
doc.setMarkdown(text)
out = doc.toHtml()
code_style = f"font-family:'{theme.mono_font_family()}'; color:{theme.CODE};"
# Qt writes the platform's fixed font for code: 'Courier New' on Windows,
# 'monospace' elsewhere.
for fixed in {QFontDatabase.systemFont(QFontDatabase.FixedFont).family(), "monospace", "Courier New"}:
out = out.replace(f"font-family:'{fixed}';", code_style)
return out.replace("font-family:'Sans Serif';", "")
# ── transcript entries ──────────────────────────────────────────────────────
class UserMessage(QWidget):
def __init__(self, text: str, parent=None):
super().__init__(parent)
row = QHBoxLayout(self)
row.setContentsMargins(80, 6, 0, 6)
row.addStretch(1)
bubble = QFrame()
bubble.setObjectName("UserBubble")
inner = QVBoxLayout(bubble)
inner.setContentsMargins(16, 10, 16, 10)
label = _label(text, selectable=True)
label.setTextFormat(Qt.PlainText)
# A word-wrapped QLabel's size hint is a guess that wraps short text
# early; give it the width its longest line actually needs.
widest = max(label.fontMetrics().horizontalAdvance(line) for line in text.splitlines() or [""])
label.setMinimumWidth(min(widest + 4, 588))
inner.addWidget(label)
bubble.setMaximumWidth(620)
bubble.setSizePolicy(QSizePolicy.Maximum, QSizePolicy.Preferred)
row.addWidget(bubble)
class AssistantMessage(QWidget):
"""Bolt's reply. Streamed sentences are appended; a sentence that starts a
list item or heading goes on its own line so the markdown still renders."""
_BLOCK_START = re.compile(r"^(?:[-*+] |\d+[.)] |#{1,6} |```|>)")
def __init__(self, text: str = "", parent=None):
super().__init__(parent)
self._text = ""
row = QHBoxLayout(self)
row.setContentsMargins(0, 8, 0, 8)
row.setSpacing(14)
avatar = icons.IconLabel("bolt", theme.ACCENT_INK, 26, tile=theme.ACCENT)
row.addWidget(avatar, 0, Qt.AlignTop)
self.label = _label(name="AssistantText", selectable=True)
self.label.setTextFormat(Qt.RichText)
self.label.setOpenExternalLinks(True)
self.label.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Preferred)
row.addWidget(self.label, 1)
if text:
self.set_text(text)
def set_text(self, text: str) -> None:
self._text = text
self.label.setText(markdown_html(text, self.label.font()))
def append(self, sentence: str) -> None:
if not self._text:
joiner = ""
elif self._BLOCK_START.match(sentence) or self._text.endswith("\n"):
joiner = "\n\n" if not self._text.endswith("\n") else ""
else:
joiner = " "
self.set_text(self._text + joiner + sentence)
def text(self) -> str:
return self._text
class HoldingLine(QWidget):
def __init__(self, text: str, parent=None):
super().__init__(parent)
row = QHBoxLayout(self)
row.setContentsMargins(40, 0, 0, 2)
row.addWidget(_label(text, "Holding"))
class Notice(QWidget):
def __init__(self, level: str, text: str, parent=None):
super().__init__(parent)
row = QHBoxLayout(self)
row.setContentsMargins(40, 4, 0, 4)
label = _label(text, "NoticeError" if level == "error" else "Notice", selectable=True)
label.setTextFormat(Qt.PlainText)
if level == "warn":
label.setStyleSheet(f"color: {theme.ACCENT};")
row.addWidget(label, 1)
class Pill(QLabel):
TEXT = {"running": "Running", "ok": "Done", "error": "Failed",
"waiting": "Needs approval", "declined": "Declined"}
def __init__(self, kind: str = "running", parent=None):
super().__init__(parent)
self.setObjectName("Pill")
self.set_kind(kind)
def set_kind(self, kind: str) -> None:
self.setProperty("kind", kind)
self.setText(self.TEXT.get(kind, kind))
repolish(self)
class _ClickableFrame(QFrame):
clicked = Signal()
def mouseReleaseEvent(self, event) -> None:
if event.button() == Qt.LeftButton and self.rect().contains(event.position().toPoint()):
self.clicked.emit()
super().mouseReleaseEvent(event)
class ToolCard(QFrame):
"""One relayed command: header (icon, name, argument, status), optional
diff, output that expands on click, and an inline approval bar."""
answered = Signal(str) # yes / no / always
def __init__(self, name: str, argument: str, parent=None):
super().__init__(parent)
self.setObjectName("ToolCard")
self._output = ""
self._expanded = False
self._error = False
outer = QVBoxLayout(self)
outer.setContentsMargins(0, 0, 0, 0)
outer.setSpacing(0)
self.header = _ClickableFrame()
self.header.setObjectName("ToolHeader")
self.header.setCursor(Qt.PointingHandCursor)
head = QHBoxLayout(self.header)
head.setContentsMargins(12, 9, 12, 9)
head.setSpacing(10)
self.icon = icons.IconLabel(TOOL_ICONS.get(name, "shell"), theme.TEXT_DIM, 16)
head.addWidget(self.icon)
title = _label(name, "ToolName", wrap=False)
head.addWidget(title)
shown = argument if len(argument) <= 110 else argument[:107] + "..."
arg = _label(shown, "ToolArg", wrap=False)
arg.setTextFormat(Qt.PlainText)
arg.setToolTip(argument)
arg.setSizePolicy(QSizePolicy.Ignored, QSizePolicy.Preferred)
head.addWidget(arg, 1)
self.pill = Pill("running")
head.addWidget(self.pill)
self.chevron = icons.IconLabel("chevron", theme.TEXT_FAINT, 14)
self.chevron.hide()
head.addWidget(self.chevron)
self.header.clicked.connect(self.toggle)
outer.addWidget(self.header)
self.diff = _label(name="ToolOutput", wrap=True, selectable=True)
self.diff.setTextFormat(Qt.RichText)
self.diff.hide()
outer.addWidget(self.diff)
self.output = _label(name="ToolOutput", wrap=True, selectable=True)
self.output.setTextFormat(Qt.PlainText)
self.output.hide()
outer.addWidget(self.output)
self.approval = QFrame()
self.approval.setObjectName("ApprovalBar")
bar = QHBoxLayout(self.approval)
bar.setContentsMargins(14, 10, 10, 10)
bar.setSpacing(8)
self.question = _label("", "ApprovalQuestion", wrap=False)
bar.addWidget(self.question, 1)
for text, value, style in (("Deny", "no", "Ghost"), ("Always allow", "always", "Secondary"),
("Allow", "yes", "Primary")):
button = QPushButton(text)
button.setObjectName(style)
button.setCursor(Qt.PointingHandCursor)
button.clicked.connect(lambda _=False, v=value: self._answer(v))
bar.addWidget(button)
if value == "yes":
self.allow_button = button
self.approval.hide()
outer.addWidget(self.approval)
# ── state ────────────────────────────────────────────────────────────
def set_result(self, text: str, error: bool) -> None:
if self.pill.property("kind") == "declined":
return
self._output, self._error = text.rstrip(), error
self.pill.set_kind("error" if error else "ok")
self.icon.set_icon(self.icon._name, theme.RED if error else theme.TEXT_DIM)
self._render_output()
def set_diff(self, old: str, new: str, limit: int = 40) -> None:
rows = []
for sign, text, fg, bg in (("-", old, theme.RED, theme.DIFF_DEL_BG),
("+", new, theme.GREEN, theme.DIFF_ADD_BG)):
lines = text.splitlines() or [""]
for line in lines[:limit]:
rows.append(
f'<tr><td style="background:{bg}; color:{fg}; padding:1px 8px; '
f'white-space:pre;">{sign} {html.escape(line) or "&nbsp;"}</td></tr>')
if len(lines) > limit:
rows.append(f'<tr><td style="color:{theme.TEXT_FAINT}; padding:1px 8px;">'
f' … {len(lines) - limit} more lines</td></tr>')
self.diff.setText('<table width="100%" cellspacing="0" cellpadding="0">' + "".join(rows) + "</table>")
self.diff.show()
def ask(self, question: str) -> None:
self.question.setText(question)
self.pill.set_kind("waiting")
self.setProperty("state", "waiting")
repolish(self)
self.approval.show()
self.allow_button.setFocus()
def _answer(self, value: str) -> None:
self.approval.hide()
self.setProperty("state", "")
repolish(self)
if value == "no":
self.pill.set_kind("declined")
else:
self.pill.set_kind("running")
self.answered.emit(value)
def toggle(self) -> None:
if self._output:
self._expanded = not self._expanded
self._render_output()
def _render_output(self) -> None:
lines = align_columns(self._output.splitlines())
if not lines:
self.output.hide()
self.chevron.hide()
return
long = len(lines) > COLLAPSED_LINES
shown = lines if (self._expanded or not long) else lines[:COLLAPSED_LINES]
text = "\n".join(shown)
if long and not self._expanded:
text += f"\n… {len(lines) - COLLAPSED_LINES} more lines — click to expand"
self.output.setText(text)
self.output.setStyleSheet(f"color: {theme.RED};" if self._error else "")
self.output.show()
self.chevron.setVisible(long)
def align_columns(lines):
"""Tab-separated output (filectl list's "name<TAB>size") into padded
columns — a QLabel's tab stops don't line up with anything."""
rows = [line.split(" ") for line in lines]
if not any(len(r) > 1 for r in rows):
return lines
width = max(len(r[0]) for r in rows if len(r) > 1)
return [r[0].ljust(width + 2) + " ".join(r[1:]) if len(r) > 1 else r[0] for r in rows]
class ThinkingIndicator(QWidget):
"""Three dots breathing in sequence, next to a quiet 'Thinking'."""
def __init__(self, parent=None):
super().__init__(parent)
self.setFixedHeight(34)
self._phase = 0.0
self._timer = QTimer(self)
self._timer.setInterval(40)
self._timer.timeout.connect(self._tick)
self.label = "Thinking"
def start(self) -> None:
self._phase = 0.0
self.show()
self._timer.start()
def stop(self) -> None:
self._timer.stop()
self.hide()
def _tick(self) -> None:
self._phase += 0.12
self.update()
def paintEvent(self, _event) -> None:
p = QPainter(self)
p.setRenderHint(QPainter.Antialiasing)
p.setPen(Qt.NoPen)
x, y = 50.0, self.height() / 2
for i in range(3):
level = (math.sin(self._phase - i * 0.9) + 1) / 2
colour = QColor(theme.ACCENT)
colour.setAlphaF(0.25 + 0.75 * level)
p.setBrush(colour)
r = 2.6 + 1.0 * level
p.drawEllipse(QRectF(x + i * 12 - r, y - r, 2 * r, 2 * r))
p.setPen(QColor(theme.TEXT_FAINT))
p.drawText(QRectF(x + 34, 0, 300, self.height()), Qt.AlignVCenter | Qt.AlignLeft, self.label)
# ── composer ────────────────────────────────────────────────────────────────
class _SendButton(QPushButton):
def __init__(self, parent=None):
super().__init__(parent)
self.setFixedSize(34, 34)
self.setCursor(Qt.PointingHandCursor)
self.setToolTip("Send (Enter)")
self.setFlat(True)
def paintEvent(self, _event) -> None:
p = QPainter(self)
p.setRenderHint(QPainter.Antialiasing)
p.setPen(Qt.NoPen)
enabled = self.isEnabled()
fill = QColor(theme.ACCENT_HI if self.underMouse() and enabled else theme.ACCENT)
if not enabled:
fill = QColor(theme.BORDER_HI)
p.setBrush(fill)
p.drawEllipse(QRectF(self.rect()).adjusted(1, 1, -1, -1))
icons.paint(p, "send", QRectF(self.rect()).adjusted(8, 8, -8, -8),
theme.ACCENT_INK if enabled else theme.TEXT_FAINT)
class MicButton(QPushButton):
"""Dictation toggle. idle: a quiet mic outline. recording: red, with a
ring that swells with your voice, and a stop square. finishing: an amber
arc while the last words come back. unavailable: faint, and the tooltip
says what's missing."""
TIPS = {
"idle": "Dictate (Ctrl+M)\nClick, talk, click again (or pause) to stop",
"recording": "Listening. Click or pause to stop, Esc to cancel",
"finishing": "Transcribing...",
}
def __init__(self, parent=None):
super().__init__(parent)
self.setFixedSize(34, 34)
self.setCursor(Qt.PointingHandCursor)
self.setFlat(True)
self.state = "idle"
self._level = 0.0
self._phase = 0.0
self._timer = QTimer(self)
self._timer.setInterval(33)
self._timer.timeout.connect(self._tick)
self.set_state("idle")
def set_state(self, state: str, reason: str = "") -> None:
self.state = state
self.setToolTip(reason if state == "unavailable" else self.TIPS.get(state, ""))
if state in ("recording", "finishing"):
self._timer.start()
else:
self._timer.stop()
self._level = 0.0
self.update()
def set_level(self, level: float) -> None:
# Fast attack, slow release, so the ring breathes instead of flickering.
self._level = level if level > self._level else self._level * 0.85 + level * 0.15
def _tick(self) -> None:
self._phase += 0.2
self._level *= 0.97
self.update()
def paintEvent(self, _event) -> None:
p = QPainter(self)
p.setRenderHint(QPainter.Antialiasing)
rect = QRectF(self.rect()).adjusted(1, 1, -1, -1)
centre = rect.center()
if self.state == "recording":
ring = QColor(theme.RED)
ring.setAlphaF(0.22 + 0.3 * self._level)
p.setPen(Qt.NoPen)
p.setBrush(ring)
p.drawEllipse(centre, 16.0, 16.0)
p.setBrush(QColor(theme.RED))
radius = 10.5 + 3.5 * self._level
p.drawEllipse(centre, radius, radius)
icons.paint(p, "stop", QRectF(centre.x() - 8, centre.y() - 8, 16, 16), "#ffffff")
return
if self.state == "finishing":
pen = QColor(theme.ACCENT)
p.setPen(icons._pen(pen, 2.2))
p.drawArc(rect.adjusted(4, 4, -4, -4), int(-self._phase * 16 * 57), 100 * 16)
icons.paint(p, "mic", rect.adjusted(9, 9, -9, -9), theme.TEXT_DIM)
return
if self.state == "idle" and self.underMouse():
p.setPen(Qt.NoPen)
p.setBrush(QColor(theme.SURFACE_HI))
p.drawEllipse(rect)
colour = theme.TEXT_FAINT if self.state == "unavailable" else (
theme.TEXT if self.underMouse() else theme.TEXT_DIM)
icons.paint(p, "mic", rect.adjusted(8, 8, -8, -8), colour)
class _ComposerEdit(QPlainTextEdit):
submitted = Signal()
def keyPressEvent(self, event) -> None:
if event.key() in (Qt.Key_Return, Qt.Key_Enter) and not (event.modifiers() & Qt.ShiftModifier):
self.submitted.emit()
return
super().keyPressEvent(event)
class Composer(QFrame):
"""Rounded input that grows with its text. Enter sends, Shift+Enter
starts a new line. While a turn runs you can keep typing; sending waits."""
submitted = Signal(str)
MAX_LINES = 9
def __init__(self, parent=None):
super().__init__(parent)
self.setObjectName("Composer")
row = QHBoxLayout(self)
row.setContentsMargins(18, 10, 10, 10)
row.setSpacing(10)
self.edit = _ComposerEdit()
self.edit.setObjectName("ComposerEdit")
self.edit.setPlaceholderText("Ask Bolt to read, change or run something…")
self.edit.setWordWrapMode(QTextOption.WrapAtWordBoundaryOrAnywhere)
self.edit.setVerticalScrollBarPolicy(Qt.ScrollBarAsNeeded)
self.edit.setHorizontalScrollBarPolicy(Qt.ScrollBarAlwaysOff)
self.edit.document().setDocumentMargin(2)
self.edit.textChanged.connect(self._resize)
self.edit.submitted.connect(self._submit)
self.edit.installEventFilter(self)
row.addWidget(self.edit, 1, Qt.AlignVCenter)
self.mic = MicButton()
row.addWidget(self.mic, 0, Qt.AlignBottom)
self._dictation_prefix = None # composer text from before a recording
self.send = _SendButton()
self.send.clicked.connect(self._submit)
row.addWidget(self.send, 0, Qt.AlignBottom)
self._busy = False
self._resize()
self._sync_button()
self.edit.textChanged.connect(self._sync_button)
def eventFilter(self, obj, event) -> bool:
if obj is self.edit and event.type() in (QEvent.FocusIn, QEvent.FocusOut):
self.setProperty("focused", event.type() == QEvent.FocusIn)
repolish(self)
return False
def set_busy(self, busy: bool) -> None:
self._busy = busy
self._sync_button()
def set_text(self, text: str) -> None:
self.edit.setPlainText(text)
cursor = self.edit.textCursor()
cursor.movePosition(QTextCursor.MoveOperation.End)
self.edit.setTextCursor(cursor)
self.edit.setFocus()
# ── dictation: the words appear in the box as you speak ──────────────
def begin_dictation(self) -> None:
self._dictation_prefix = self.edit.toPlainText().rstrip()
self.edit.setReadOnly(True) # typing into text that's being rewritten would be lost
self._sync_button()
def show_dictation(self, final: str, interim: str = "") -> None:
if self._dictation_prefix is None:
return
spoken = " ".join(part for part in (final, interim) if part)
self._show(" ".join(part for part in (self._dictation_prefix, spoken) if part))
def end_dictation(self, text: str) -> None:
if self._dictation_prefix is None:
return
self._show(" ".join(part for part in (self._dictation_prefix, text.strip()) if part))
self._dictation_prefix = None
self.edit.setReadOnly(False)
self.edit.setFocus()
self._sync_button()
@property
def dictating(self) -> bool:
return self._dictation_prefix is not None
def _show(self, text: str) -> None:
self.edit.setPlainText(text)
cursor = self.edit.textCursor()
cursor.movePosition(QTextCursor.MoveOperation.End)
self.edit.setTextCursor(cursor)
def _sync_button(self) -> None:
ready = not self._busy and not self.dictating and bool(self.edit.toPlainText().strip())
self.send.setEnabled(ready)
self.send.update()
def _resize(self) -> None:
line = self.edit.fontMetrics().lineSpacing()
lines = max(1, min(self.MAX_LINES, int(self.edit.document().size().height())))
self.edit.setFixedHeight(int(lines * line + 8))
def _submit(self) -> None:
text = self.edit.toPlainText().strip()
if not text or self._busy or self.dictating:
return
self.edit.clear()
self.submitted.emit(text)
# ── header ──────────────────────────────────────────────────────────────────
class StatusDot(QWidget):
COLOURS = {"unknown": theme.TEXT_FAINT, "ok": theme.GREEN, "error": theme.RED}
def __init__(self, parent=None):
super().__init__(parent)
self.setFixedSize(QSize(10, 10))
self._state = "unknown"
def set_state(self, state: str, tooltip: str) -> None:
self._state = state
self.setToolTip(tooltip)
self.update()
def paintEvent(self, _event) -> None:
p = QPainter(self)
p.setRenderHint(QPainter.Antialiasing)
p.setPen(Qt.NoPen)
p.setBrush(QColor(self.COLOURS.get(self._state, theme.TEXT_FAINT)))
p.drawEllipse(QRectF(1, 1, 8, 8))
class FolderChip(QPushButton):
"""A pill button with a painted icon on the left (the folder by default)."""
def __init__(self, parent=None, icon_name: str = "folder", text: str = ""):
super().__init__(text, parent)
self._icon_name = icon_name
self.setObjectName("Chip")
self.setCursor(Qt.PointingHandCursor)
self.setFixedHeight(30)
def set_path(self, path: str) -> None:
name = path.rstrip("\\/").replace("\\", "/").split("/")[-1] or path
self.setText(name)
self.setToolTip(f"Working directory: {path}\nClick to change")
def paintEvent(self, event) -> None:
super().paintEvent(event)
p = QPainter(self)
paint_colour = theme.TEXT if self.underMouse() else theme.TEXT_DIM
if not self.isEnabled():
paint_colour = theme.TEXT_FAINT
icons.paint(p, self._icon_name, QRectF(10, (self.height() - 15) / 2, 15, 15), paint_colour)
class ModeSwitch(QFrame):
"""Segmented control: ask before changes, or run everything."""
changed = Signal(str)
def __init__(self, mode: str, parent=None):
super().__init__(parent)
self.setObjectName("Segment")
self.setFixedHeight(30)
row = QHBoxLayout(self)
row.setContentsMargins(2, 2, 2, 2)
row.setSpacing(2)
self._group = QButtonGroup(self)
self.buttons = {}
for value, text, tip in (
("ask", "Ask first", "Confirm shell commands, writes and edits"),
("auto", "Auto", "Run everything Bolt asks for without confirming"),
):
button = QPushButton(text)
button.setObjectName("SegmentButton")
button.setCheckable(True)
button.setCursor(Qt.PointingHandCursor)
button.setToolTip(tip)
if value == "auto":
button.setProperty("danger", True)
self._group.addButton(button)
row.addWidget(button)
self.buttons[value] = button
button.clicked.connect(lambda _=False, v=value: self.changed.emit(v))
self.set_mode(mode)
def set_mode(self, mode: str) -> None:
self.buttons.get(mode, self.buttons["ask"]).setChecked(True)
# ── empty state ─────────────────────────────────────────────────────────────
SUGGESTIONS = (
"What's in this folder?",
"Summarise the README",
"Git status and recent commits",
"Find TODO comments in this project",
)
class EmptyState(QWidget):
picked = Signal(str)
def __init__(self, parent=None):
super().__init__(parent)
col = QVBoxLayout(self)
col.setContentsMargins(0, 0, 0, 0)
col.addStretch(3)
col.addWidget(icons.IconLabel("bolt", theme.ACCENT_INK, 52, tile=theme.ACCENT), 0, Qt.AlignHCenter)
col.addSpacing(18)
hero = _label("What are we working on?", "Hero")
hero.setAlignment(Qt.AlignHCenter)
col.addWidget(hero)
self.sub = _label("", "HeroSub")
self.sub.setAlignment(Qt.AlignHCenter)
col.addWidget(self.sub)
col.addSpacing(26)
grid = QVBoxLayout()
grid.setSpacing(10)
for i in range(0, len(SUGGESTIONS), 2):
pair = QHBoxLayout()
pair.setSpacing(10)
for text in SUGGESTIONS[i:i + 2]:
button = QPushButton(text)
button.setObjectName("Suggestion")
button.setCursor(Qt.PointingHandCursor)
button.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Fixed)
button.clicked.connect(lambda _=False, t=text: self.picked.emit(t))
pair.addWidget(button)
grid.addLayout(pair)
wrap = QHBoxLayout()
wrap.addStretch(1)
holder = QWidget()
holder.setLayout(grid)
holder.setMaximumWidth(560)
holder.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Fixed)
wrap.addWidget(holder, 4)
wrap.addStretch(1)
col.addLayout(wrap)
col.addStretch(4)
def set_folder(self, path: str) -> None:
self.sub.setText(f"Bolt can read, edit and run things in {path}")
bolt_console/gui/window.py
+528
View File
@@ -0,0 +1,528 @@
"""The main window and the worker thread a turn runs on.
Turn logic is `console.Session` — the same object the terminal uses — driven
on a QThread with a `Bridge` standing in for the terminal. The window only
ever reacts to Bridge signals, so nothing here knows about HTTP or tools.
"""
from __future__ import annotations
import os
import sys
import threading
from typing import Optional
from PySide6.QtCore import QObject, Qt, QThread, QTimer, Signal, Slot
from PySide6.QtGui import QFont, QKeySequence, QShortcut
from PySide6.QtWidgets import (
QApplication, QFileDialog, QFrame, QHBoxLayout, QLabel, QMainWindow,
QScrollArea, QSizePolicy, QVBoxLayout, QWidget,
)
from .. import __version__, config, dictation, server_client
from ..console import Session
from ..tools import Tools
from . import icons, theme
from .bridge import Bridge
from .chat_list_window import ChatListDialog, ChatSignals, run_in_pool
from .widgets import (
AssistantMessage, Composer, EmptyState, FolderChip, HoldingLine, ModeSwitch,
Notice, StatusDot, ThinkingIndicator, ToolCard, UserMessage,
)
class TurnRunner(QObject):
"""Lives on the worker thread; runs one Session.send per request."""
finished = Signal()
def __init__(self, session: Session):
super().__init__()
self.session = session
@Slot(str)
def run(self, text: str) -> None:
try:
self.session.send(text)
except Exception as exc: # never let a bug kill the worker thread
self.session.ui.error(f"Unexpected error: {exc}")
self.session.ui.stop_spinner()
finally:
self.finished.emit()
class HealthProbe(QObject):
done = Signal(bool, str)
def start(self) -> None:
threading.Thread(target=self._probe, daemon=True).start()
def _probe(self) -> None:
try:
server_client.check_health(timeout=8)
self.done.emit(True, f"Connected to {config.SERVER_URL}")
except Exception as exc:
self.done.emit(False, f"Can't reach {config.SERVER_URL}: {exc}")
class DictationSignals(QObject):
"""Dictation calls back on background threads; these queue it onto the
GUI thread."""
update = Signal(str, str) # final text so far, current interim guess
level = Signal(float) # 0..1 mic loudness
done = Signal(str, str) # text, error ("" when none)
stopping = Signal() # recording over, transcript on its way
class Column(QWidget):
"""Centres its content in a reading column of at most COLUMN_WIDTH."""
def __init__(self, parent=None):
super().__init__(parent)
row = QHBoxLayout(self)
row.setContentsMargins(24, 0, 24, 0)
row.addStretch(1)
self.inner = QWidget()
self.inner.setMaximumWidth(theme.COLUMN_WIDTH)
self.inner.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Preferred)
row.addWidget(self.inner, 100)
row.addStretch(1)
self.layout_ = QVBoxLayout(self.inner)
self.layout_.setContentsMargins(0, 0, 0, 0)
class MainWindow(QMainWindow):
submit_turn = Signal(str)
HINT = "Enter to send · Shift+Enter for a new line · Ctrl+M dictate · Ctrl+N new chat"
def __init__(self, mode: Optional[str] = None, session: Optional[Session] = None,
dictation_factory=None, list_chats=None, create_chat=None):
super().__init__()
# Injectable so tests never hit the network; the real ones run on the pool.
self._list_chats = list_chats or server_client.list_chats
self._create_chat = create_chat or server_client.create_chat
self._dictation_factory = dictation_factory or dictation.Dictation
self._dictation = None
self.setWindowTitle("Bolt")
self.setWindowIcon(icons.app_icon())
self.resize(1040, 800)
self.setMinimumSize(620, 480)
self.bridge = Bridge()
if session is None:
session = Session(self.bridge, tools=Tools(self.bridge, mode=mode or config.APPROVAL_MODE))
self.session = session
self._cards = {}
self._assistant: Optional[AssistantMessage] = None
self._busy = False
self._stick_to_bottom = True
root = QWidget()
root.setObjectName("Root")
self.setCentralWidget(root)
page = QVBoxLayout(root)
page.setContentsMargins(0, 0, 0, 0)
page.setSpacing(0)
page.addWidget(self._build_header())
# transcript
self.scroll = QScrollArea()
self.scroll.setWidgetResizable(True)
self.scroll.setHorizontalScrollBarPolicy(Qt.ScrollBarAlwaysOff)
transcript = QWidget()
transcript.setObjectName("Transcript")
body = QVBoxLayout(transcript)
body.setContentsMargins(0, 18, 0, 18)
self.column = Column()
body.addWidget(self.column)
body.addStretch(1)
self.entries = self.column.layout_
self.entries.setSpacing(6)
self.empty = EmptyState()
self.empty.picked.connect(self._send)
self.entries.addWidget(self.empty)
self.thinking = ThinkingIndicator()
self.thinking.hide()
self.entries.addWidget(self.thinking)
self.scroll.setWidget(transcript)
self.scroll.verticalScrollBar().rangeChanged.connect(self._follow)
self.scroll.verticalScrollBar().valueChanged.connect(self._track_scroll)
page.addWidget(self.scroll, 1)
# composer
dock = Column()
dock.layout_.setContentsMargins(0, 6, 0, 14)
dock.layout_.setSpacing(6)
self.composer = Composer()
self.composer.submitted.connect(self._send)
dock.layout_.addWidget(self.composer)
self.hint = QLabel(self.HINT)
self.hint.setObjectName("Hint")
self.hint.setAlignment(Qt.AlignHCenter)
dock.layout_.addWidget(self.hint)
self._hint_timer = QTimer(self)
self._hint_timer.setSingleShot(True)
self._hint_timer.timeout.connect(lambda: self.hint.setText(self.HINT))
page.addWidget(dock)
QShortcut(QKeySequence("Ctrl+L"), self, activated=self.clear_transcript)
QShortcut(QKeySequence("Ctrl+O"), self, activated=self.choose_folder)
QShortcut(QKeySequence("Ctrl+N"), self, activated=self.start_new_chat)
QShortcut(QKeySequence("Ctrl+K"), self, activated=self.open_chats)
self.chat_signals = ChatSignals()
self.chat_signals.created.connect(lambda session_id: self.switch_chat(session_id, "New chat"))
self.chat_signals.failed.connect(self._new_chat_failed)
self._creating_chat = False
QShortcut(QKeySequence("Ctrl+M"), self, activated=self.toggle_dictation)
QShortcut(QKeySequence("Escape"), self, activated=self.cancel_dictation)
self.dictation_signals = DictationSignals()
self.dictation_signals.update.connect(self.composer.show_dictation)
self.dictation_signals.level.connect(self.composer.mic.set_level)
self.dictation_signals.done.connect(self._dictation_done)
self.dictation_signals.stopping.connect(
lambda: self._dictation is not None and self.composer.mic.set_state("finishing"))
self.composer.mic.clicked.connect(self.toggle_dictation)
self._mic_unavailable = dictation.unavailable_reason()
if self._mic_unavailable:
self.composer.mic.set_state("unavailable", self._mic_unavailable)
self._wire_bridge()
self._start_worker()
self._refresh_folder()
self.health = HealthProbe()
self.health.done.connect(self._health_done)
QTimer.singleShot(0, self.health.start)
self.composer.edit.setFocus()
# ── construction ─────────────────────────────────────────────────────
def _build_header(self) -> QWidget:
header = QFrame()
header.setObjectName("Header")
header.setFixedHeight(56)
row = QHBoxLayout(header)
row.setContentsMargins(18, 0, 18, 0)
row.setSpacing(10)
row.addWidget(icons.IconLabel("bolt", theme.ACCENT_INK, 26, tile=theme.ACCENT))
word = QLabel("Bolt")
word.setObjectName("Wordmark")
row.addWidget(word)
sub = QLabel(f"Console {__version__}")
sub.setObjectName("WordmarkSub")
row.addWidget(sub)
row.addSpacing(6)
self.status = StatusDot()
self.status.set_state("unknown", "Checking the server…")
row.addWidget(self.status)
row.addStretch(1)
self.chats = FolderChip(icon_name="chats", text="Chats")
self.chats.setToolTip("All your chats with Bolt, from every device (Ctrl+K)")
self.chats.clicked.connect(self.open_chats)
row.addWidget(self.chats)
self.new_chat = FolderChip(icon_name="plus", text="New chat")
self.new_chat.setToolTip("Start a fresh conversation (Ctrl+N)\n"
"Fresh history; Bolt still remembers you")
self.new_chat.clicked.connect(self.start_new_chat)
row.addWidget(self.new_chat)
self.folder = FolderChip()
self.folder.clicked.connect(self.choose_folder)
row.addWidget(self.folder)
self.mode = ModeSwitch(self.session.tools.mode)
self.mode.changed.connect(self._set_mode)
row.addWidget(self.mode)
return header
def _wire_bridge(self) -> None:
b = self.bridge
b.busy.connect(self._set_busy)
b.chunk.connect(self._on_chunk)
b.reply_ended.connect(self._end_assistant)
b.reply.connect(self._on_reply)
b.holding_line.connect(lambda text: self._add(HoldingLine(text)))
b.tool_started.connect(self._on_tool_started)
b.tool_output.connect(self._on_tool_output)
b.tool_diff.connect(self._on_tool_diff)
b.approval_needed.connect(self._on_approval)
b.notice.connect(lambda level, text: self._add(Notice(level, text)))
def _start_worker(self) -> None:
self.thread_ = QThread(self)
self.runner = TurnRunner(self.session)
self.runner.moveToThread(self.thread_)
self.submit_turn.connect(self.runner.run)
self.runner.finished.connect(self._turn_finished)
self.thread_.start()
# ── transcript helpers ───────────────────────────────────────────────
def _add(self, widget: QWidget) -> None:
if self.empty.isVisible():
self.empty.hide()
# Always insert above the thinking indicator, which stays last.
self.entries.insertWidget(self.entries.count() - 1, widget)
def _track_scroll(self, value: int) -> None:
bar = self.scroll.verticalScrollBar()
self._stick_to_bottom = value >= bar.maximum() - 40
def _follow(self, _min: int, maximum: int) -> None:
if self._stick_to_bottom:
self.scroll.verticalScrollBar().setValue(maximum)
def clear_transcript(self) -> None:
if self._busy:
return
keep = (self.empty, self.thinking)
for i in reversed(range(self.entries.count())):
widget = self.entries.itemAt(i).widget()
if widget is not None and widget not in keep:
self.entries.takeAt(i)
widget.deleteLater()
self._cards.clear()
self._assistant = None
self.empty.show()
# ── user actions ─────────────────────────────────────────────────────
# ── chats ────────────────────────────────────────────────────────────
def start_new_chat(self) -> None:
"""Ask the server for a fresh chat id, off the UI thread; the switch
happens when it arrives (switch_chat)."""
if self._busy or self._creating_chat:
return
self._creating_chat = True
self.new_chat.setEnabled(False)
self._flash_hint("Starting a new chat...")
run_in_pool("create", self._create_chat, self.chat_signals)
def _new_chat_failed(self, _operation: str, message: str) -> None:
self._creating_chat = False
self.new_chat.setEnabled(not self._busy)
self._flash_hint(f"Couldn't start a new chat: {message}")
def open_chats(self) -> None:
if self._busy:
self._flash_hint("Wait for Bolt to finish before switching chats.")
return
dialog = ChatListDialog(self, list_chats=self._list_chats, create_chat=self._create_chat)
dialog.chosen.connect(self.switch_chat)
self.chat_dialog = dialog # kept for tests; the dialog is modal
dialog.open()
def switch_chat(self, session_id: str, label: str = "") -> None:
"""Point every server call at *session_id* and start the local
transcript over: there's no endpoint for a past chat's messages, so
the window can't show them (Bolt still has them server-side)."""
self._creating_chat = False
self.new_chat.setEnabled(not self._busy)
if self._busy:
return
note = self.session.switch_to(session_id)
self.clear_transcript()
self.chats.setToolTip(f"All your chats with Bolt, from every device (Ctrl+K)\n{note}")
self._flash_hint(f"Now in: {label or session_id}")
def _send(self, text: str) -> None:
text = text.strip()
if not text or self._busy:
return
self._stick_to_bottom = True
self._add(UserMessage(text))
self._assistant = None
self._set_busy(True)
self.submit_turn.emit(text)
# ── dictation ────────────────────────────────────────────────────────
def toggle_dictation(self) -> None:
if self._mic_unavailable:
self._flash_hint(self._mic_unavailable)
return
if self._dictation is not None:
if self.composer.mic.state == "recording":
self.composer.mic.set_state("finishing")
self._dictation.stop()
return
sig = self.dictation_signals
recording = self._dictation_factory(
sig.update.emit, sig.level.emit, lambda text, error: sig.done.emit(text, error or ""),
silence_seconds=config.DICTATION_SILENCE_SECONDS, on_stopping=sig.stopping.emit,
)
self.composer.begin_dictation()
try:
recording.start()
except Exception as exc:
self.composer.end_dictation("")
self._flash_hint(f"Couldn't open the microphone: {exc}")
return
self._dictation = recording
self.composer.mic.set_state("recording")
if not recording.live:
self._flash_hint("Live transcription is unavailable; the words will appear when you stop.")
def cancel_dictation(self) -> None:
if self._dictation is not None:
self._dictation.cancel()
def _dictation_done(self, text: str, error: str) -> None:
self._dictation = None
self.composer.mic.set_state("unavailable" if self._mic_unavailable else "idle",
self._mic_unavailable or "")
self.composer.end_dictation(text)
if error:
self._flash_hint(error)
def _flash_hint(self, text: str, seconds: float = 6.0) -> None:
self.hint.setText(text)
self._hint_timer.start(int(seconds * 1000))
def choose_folder(self) -> None:
if self._busy:
return
path = QFileDialog.getExistingDirectory(self, "Working directory for Bolt", os.getcwd())
if path:
try:
os.chdir(path)
except OSError as exc:
self._add(Notice("error", f"Couldn't switch to {path}: {exc}"))
return
self._refresh_folder()
if not self.empty.isVisible():
self._add(Notice("info", f"Working directory is now {os.getcwd()}"))
def _refresh_folder(self) -> None:
cwd = os.getcwd()
self.folder.set_path(cwd)
self.empty.set_folder(cwd)
def _set_mode(self, mode: str) -> None:
self.session.tools.mode = mode
# ── bridge slots ─────────────────────────────────────────────────────
def _set_busy(self, busy: bool) -> None:
self._busy = busy
self.composer.set_busy(busy)
self.folder.setEnabled(not busy)
self.new_chat.setEnabled(not busy and not self._creating_chat)
self.chats.setEnabled(not busy)
if busy:
self.thinking.start()
else:
self.thinking.stop()
def _turn_finished(self) -> None:
self._set_busy(False)
self._assistant = None
# Pick up a mode switched by "Always allow" on the worker side.
self.mode.set_mode(self.session.tools.mode)
self.composer.edit.setFocus()
def _on_chunk(self, sentence: str) -> None:
if self._assistant is None:
self._assistant = AssistantMessage()
self._add(self._assistant)
self._assistant.append(sentence)
def _end_assistant(self) -> None:
self._assistant = None
def _on_reply(self, text: str) -> None:
self._add(AssistantMessage(text))
self._assistant = None
def _on_tool_started(self, tool_id: int, name: str, argument: str) -> None:
self._assistant = None # text after a tool call starts a new block
card = ToolCard(name, argument)
card.answered.connect(self._on_answer)
self._cards[tool_id] = card
self._add(card)
def _on_tool_output(self, tool_id: int, text: str, error: bool) -> None:
card = self._cards.get(tool_id)
if card is not None:
card.set_result(text, error)
def _on_tool_diff(self, tool_id: int, old: str, new: str) -> None:
card = self._cards.get(tool_id)
if card is not None:
card.set_diff(old, new)
def _on_approval(self, tool_id: int, question: str) -> None:
card = self._cards.get(tool_id)
if card is None:
self.bridge.answer("no")
return
self.thinking.stop()
card.ask(question)
self._stick_to_bottom = True
QTimer.singleShot(0, lambda: self.scroll.ensureWidgetVisible(card, 0, 40))
QApplication.alert(self) # flash the taskbar if the window is behind others
def _on_answer(self, value: str) -> None:
if value == "always":
self.mode.set_mode("auto")
self.thinking.start()
self.bridge.answer(value)
def _health_done(self, ok: bool, detail: str) -> None:
self.status.set_state("ok" if ok else "error", detail)
# ── window chrome ────────────────────────────────────────────────────
def resizeEvent(self, event) -> None:
super().resizeEvent(event)
# The transcript grows from the top; give the empty state the whole
# viewport so its stretches can centre it.
self.empty.setMinimumHeight(max(0, self.scroll.viewport().height() - 40))
def showEvent(self, event) -> None:
super().showEvent(event)
theme.dark_title_bar(self)
def closeEvent(self, event) -> None:
self.cancel_dictation()
self.bridge.close()
self.thread_.quit()
self.thread_.wait(1500)
super().closeEvent(event)
def make_app(argv=None) -> QApplication:
app = QApplication.instance() or QApplication(argv or sys.argv)
app.setApplicationName("Bolt Console")
app.setWindowIcon(icons.app_icon())
app.setStyle("Fusion") # a neutral base, so the stylesheet decides the look
app.setPalette(theme.palette())
font = QFont(theme.ui_font_family())
font.setPointSizeF(10.5)
font.setHintingPreference(QFont.PreferNoHinting)
app.setFont(font)
app.setStyleSheet(theme.stylesheet())
return app
def run(argv=None) -> int:
if sys.platform == "win32":
try: # own taskbar group + icon instead of python.exe's
import ctypes
ctypes.windll.shell32.SetCurrentProcessExplicitAppUserModelID("TheMajesticNetwork.BoltConsole")
except Exception:
pass
argv = list(sys.argv[1:] if argv is None else argv)
app = make_app()
if not config.is_configured():
from PySide6.QtWidgets import QMessageBox
QMessageBox.critical(
None, "Bolt Console",
"Missing " + ", ".join(config.missing_config())
+ ".\n\nCopy .env.example to .env next to the app and fill them in.")
return 2
window = MainWindow(mode="auto" if "--auto" in argv else None)
window.show()
return app.exec()
bolt_pet/relay_json.py → bolt_console/relay_json.py
+9
View File
@@ -56,6 +56,15 @@ _REPAIRS: tuple[tuple[str, Callable[[str], str]], ...] = (
lambda text: text.translate(str.maketrans({"“": '"', "”": '"',
"‘": "'", "’": "'"})),
),
(
"doubled backslashes that weren't valid JSON escapes",
# {"path": "C:\Users\x"} -> {"path": "C:\\Users\\x"} — a Windows path
# typed raw. Only *invalid* escapes are touched, and `\\` pairs are
# consumed whole, so nothing that already parsed can change meaning.
lambda text: re.sub(
r'\\([\\/"bfnrtu])|\\',
lambda m: m.group(0) if m.group(1) else "\\\\", text),
),
(
"removed a stray quote after a bare value",
# {"recursive":false"} -> {"recursive":false}
bolt_console/server_client.py
+270
View File
@@ -0,0 +1,270 @@
"""HTTP client for Bolt's desk API (ai/desk_api.py on the server).
Protocol is identical to desk_client/bolt_desk.py in the main tmn-api repo —
the console is just another desk client, so it gets the exact same brain,
memory, tools, and persona as Discord chat and the other desk clients:
text -> POST /desk/converse (or /desk/converse_stream)
[server may relay a command back to run on THIS machine]
... -> POST /desk/tool_result (repeat until the server sends a reply)
reply <- returned to caller
Kept dependency-free beyond `requests` so it's easy to unit test with mocks.
"""
from __future__ import annotations
import json
import os
import subprocess
from typing import Callable, NamedTuple, Optional
import requests
from . import config
_MAX_RELAY_HOPS = 16
_MAX_OUTPUT_CHARS = 6000
class ServerError(Exception):
"""Raised when the server responds with an error payload or unreachable."""
class Reply(NamedTuple):
"""One final reply from the desk API."""
text: str
# True when the text already reached the screen sentence by sentence while
# it streamed in — the caller must not print it a second time.
streamed: bool = False
def _headers() -> dict:
return {"X-Desk-Api-Key": config.API_KEY}
def check_health(timeout: float = 10.0) -> dict:
response = requests.get(f"{config.SERVER_URL}/desk/health", headers=_headers(), timeout=timeout)
response.raise_for_status()
return response.json()
def run_local_command(command: str, timeout: Optional[int] = None) -> str:
"""Execute a relayed shell command as the current user, in the console's
working directory (see /cd). Output is captured for the server, so stdin
is closed rather than inherited — a command waiting on input would
otherwise hang on a prompt nobody can see. `sudo` still works: it asks on
the terminal itself, not on stdin."""
timeout = timeout or config.COMMAND_TIMEOUT_SECONDS
try:
completed = subprocess.run(
command, shell=True, capture_output=True, text=True, errors="replace",
stdin=subprocess.DEVNULL, timeout=timeout, cwd=os.getcwd(),
# From the window (pythonw, no console) every cmd.exe would
# otherwise flash a console window. Output is captured either way.
creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0),
)
output = (completed.stdout or "") + (completed.stderr or "")
return f"[exit {completed.returncode}]\n{output}"[:_MAX_OUTPUT_CHARS]
except subprocess.TimeoutExpired:
return f"[command timed out after {timeout}s]"
except Exception as exc:
return f"[command failed: {exc}]"
def _reply_from(payload: dict, streamed: bool = False) -> Reply:
if payload.get("type") == "reply":
return Reply(text=str(payload.get("text") or ""), streamed=streamed)
raise ServerError(str(payload.get("error") or "unknown server response"))
def _relay(payload: dict, on_command, on_say) -> dict:
"""Run every command the server hands back, posting each output to
/desk/tool_result, until the payload is no longer a command."""
for _ in range(_MAX_RELAY_HOPS):
if payload.get("type") != "command":
break
holding = str(payload.get("say") or "").strip()
if holding and on_say is not None:
# Shown *before* the command runs — that's the point of it.
try:
on_say(holding)
except Exception:
pass # a failed acknowledgement must not cost the tool call
output = on_command(str(payload.get("command") or ""))
try:
response = requests.post(
f"{config.SERVER_URL}/desk/tool_result",
json={"session_id": config.SESSION_ID,
"token": payload.get("token"), "output": output},
headers=_headers(), timeout=180,
)
payload = response.json()
except Exception as exc:
raise ServerError(f"couldn't reach the server during tool relay: {exc}") from exc
return payload
def converse(
text: str,
on_command: Callable[[str], str] = run_local_command,
timeout: float = 120.0,
on_say: Optional[Callable[[str], None]] = None,
) -> Reply:
"""Send one turn of conversation, relaying any commands the server sends
back until it produces a final reply.
*on_say* receives the short holding line ("give me a sec") the server can
send alongside a command."""
try:
response = requests.post(
f"{config.SERVER_URL}/desk/converse",
json={"session_id": config.SESSION_ID, "text": text},
headers=_headers(), timeout=timeout,
)
payload = response.json()
except Exception as exc:
raise ServerError(f"couldn't reach the server: {exc}") from exc
return _reply_from(_relay(payload, on_command, on_say))
def converse_stream(
text: str,
on_say: Callable[[str], None],
on_command: Callable[[str], str] = run_local_command,
timeout: float = 180.0,
) -> Reply:
"""Same turn as converse(), but handing each sentence to *on_say* as it
arrives, so the wait is time-to-first-sentence.
Raises ServerError if the stream fails before anything was shown — the
caller then retries with converse(). Once a sentence *has* been shown,
a late failure ends the turn with what was said rather than repeating it.
"""
shown_anything = False
said = []
try:
response = requests.post(
f"{config.SERVER_URL}/desk/converse_stream",
json={"session_id": config.SESSION_ID, "text": text},
headers=_headers(), timeout=timeout, stream=True,
)
response.raise_for_status()
for raw in response.iter_lines(decode_unicode=True):
if not raw:
continue
try:
event = json.loads(raw)
except (TypeError, ValueError):
continue
kind = str(event.get("type") or "")
if kind == "say":
line = str(event.get("text") or "").strip()
if line:
shown_anything = True
said.append(line)
on_say(line)
elif kind == "command":
# The tool loop is request/response, so the rest of the turn
# finishes through the ordinary relay rather than the stream.
response.close()
return _reply_from(_relay(event, on_command, on_say))
elif kind == "reply":
if event.get("already_spoken"):
return _reply_from(event, streamed=True)
return _unsaid(_reply_from(event), said)
elif kind == "error":
raise ServerError(str(event.get("error") or "stream failed"))
except ServerError:
raise
except Exception as exc:
if shown_anything:
return Reply(text="", streamed=True)
raise ServerError(f"streaming failed: {exc}") from exc
raise ServerError("stream ended without a reply")
def _unsaid(reply: Reply, said: list) -> Reply:
"""The part of *reply* not already shown as streamed sentences.
Seen live: the server streams the sentences, then sends the whole text
again in the final reply *without* `already_spoken`, so everything showed
twice. Compared ignoring whitespace, since the stream splits on sentences
and drops the line breaks the full text keeps."""
if not said:
return reply
target = len("".join("".join(said).split())) # non-whitespace characters shown
seen, cut = 0, None
for index, char in enumerate(reply.text):
if not char.isspace():
seen += 1
if seen == target:
cut = index + 1
break
if cut is None or "".join(reply.text[:cut].split()) != "".join("".join(said).split()):
return reply # a different text, not a repeat: show all of it
rest = reply.text[cut:].strip()
return Reply(text=rest, streamed=not rest)
def list_chats(timeout: float = 15.0) -> list:
"""This user's conversations across every client (console, pet, phone),
newest first — each {session_id, label, last_message, last_active}. A
chat only appears once it's been used, so a just-minted one isn't here."""
try:
response = requests.get(
f"{config.SERVER_URL}/desk/chats", headers=_headers(), timeout=timeout,
)
response.raise_for_status()
return list(response.json().get("chats") or [])
except Exception as exc:
raise ServerError(f"couldn't list chats: {exc}") from exc
def create_chat(timeout: float = 15.0) -> str:
"""Mint a brand-new, never-used session id on the server. Bolt's memory is
scoped to the user, not the session, so a new chat is fresh history, not
a memory reset."""
try:
response = requests.post(
f"{config.SERVER_URL}/desk/chats", headers=_headers(), timeout=timeout,
)
response.raise_for_status()
session_id = str(response.json().get("session_id") or "")
except Exception as exc:
raise ServerError(f"couldn't create a chat: {exc}") from exc
if not session_id:
raise ServerError("the server didn't return a session_id for the new chat")
return session_id
def list_outbox_files(timeout: float = 15.0) -> list:
"""Files the server has queued for this session via its deliver_files
tool — each entry has id/name/size. Downloading one dequeues it
server-side, so a file is only ever handed out once."""
try:
response = requests.get(
f"{config.SERVER_URL}/desk/files",
params={"session_id": config.SESSION_ID},
headers=_headers(), timeout=timeout,
)
response.raise_for_status()
return list(response.json().get("files") or [])
except Exception as exc:
raise ServerError(f"couldn't list delivered files: {exc}") from exc
def download_outbox_file(file_id: str, timeout: float = 60.0) -> bytes:
"""Fetches and dequeues one file listed by list_outbox_files()."""
try:
response = requests.get(
f"{config.SERVER_URL}/desk/files/{file_id}",
params={"session_id": config.SESSION_ID},
headers=_headers(), timeout=timeout,
)
response.raise_for_status()
return response.content
except Exception as exc:
raise ServerError(f"couldn't download delivered file {file_id!r}: {exc}") from exc
bolt_console/tools.py
+201
View File
@@ -0,0 +1,201 @@
"""What happens to a command the server relays back to this machine.
The desk API has one tool channel — a `command` string — and this module is
where the console decides what that string means:
filectl <json> file list/read/write/edit (file_ops.py), no shell
petctl / dialoguectl the pet's body and voice — answered with an
explanation instead of being run, since there is
no pet here and `petctl` isn't a real program
anything else a real shell command, in the console's cwd
Reading never asks. Anything that changes the machine (a shell command, a
write, an edit) asks first unless the session is in auto mode — the pet ran
everything unasked because nobody was sitting at a prompt to answer; a
console has someone right there, so the default is to use them.
"""
from __future__ import annotations
import re
from typing import Callable
from . import file_ops, server_client
# Pseudo-commands from the pet's vocabulary. The server may still emit them —
# the session is a pet session as far as it knows — so they get an answer the
# model can act on rather than "'petctl' is not recognized as a command".
_PET_PREFIXES = ("petctl", "bolt-pet", "dialoguectl")
PET_UNAVAILABLE = (
"[console] {name} isn't available: this is Bolt Console, a text-only terminal "
"client with no pet body, voice, or screen reading. Use filectl for files and "
"ordinary shell commands for everything else."
)
DECLINED = (
"[console] the user declined this {what}, so it did not run. Don't retry it "
"as-is; ask what they'd like instead."
)
def is_pet_command(command: str) -> bool:
parts = (command or "").split(None, 1)
return bool(parts) and parts[0].lower() in _PET_PREFIXES
# The model sometimes writes the tool marker's own label into its argument
# ("command: powershell -Command ...") or wraps the whole thing in backticks.
# Neither is ever part of a real command — cmd.exe answers "'command:' is not
# recognized" — so they're stripped, and the model is told, like relay_json's
# repairs: a silent fix works today and guarantees the same mistake tomorrow.
_LABEL = re.compile(r"^(?:command|cmd)\s*:\s*", re.IGNORECASE)
def clean_command(command: str):
"""Returns (command, repairs) with a stray marker label and a wrapping
pair of backticks removed."""
command = (command or "").strip()
repairs = []
stripped = _LABEL.sub("", command, count=1)
if stripped != command:
repairs.append('removed a leading "command:" label')
command = stripped.strip()
if len(command) > 2 and command[0] == command[-1] == "`" and "`" not in command[1:-1]:
repairs.append("removed wrapping backticks")
command = command[1:-1].strip()
return command, repairs
def repair_note(repairs) -> str:
if not repairs:
return ""
return (
"\n(note: your command was malformed — I " + "; ".join(repairs)
+ " and ran it anyway. Send only the command itself next time.)"
)
class Tools:
"""The `on_command` callable handed to server_client.converse*.
*ui* is a ui.Terminal (or anything with tool_call/tool_result/diff/
confirm); *run_shell* is injectable so tests never spawn a process."""
def __init__(
self,
ui,
mode: str = "ask",
run_shell: Callable[[str], str] = server_client.run_local_command,
):
self.ui = ui
self.mode = "auto" if mode == "auto" else "ask"
self.run_shell = run_shell
# Commands that actually reached this machine — the console compares
# it with what a reply *claims* happened (see console.fabricated).
self.calls = 0
def __call__(self, command: str) -> str:
return self.handle(command)
def handle(self, command: str) -> str:
self.calls += 1
command, repairs = clean_command(command)
if not command:
return "[console] empty command"
return self._dispatch(command) + repair_note(repairs)
def _dispatch(self, command: str) -> str:
if is_pet_command(command):
name = command.split(None, 1)[0]
self.ui.tool_call("Pet", command)
output = PET_UNAVAILABLE.format(name=name)
self.ui.tool_result("not available in the console", error=True)
return output
try:
action = file_ops.parse(command)
except file_ops.FileOpError as exc:
self.ui.tool_call("File", command)
self.ui.tool_result(str(exc), error=True)
return f"[filectl] {exc}"
if action is not None:
return self._file(action)
return self._shell(command)
# ── approval ─────────────────────────────────────────────────────────
def _approved(self, question: str) -> bool:
if self.mode == "auto":
return True
answer = self.ui.confirm(question)
if answer == "always":
self.mode = "auto"
return True
return answer == "yes"
# ── shell ────────────────────────────────────────────────────────────
def _shell(self, command: str) -> str:
self.ui.tool_call("Shell", command)
if not self._approved("Run this command?"):
self.ui.tool_result("declined", error=True)
return DECLINED.format(what="command")
output = self.run_shell(command)
status, _, body = output.partition("\n")
failed = status != "[exit 0]"
# A clean exit shows just the output; a failure keeps its status line.
shown = body.strip() or status
if failed and body.strip():
shown = f"{status}\n{body.strip()}"
self.ui.tool_result(shown, error=failed)
return output
# ── filectl ──────────────────────────────────────────────────────────
def _file(self, action: dict) -> str:
kind = action["action"]
path = action.get("path", "")
if kind == "help":
return file_ops.execute(action)
if kind == "list":
label = path if action["pattern"] == "*" else f"{path}, {action['pattern']}"
self.ui.tool_call("List", label + (", recursive" if action["recursive"] else ""))
elif kind == "read":
span = ""
if action.get("start") or action.get("end"):
span = f", lines {action.get('start') or 1}-{action.get('end') or 'end'}"
self.ui.tool_call("Read", path + span)
elif kind == "write":
lines = action["content"].count("\n") + 1
self.ui.tool_call("Write", path)
self.ui.tool_result(f"{lines} lines, {len(action['content'])} chars")
if not self._approved("Write this file?"):
return self._declined("write")
elif kind == "edit":
self.ui.tool_call("Edit", path)
self.ui.diff(action["old"], action["new"])
if not self._approved("Make this edit?"):
return self._declined("edit")
try:
output = file_ops.execute(action)
except file_ops.FileOpError as exc:
self.ui.tool_result(str(exc), error=True)
return f"[filectl] {exc}"
if kind == "read":
count = output.count("\n") + 1 if output != "[empty file]" else 0
self.ui.tool_result(f"Read {count} lines")
elif kind == "list":
self.ui.tool_result(output)
else:
self.ui.tool_result(output.replace("[filectl] ", ""))
return output
def _declined(self, what: str) -> str:
self.ui.tool_result("declined", error=True)
return DECLINED.format(what=what)
bolt_console/ui.py
+257
View File
@@ -0,0 +1,257 @@
"""Terminal rendering: colour, the thinking spinner, tool-call lines, prompts.
Plain ANSI and the standard library only, so the console has no dependency
beyond `requests`. Everything writes through `Terminal._emit`, which holds the
same lock the spinner thread draws under — that is what keeps a spinner frame
from landing in the middle of a line of output.
"""
from __future__ import annotations
import itertools
import os
import sys
import threading
import time
from typing import Callable, List, Optional, TextIO
_RESET = "\033[0m"
_STYLES = {
"bold": "\033[1m",
"dim": "\033[2m",
"red": "\033[31m",
"green": "\033[32m",
"yellow": "\033[33m",
"blue": "\033[34m",
"magenta": "\033[35m",
"cyan": "\033[36m",
}
# Output lines shown under a tool call before collapsing to "… +N lines".
TOOL_PREVIEW_LINES = 4
def enable_windows_ansi() -> None:
"""Turn on VT escape processing for the Windows console (off by default
for older hosts). Harmless everywhere else."""
if os.name != "nt":
return
try:
import ctypes
kernel32 = ctypes.windll.kernel32
handle = kernel32.GetStdHandle(-11) # STD_OUTPUT_HANDLE
mode = ctypes.c_uint32()
if kernel32.GetConsoleMode(handle, ctypes.byref(mode)):
kernel32.SetConsoleMode(handle, mode.value | 0x0004) # ENABLE_VIRTUAL_TERMINAL_PROCESSING
except Exception:
pass
def _supports_unicode(stream: TextIO) -> bool:
encoding = (getattr(stream, "encoding", "") or "").lower().replace("-", "")
return encoding.startswith("utf")
class Glyphs:
def __init__(self, unicode: bool):
self.bullet = "●" if unicode else "*"
self.elbow = "⎿" if unicode else "L"
self.ellipsis = "…" if unicode else "..."
self.prompt = "›" if unicode else ">"
self.spinner = "⠋⠙⠹⠸⠼⠴⠦⠧⠇⠏" if unicode else "|/-\\"
def preview(text: str, limit: int = TOOL_PREVIEW_LINES) -> List[str]:
"""The first *limit* lines of *text*, plus a count of what was cut."""
lines = text.rstrip("\n").splitlines() or [""]
if len(lines) <= limit:
return lines
return lines[:limit] + [f"... +{len(lines) - limit} lines"]
class Terminal:
def __init__(
self,
out: Optional[TextIO] = None,
color: bool = True,
input_fn: Callable[[str], str] = input,
animate: Optional[bool] = None,
):
self.out = out or sys.stdout
tty = bool(getattr(self.out, "isatty", lambda: False)())
self.color = color and tty
self.animate = tty if animate is None else animate
self.glyphs = Glyphs(_supports_unicode(self.out))
self._input = input_fn
self._lock = threading.RLock()
self._spinner_label = ""
self._spinner_started = 0.0
self._spinner_line = False # a spinner frame is currently on screen
self._spinner_thread: Optional[threading.Thread] = None
self._spinner_stop = threading.Event()
self._spinner_paused = False
self._in_reply = False # streamed sentences continue one bulleted block
# ── primitives ───────────────────────────────────────────────────────
def style(self, text: str, *names: str) -> str:
if not self.color or not names:
return text
return "".join(_STYLES[n] for n in names) + text + _RESET
def _clear_spinner_line(self) -> None:
if self._spinner_line:
self.out.write("\r\033[2K" if self.color else "\r" + " " * 60 + "\r")
self._spinner_line = False
def _emit(self, text: str) -> None:
with self._lock:
self._clear_spinner_line()
self.out.write(text)
self.out.flush()
def line(self, text: str = "") -> None:
self._emit(text + "\n")
# ── spinner ──────────────────────────────────────────────────────────
def start_spinner(self, label: str = "Thinking") -> None:
self._spinner_label = label
self._spinner_started = time.monotonic()
self._spinner_paused = False
if not self.animate or self._spinner_thread is not None:
return
self._spinner_stop.clear()
self._spinner_thread = threading.Thread(target=self._spin, daemon=True)
self._spinner_thread.start()
def stop_spinner(self) -> None:
thread = self._spinner_thread
if thread is not None:
self._spinner_stop.set()
thread.join(timeout=1)
self._spinner_thread = None
with self._lock:
self._clear_spinner_line()
self.out.flush()
def _spin(self) -> None:
for frame in itertools.cycle(self.glyphs.spinner):
if self._spinner_stop.wait(0.1):
return
with self._lock:
if self._spinner_paused:
continue
elapsed = int(time.monotonic() - self._spinner_started)
text = f"{frame} {self._spinner_label}{self.glyphs.ellipsis} ({elapsed}s, ctrl+c to interrupt)"
self.out.write("\r" + self.style(text, "magenta"))
self.out.flush()
self._spinner_line = True
# ── conversation ─────────────────────────────────────────────────────
def _bulleted(self, text: str, *bullet_style: str, body_style=()) -> None:
self._in_reply = False
lines = text.splitlines() or [""]
first, rest = lines[0], lines[1:]
out = [self.style(self.glyphs.bullet, *bullet_style) + " " + self.style(first, *body_style)]
out += [" " + self.style(line, *body_style) for line in rest]
self.line("\n".join(out))
def assistant(self, text: str) -> None:
if text.strip():
self._bulleted(text.strip())
self.line()
def assistant_chunk(self, sentence: str) -> None:
"""One streamed sentence. The first opens a bulleted block and the
rest continue it on their own lines — never mid-line, because the
spinner redraws by erasing the current line."""
sentence = sentence.strip()
if not sentence:
return
if self._in_reply:
self.line("\n".join(" " + part for part in sentence.splitlines()))
else:
self._bulleted(sentence)
self._in_reply = True
def end_reply(self) -> None:
self._in_reply = False
self.line()
def holding(self, text: str) -> None:
self._bulleted(text.strip(), "dim", body_style=("dim",))
def tool_call(self, name: str, argument: str) -> None:
argument = argument if len(argument) <= 160 else argument[:157] + "..."
self._bulleted(self.style(name, "bold") + self.style(f"({argument})", "dim"), "green")
def tool_result(self, text: str, error: bool = False) -> None:
lines = preview(text)
elbow = self.glyphs.elbow
rendered = [f" {elbow} " + self.style(lines[0], "red" if error else "dim")]
rendered += [" " + self.style(line, "red" if error else "dim") for line in lines[1:]]
self.line("\n".join(rendered))
def diff(self, old: str, new: str, limit: int = 12) -> None:
rows = []
for sign, text, colour in (("-", old, "red"), ("+", new, "green")):
lines = text.splitlines() or [""]
for line in lines[:limit]:
rows.append(" " + self.style(f"{sign} {line}", colour))
if len(lines) > limit:
rows.append(" " + self.style(f" ... +{len(lines) - limit} lines", "dim"))
self.line("\n".join(rows))
def info(self, text: str) -> None:
self.line(self.style(text, "dim"))
def warn(self, text: str) -> None:
self.line(self.style(text, "yellow"))
def error(self, text: str) -> None:
self.line(self.style(text, "red"))
# ── input ────────────────────────────────────────────────────────────
def ask(self, prompt: str) -> str:
"""Read one line with the spinner held off the screen. EOF and
Ctrl+C both come back as an empty answer — the caller decides what
an empty answer means."""
with self._lock:
self._spinner_paused = True
self._clear_spinner_line()
self.out.flush()
try:
return self._input(prompt)
except (EOFError, KeyboardInterrupt):
self.line()
return ""
finally:
with self._lock:
self._spinner_paused = False
self._spinner_started = time.monotonic()
def confirm(self, question: str) -> str:
"""y / n / a(lways). Anything unrecognised — including Ctrl+C — is no."""
answer = self.ask(
" " + self.style(question, "yellow") + " "
+ self.style("[y]es / [n]o / [a]lways this session", "dim") + " "
).strip().lower()
if answer in ("y", "yes"):
return "yes"
if answer in ("a", "always"):
return "always"
return "no"
def read_message(self) -> str:
"""The main prompt. A trailing backslash continues onto the next line.
Raises EOFError/KeyboardInterrupt for the REPL to handle."""
prompt = self.style(self.glyphs.prompt + " ", "bold", "cyan")
lines = [self._input(prompt)]
while lines[-1].endswith("\\"):
lines[-1] = lines[-1][:-1]
lines.append(self._input(self.style(self.glyphs.ellipsis + " ", "dim")))
return "\n".join(lines)
bolt_pet/__init__.py
-7
View File
@@ -1,7 +0,0 @@
"""Bolt desktop pet.
__version__ is what the auto-updater compares against the newest tag on the
Gitea releases page (see updater.py), so bump it in the same commit you tag.
"""
__version__ = "0.2.3"
bolt_pet/__main__.py
-15
View File
@@ -1,15 +0,0 @@
"""Entry point: python -m bolt_pet [--doctor [--deep]]"""
import sys
if __name__ == "__main__":
if "--doctor" in sys.argv:
# Imported inside the branch, not at module scope: the doctor exists to
# diagnose installs where importing the UI would itself blow up.
from .doctor import main as doctor
sys.exit(doctor(sys.argv[1:]))
from .ui.app import run
sys.exit(run())
bolt_pet/assets/sprites/README.md
-92
View File
@@ -1,92 +0,0 @@
# Sprite assets
Art: Bolt himself — a cream shepherd pup with a slate cap, a lightning blaze
on his forehead and a bolt tag on his collar. The frames are **generated, not
hand-drawn**: `scripts/generate_bolt_sprites.py` draws every one of them with
Pillow and writes this folder.
```bash
python scripts/generate_bolt_sprites.py # rewrite this folder
python scripts/generate_bolt_sprites.py --out /tmp/prev # preview elsewhere first
python scripts/generate_bolt_sprites.py --states idle # just one state
```
That means tweaking the art is editing code, not 24 PNGs: the palette is a
block of constants at the top of the script, the body/head/ear/tail shapes are
one function each in normalised 0..1 coordinates, and each state's animation is
a list of pose dicts in `frames_for()`. Everything is super-sampled 4x and
downscaled on save, because PIL's draw primitives have no antialiasing.
**Regenerate after editing** — the PNGs here are committed, so a change to the
script alone doesn't move the pet.
Convention the loader (`bolt_pet/ui/sprite.py`) expects:
```
assets/sprites/
idle/ frame_00..07.png breathing, tail wag, blink on frame 06
listening/ frame_00..03.png ears perked, head tilted in, collar tag lit, sound arcs
thinking/ frame_00..05.png eyes up, head cocked, cycling dots
talking/ frame_00..03.png mouth open/close with tongue, ears bouncing
error/ frame_00..01.png X eyes, ears drooped, red spark
walk/ frame_00..07.png side-view walk cycle (see below)
```
- One subfolder per pet state (matches `bolt_pet.state.PetState`), **plus
`walk/`**, which is not a state — see below.
- Any `*.png` filenames work — they're played back in alphabetical-sort
order, looping, at `IDLE_ANIMATION_FPS` (see `.env`). At the default 6fps
the 8-frame idle loop runs about 1.3s.
- Frames are square (320px, 2x the default `PET_SIZE` of 160) so they
downscale cleanly; the loader scales to fit `PET_SIZE` keeping aspect ratio
and centres them in the square pet window.
- A state directory with no frames in it falls back to a small
procedurally-drawn placeholder blob (see `_placeholder_frames` in
`sprite.py`).
## The walk cycle
`walk/` is the one animation that isn't a `PetState`. Walking is a property of
*movement* — orthogonal to whether he's idle, listening or talking — so it
stays out of the state machine and is keyed by name instead
(`sprite.EXTRA_ANIMATIONS`). `PetWindow` uses it whenever the pet is actually
travelling and falls back to the state animation the moment it stops.
Three things about it are load-bearing if you redraw it:
- **It's a side view, drawn facing right.** The other poses are a
front-facing sit, which is fine standing still but slides like a chess
piece when moving. `PetWindow._oriented()` mirrors the frames (cached) when
he walks left, so only the right-facing version exists on disk.
- **The cycle is advanced by distance travelled, not by the animation
timer** (`_WALK_PIXELS_PER_FRAME`, one frame per ~13px). That's what keeps
a planted paw tracking backwards at exactly the speed the window moves
forwards. Drive it off the clock and the feet skate whenever
`PET_WANDER_SPEED` doesn't happen to match `IDLE_ANIMATION_FPS`. If you
change the number of frames or the stride length in
`paw_position()`, retune that constant to match or he'll moonwalk.
- **The frames carry their own vertical bob**, so the window's own bob is
switched off while they're in use. Only the no-walk-art fallback still
bobs in code.
Delete `walk/` and everything still runs — he reverts to sliding with a small
coded bob, which is what the pet did before the cycle existed.
## Swapping in different art
Replace any state's PNGs (same alphabetical-order-loops convention) to
change its look — no code changes needed, and nothing forces you to keep
using the generator. If your source is a single grid spritesheet (rows/cols
of frames in one PNG) rather than one-file-per-frame, use
`scripts/slice_spritesheet.py` to cut it into this folder-of-frames
convention:
```bash
python scripts/slice_spritesheet.py path/to/idle_sheet.png assets/sprites/idle \
--cols 6 --rows 1
```
If your format is something else entirely (a single animated GIF/APNG, a
Spine/DragonBones skeletal export, an Aseprite `.json` atlas, etc.) — tell me
the format and I'll adapt `sprite.py`'s loader rather than making you convert
by hand.
bolt_pet/assets/sprites/error/frame_00.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 55 KiB

bolt_pet/assets/sprites/error/frame_01.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 54 KiB

bolt_pet/assets/sprites/idle/frame_00.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/idle/frame_01.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/idle/frame_02.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/idle/frame_03.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/idle/frame_04.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/idle/frame_05.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/idle/frame_06.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 56 KiB

bolt_pet/assets/sprites/idle/frame_07.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/listening/frame_00.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 65 KiB

bolt_pet/assets/sprites/listening/frame_01.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 64 KiB

bolt_pet/assets/sprites/listening/frame_02.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 64 KiB

bolt_pet/assets/sprites/listening/frame_03.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 63 KiB

bolt_pet/assets/sprites/listening/frame_04.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 63 KiB

bolt_pet/assets/sprites/listening/frame_05.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 64 KiB

bolt_pet/assets/sprites/talking/frame_00.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/talking/frame_01.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 59 KiB

bolt_pet/assets/sprites/talking/frame_02.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 59 KiB

bolt_pet/assets/sprites/talking/frame_03.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/talking/frame_04.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 58 KiB

bolt_pet/assets/sprites/talking/frame_05.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 57 KiB

bolt_pet/assets/sprites/thinking/frame_00.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 61 KiB

bolt_pet/assets/sprites/thinking/frame_01.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 61 KiB

bolt_pet/assets/sprites/thinking/frame_02.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 62 KiB

bolt_pet/assets/sprites/thinking/frame_03.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 62 KiB

bolt_pet/assets/sprites/thinking/frame_04.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 59 KiB

bolt_pet/assets/sprites/thinking/frame_05.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 61 KiB

bolt_pet/assets/sprites/walk/frame_00.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 46 KiB

bolt_pet/assets/sprites/walk/frame_01.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 46 KiB

bolt_pet/assets/sprites/walk/frame_02.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 43 KiB

bolt_pet/assets/sprites/walk/frame_03.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 43 KiB

bolt_pet/assets/sprites/walk/frame_04.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 46 KiB

bolt_pet/assets/sprites/walk/frame_05.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 45 KiB

bolt_pet/assets/sprites/walk/frame_06.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 42 KiB

bolt_pet/assets/sprites/walk/frame_07.png
BIN
View File
Binary file not shown.
Side by Side

Before

Width:  |  Height:  |  Size: 43 KiB

bolt_pet/audio/__init__.py
View File
bolt_pet/audio/barge_in.py
-190
View File
@@ -1,190 +0,0 @@
"""Barge-in: notice that the user wants to interrupt *while the pet is
talking* so playback can be cut short mid-sentence.
Two detectors, picked by BARGE_IN_MODE:
- **wake** (default) — the interruption has to be the wake word. Every mic
frame goes through the same openWakeWord model the idle listener uses, so
a sneeze, a door, or the TV can't cut Bolt off mid-sentence; only saying
"thunderbolt" does.
- **energy** — the original behaviour: N consecutive frames above
BARGE_IN_RMS_THRESHOLD. Faster to trigger and needs no model inference,
but it fires on any sustained noise. Deliberately dumber than the
utterance VAD in mic.py, since the mic hears the pet's own voice coming
back out of the speakers, so the threshold defaults to 4x the VAD one.
Both take the same injectable stream shape as mic.record_utterance and expose
the same reset()/check() pair, so tests feed them fake frames instead of real
audio hardware and controller.py doesn't care which one it holds.
"""
from __future__ import annotations
from typing import Callable, Optional, Union
import numpy as np
from .. import config
from .mic import AudioStream, rms
from .wake_word import WakeModel, _default_model, hard_reset
def _read_frame(stream: AudioStream, frame_len: int) -> Optional[np.ndarray]:
"""One mono frame, or None if the mic hiccuped or gave us nothing. Never
raises: a bad frame mid-playback should mean "no barge-in this frame",
not a dead reply."""
try:
chunk, _ = stream.read(frame_len)
except Exception:
return None
frame = np.asarray(chunk)
if frame.ndim > 1:
frame = frame[:, 0]
return frame if frame.size else None
class BargeInDetector:
"""Energy mode. Poll-driven: call check() repeatedly while audio plays.
Each call consumes exactly one mic frame (80ms at the default frame
length), which is also what paces the playback loop's polling."""
def __init__(
self,
stream: AudioStream,
threshold: int = None,
required_frames: int = None,
frame_len: int = config.FRAME_LEN,
):
self._stream = stream
self._threshold = config.BARGE_IN_RMS_THRESHOLD if threshold is None else threshold
self._required = max(1, config.BARGE_IN_FRAMES if required_frames is None else required_frames)
self._frame_len = frame_len
self._loud_frames = 0
@property
def loud_frames(self) -> int:
return self._loud_frames
def reset(self) -> None:
self._loud_frames = 0
def check(self) -> bool:
"""True once the user has been loud for long enough to count as an
interruption."""
frame = _read_frame(self._stream, self._frame_len)
if frame is None:
return False
if rms(frame) >= self._threshold:
self._loud_frames += 1
else:
self._loud_frames = 0 # a single thump/cough shouldn't count
return self._loud_frames >= self._required
class WakeWordBargeIn:
"""Wake-word mode: only "thunderbolt" interrupts.
Same per-frame predict() loop as listen_for_wake_word, just driven by the
playback poll instead of its own read loop. The model instance is shared
with the idle listener by default — the two never run at the same time
(the pipeline is either speaking or listening), and reusing it avoids
loading a second copy of the ONNX graph.
Two wrinkles the energy detector doesn't have:
- The mic hears the pet's own voice, so the model is scoring Bolt's
speech too. That's harmless unless Bolt says its own wake word, which
is why the threshold can be raised independently
(BARGE_IN_WAKE_THRESHOLD) without desensitizing the idle listener.
- reset() has to be a *hard* reset. openwakeword keeps ~10s of audio
history in its preprocessor, so the "thunderbolt" that started this
turn is still in the model's window when playback begins — feed it one
new frame and it fires on the old phrase, cutting the reply off a word
in. Clearing that window is what makes wake-mode barge-in work at all.
"""
def __init__(
self,
stream: AudioStream,
model: Optional[WakeModel] = None,
threshold: Union[float, Callable[[], float], None] = None,
frame_len: int = config.FRAME_LEN,
on_score: Optional[Callable[[float, float], None]] = None,
):
self._stream = stream
self._model = model if model is not None else _default_model
if threshold is None:
threshold = config.BARGE_IN_WAKE_THRESHOLD or config.WAKE_WORD_THRESHOLD
self._resolve_threshold = threshold if callable(threshold) else (lambda: threshold)
self._frame_len = frame_len
self._on_score = on_score
self._frames = 0
self._peak = 0.0
self._last = 0.0
self._last_threshold = 0.0
# Scoring history for the current reply. Without this an interruption is
# indistinguishable from a crash in the logs — you can't tell a genuine
# "thunderbolt" from the model firing on Bolt's own voice, or on the first
# frame (a stale window) versus halfway through (something it heard).
@property
def frames_checked(self) -> int:
return self._frames
@property
def seconds_checked(self) -> float:
return self._frames * self._frame_len / config.SAMPLE_RATE
@property
def peak_score(self) -> float:
return self._peak
@property
def last_score(self) -> float:
return self._last
@property
def last_threshold(self) -> float:
return self._last_threshold
def reset(self) -> None:
hard_reset(self._model) # never raises
self._frames = 0
self._peak = 0.0
self._last = 0.0
def check(self) -> bool:
frame = _read_frame(self._stream, self._frame_len)
if frame is None:
return False
try:
scores = self._model.predict(frame)
except Exception:
return False # same contract as a mic hiccup: no barge-in, no crash
threshold = self._resolve_threshold()
best = max(scores.values()) if scores else 0.0
self._frames += 1
self._last = best
self._peak = max(self._peak, best)
self._last_threshold = threshold
if self._on_score is not None:
self._on_score(best, threshold)
if scores and best >= threshold:
self.reset()
return True
return False
def make_detector(
stream: AudioStream,
mode: str = None,
wake_threshold: Union[float, Callable[[], float], None] = None,
on_score: Optional[Callable[[float, float], None]] = None,
):
"""Build whichever detector BARGE_IN_MODE asks for. An unrecognized mode
falls back to energy rather than raising — a typo in .env shouldn't stop
the pet from starting."""
mode = (config.BARGE_IN_MODE if mode is None else mode).strip().lower()
if mode in ("wake", "wakeword", "wake_word"):
return WakeWordBargeIn(stream, threshold=wake_threshold, on_score=on_score)
return BargeInDetector(stream)
bolt_pet/audio/mic.py
-161
View File
@@ -1,161 +0,0 @@
"""Mic capture + simple energy-based VAD utterance recording.
Ported from desk_client/bolt_desk.py's record_utterance() — same tuning
knobs, same behavior. Kept independent of any UI/threading model so it can
be unit tested by feeding it a fake "stream" object.
"""
from __future__ import annotations
import io
import wave
from typing import Optional, Protocol
import numpy as np
from .. import config
class AudioStream(Protocol):
"""Minimal shape of the object record_utterance() needs — matches
sounddevice.InputStream's .read(frames) -> (data, overflowed)."""
def read(self, frames: int): ...
def pcm_to_wav_bytes(pcm: np.ndarray, sample_rate: int = config.SAMPLE_RATE) -> bytes:
buf = io.BytesIO()
with wave.open(buf, "wb") as wf:
wf.setnchannels(1)
wf.setsampwidth(2)
wf.setframerate(sample_rate)
wf.writeframes(pcm.tobytes())
return buf.getvalue()
def rms(frame: np.ndarray) -> float:
return float(np.sqrt(np.mean(frame.astype(np.float64) ** 2)))
def _emit(on_frame, frame) -> None:
"""Hand a frame to a listener without letting it break the capture.
Streaming STT is an optimisation riding along with recording; if its
socket dies mid-utterance the recording must carry on untouched, because
the one-shot fallback is about to need the full buffer."""
if on_frame is None:
return
try:
on_frame(frame)
except Exception:
pass
def record_utterance(
stream: AudioStream,
should_continue=lambda: True,
rms_threshold: int = None,
silence_end_sec: float = None,
max_utterance_s: float = None,
min_utterance_s: float = None,
grace_s: float = None,
frame_len: int = config.FRAME_LEN,
sample_rate: int = config.SAMPLE_RATE,
on_frame=None,
) -> Optional[np.ndarray]:
"""Capture one utterance from *stream*: wait for speech to start, stop
after trailing silence. Returns None if nothing usable was heard.
*should_continue* is polled each frame so a caller can cancel recording
(e.g. the pet window was closed) without needing threading primitives
baked into this function.
*on_frame*, if given, is called with each captured frame while speech is
in progress — that is how streaming STT transcribes as you talk rather
than after (see audio/stt_stream.py). It is fire-and-forget: this function
still returns the full buffer, so a failed stream costs nothing.
*grace_s* is how long to wait for speech to *begin* before giving up.
The controller stretches it for follow-up questions, where you're being
asked something and need a moment to think rather than having just said
the wake word on purpose.
"""
rms_threshold = config.RMS_THRESHOLD if rms_threshold is None else rms_threshold
silence_end_sec = config.SILENCE_END_SEC if silence_end_sec is None else silence_end_sec
max_utterance_s = config.MAX_UTTERANCE_S if max_utterance_s is None else max_utterance_s
min_utterance_s = config.MIN_UTTERANCE_S if min_utterance_s is None else min_utterance_s
grace_s = config.GRACE_SECONDS if grace_s is None else grace_s
frames: list[np.ndarray] = []
started = False
silence_frames = 0
silence_limit = int(silence_end_sec * sample_rate / frame_len)
max_frames = int(max_utterance_s * sample_rate / frame_len)
grace_frames = int(grace_s * sample_rate / frame_len) # how long to wait for speech to begin
waited = 0
while should_continue():
chunk, _ = stream.read(frame_len)
frame = np.asarray(chunk)[:, 0].copy()
frame_rms = rms(frame)
if not started:
waited += 1
if frame_rms >= rms_threshold:
started = True
frames.append(frame)
_emit(on_frame, frame)
elif waited > grace_frames:
return None # woke it up but said nothing
continue
frames.append(frame)
_emit(on_frame, frame)
if frame_rms < rms_threshold:
silence_frames += 1
if silence_frames >= silence_limit:
break
else:
silence_frames = 0
if len(frames) >= max_frames:
break
if not frames:
return None
pcm = np.concatenate(frames)
if len(pcm) < min_utterance_s * sample_rate:
return None
return pcm
def list_input_devices() -> list[dict]:
"""Every input-capable device sounddevice can see, as
{"index", "name", "hostapi"}.
On Windows in particular, the *same* physical (or virtual) device is
normally listed once per host API — MME, DirectSound, WASAPI, WDM-KS —
all sharing (or near enough sharing) one name. That makes the name alone
an unreliable, often ambiguous way to pick a device (sounddevice matches
a string device as a substring and refuses to guess between ties), so the
tray's mic picker saves the numeric index instead, which this list exists
to let it show alongside a human-readable label."""
import sounddevice as sd
devices = []
for index, info in enumerate(sd.query_devices()):
if info.get("max_input_channels", 0) > 0:
hostapi = sd.query_hostapis(info["hostapi"])["name"]
devices.append({"index": index, "name": info["name"], "hostapi": hostapi})
return devices
def open_input_stream():
"""Real sounddevice input stream, imported lazily so pure-logic tests
(record_utterance with a fake stream) don't need PortAudio installed."""
import sounddevice as sd
return sd.InputStream(
samplerate=config.SAMPLE_RATE,
channels=1,
dtype="int16",
blocksize=config.FRAME_LEN,
device=config.MIC_DEVICE,
)
bolt_pet/audio/stt.py
-33
View File
@@ -1,33 +0,0 @@
"""Speech-to-text for the actual query, after the wake word fires.
The "guaranteed" fallback for when stt_stream's opportunistic live-feed
session didn't produce a transcript (streaming disabled, or the socket
never came up). There is no separate one-shot REST endpoint server-side any
more — /desk/stt is a websocket relay only — so this connects the exact
same way stt_stream.py does and just feeds the whole buffered utterance in
one go instead of frame-by-frame as it's captured. That connection is
unconditional (not gated by STT_STREAMING, which only controls the
opportunistic optimisation), since there is nothing left to fall back to
if it fails.
"""
from __future__ import annotations
from .. import config
from . import stt_stream
class SttError(Exception):
pass
def transcribe(pcm) -> str:
if not config.is_configured():
raise SttError("BOLT_SERVER_URL / DESK_API_KEY not set")
try:
socket = stt_stream.connect()
except Exception as exc:
raise SttError(f"couldn't reach the transcription server: {exc}") from exc
session = stt_stream.StreamingTranscriber(socket)
session.feed(pcm)
return session.finish()
bolt_pet/audio/stt_stream.py
-202
View File
@@ -1,202 +0,0 @@
"""Streaming speech-to-text — transcribing *while* you talk, not after.
The one-shot path (`stt.transcribe`) waits for the utterance to finish, then
sends the whole clip and waits again. That second wait is dead time between
you stopping and the pet reacting, and it grows with the length of what you
said — a thirty-second question costs noticeably more than a five-second one.
This connects to the Bolt server's own `/desk/stt` — the same websocket relay
the Android app uses — which forwards audio to Deepgram and Deepgram's JSON
messages back untouched. Frames go up as they are captured, so by the time the
VAD decides you have stopped, the transcript is essentially already there.
There is no local Deepgram account or API key any more; auth is this pet's own
`DESK_API_KEY`, same as every other call to the server.
Design constraints that shaped this:
- **Failure must be invisible.** No websocket, no network, a mid-utterance
disconnect — all of it falls back to the one-shot path, which still has the
full audio buffered. Streaming is an optimisation here, never a dependency —
`available()`/`open()` returning None/False is an ordinary outcome, not an
error. `stt.transcribe()` (the *guaranteed* fallback) talks to the exact same
server relay via `connect()` directly, bypassing that opportunistic gate,
since there is no second, different backend left to fall back to.
- **The VAD still decides when you stopped.** Deepgram has its own endpointing
and using it would save more, but it would also move a decision the rest of
the pipeline is built around (barge-in, follow-up listening, the grace
period) into a remote service. Not worth coupling those to the network on
the first pass.
- **The socket is per-utterance.** Holding one open across an idle pet would
bill for silence and drop on the first network blip; opening one takes
~100ms, which is already inside the time it takes a person to start talking.
The websocket client is injectable, so the whole protocol — send frames, read
`is_final` transcripts, close, take the result — is tested without a network.
"""
from __future__ import annotations
import json
import logging
import threading
from typing import Callable, Optional
import numpy as np
from .. import config
logger = logging.getLogger("bolt_pet.stt_stream")
def available() -> bool:
"""Whether the streaming (transcribe-while-talking) optimization should
be attempted opportunistically. Not a gate on transcription itself — the
server relay is the only way to transcribe at all now, so
stt.transcribe() connects via connect() directly rather than through
this, and isn't affected by STT_STREAMING being off."""
if not config.STT_STREAMING or not config.is_configured():
return False
try:
import websocket # noqa: F401 (websocket-client)
return True
except Exception:
return False
def _connect(sample_rate: int = None):
"""Open a websocket to the server's `/desk/stt` relay. Raises on any
failure — this is the "no fallback left" connector `stt.transcribe()`
uses directly, as well as the default for `StreamingTranscriber.open()`.
The server chooses the STT model and the endpointing behaviour; this
only states the audio format about to be sent, which is fixed by the
wake model upstream of it."""
import websocket
rate = sample_rate or config.SAMPLE_RATE
url = (
config.SERVER_URL.replace("http", "ws", 1)
+ "/desk/stt"
+ f"?session_id={config.SESSION_ID}&encoding=linear16&sample_rate={rate}"
)
return websocket.create_connection(
url, header=[f"X-Desk-Api-Key: {config.API_KEY}"], timeout=10,
)
# Public name for external callers (stt.py, tests) — named separately from
# the module-private def so StreamingTranscriber.open()'s `connect` parameter
# can shadow the bare name locally without losing access to this.
connect = _connect
class StreamingTranscriber:
"""One utterance's worth of live transcription.
Usage mirrors how the capture loop already works — feed frames as they
arrive, then ask what was said:
session = StreamingTranscriber.open()
...
session.feed(frame) # per mic frame, non-blocking
text = session.finish() # after the VAD says you stopped
"""
def __init__(self, socket, *, sample_rate: int = None):
self._socket = socket
self._sample_rate = sample_rate or config.SAMPLE_RATE
self._transcript: list[str] = []
self._lock = threading.Lock()
self._closed = False
self._reader = threading.Thread(
target=self._read_loop, name="stt-stream-reader", daemon=True)
self._reader.start()
# -- lifecycle ----------------------------------------------------------
@classmethod
def open(cls, *, connect: Optional[Callable] = None,
sample_rate: int = None) -> Optional["StreamingTranscriber"]:
"""Connect, or return None if streaming isn't possible right now.
None is a normal outcome, not a failure: the caller keeps the audio and
falls back to the one-shot upload."""
if connect is None and not available():
return None
rate = sample_rate or config.SAMPLE_RATE
try:
socket = connect() if connect is not None else _connect(rate)
return cls(socket, sample_rate=rate)
except Exception as exc:
logger.info("Streaming STT unavailable (%s) — using the one-shot path.", exc)
return None
def feed(self, frame: np.ndarray) -> None:
"""Send one captured frame. Never raises — a dead socket just means the
fallback will do the work."""
if self._closed:
return
try:
self._socket.send_binary(np.asarray(frame, dtype=np.int16).tobytes())
except Exception:
logger.debug("Streaming STT send failed; abandoning the stream", exc_info=True)
self._closed = True
def finish(self, timeout: float = 3.0) -> str:
"""Close the stream and return whatever was transcribed.
Deepgram flushes its final results after the close frame, so this waits
briefly for the reader — bounded, because a hung socket must not hold
up the reply."""
if not self._closed:
try:
self._socket.send(json.dumps({"type": "CloseStream"}))
except Exception:
pass
self._closed = True
self._reader.join(timeout=timeout)
try:
self._socket.close()
except Exception:
pass
with self._lock:
return " ".join(part for part in self._transcript if part).strip()
# -- the reader ---------------------------------------------------------
def _read_loop(self) -> None:
while not self._closed:
try:
message = self._socket.recv()
except Exception:
break
if not message:
break
text, is_final = self._parse(message)
if text and is_final:
with self._lock:
self._transcript.append(text)
@staticmethod
def _parse(message) -> tuple[str, bool]:
"""Pull (text, is_final) out of a Deepgram results frame.
Tolerant on purpose: anything unrecognised is ignored rather than
raising on the reader thread, where an exception would silently kill
transcription for the rest of the utterance."""
try:
if isinstance(message, bytes):
message = message.decode("utf-8", "ignore")
data = json.loads(message)
except (TypeError, ValueError):
return "", False
if not isinstance(data, dict):
return "", False
alternatives = (
((data.get("channel") or {}).get("alternatives") or [])
if data.get("type") in (None, "Results") else []
)
if not alternatives:
return "", False
text = str((alternatives[0] or {}).get("transcript") or "").strip()
return text, bool(data.get("is_final") or data.get("speech_final"))
bolt_pet/audio/tts.py
-300
View File
@@ -1,300 +0,0 @@
"""Text-to-speech: the Bolt server's own `/desk/tts` — the same endpoint the
Android app streams from — requested as raw PCM so playback is just
sounddevice — no external player binary (mpv/ffplay), unlike
desk_client/bolt_desk.py which shells out because it only targets Linux.
No local ElevenLabs account needed for this: the server picks the voice
(ELEVENLABS_VOICE_ID, or a `speak_as` override) and the synthesis model
itself, authenticated with this pet's own DESK_API_KEY. Falls back to
pyttsx3 (offline, cross-platform: SAPI5 on Windows, NSSpeech on macOS,
espeak on Linux) if the server call fails, so the pet can still talk even
with the server unreachable.
Every entry point takes an optional *voice_id* that overrides
`ELEVENLABS_VOICE_ID` for that call — that's how the server's `speak_as`
reply marker reaches the speakers (see controller._apply_voice). The offline
fallback has no such concept and always sounds like itself.
`synthesize_dialogue()` below is the one exception: multi-voice
`dialoguectl` scenes have no server endpoint, so that one call still goes
to ElevenLabs' Text to Dialogue API directly and still needs
ELEVENLABS_API_KEY — see dialogue.py.
"""
from __future__ import annotations
import time
from typing import Iterable, Iterator, Optional
import numpy as np
import requests
from .. import config, speech_text
class TtsError(Exception):
pass
def voice_for(voice_id: Optional[str] = None) -> str:
"""The voice this call should use: an override (server `speak_as`) if
given, else the configured default."""
return (voice_id or "").strip() or config.ELEVENLABS_VOICE_ID
def _headers() -> dict:
return {"X-Desk-Api-Key": config.API_KEY}
def synthesize_pcm(text: str, voice_id: Optional[str] = None) -> tuple[np.ndarray, int]:
"""Returns (pcm_int16_mono, sample_rate). Raises TtsError on failure —
callers should fall back to speak_offline() rather than treating this
as fatal."""
voice = voice_for(voice_id)
if not (config.is_configured() and voice):
raise TtsError("BOLT_SERVER_URL / DESK_API_KEY / ELEVENLABS_VOICE_ID not set")
try:
response = requests.post(
f"{config.SERVER_URL}/desk/tts",
headers=_headers(),
json={"session_id": config.SESSION_ID, "text": text, "voice_id": voice},
timeout=60,
)
response.raise_for_status()
except Exception as exc:
raise TtsError(f"server tts request failed: {exc}") from exc
pcm = np.frombuffer(response.content, dtype=np.int16)
if pcm.size == 0:
raise TtsError("server returned no audio")
return pcm, config.TTS_SAMPLE_RATE
def stream_pcm(
text: str, chunk_bytes: int = 4096, voice_id: Optional[str] = None
) -> Iterator[np.ndarray]:
"""Same audio as synthesize_pcm(), but yielded as it arrives from the
server so playback can start on the first chunk instead of after the
whole clip is synthesized. Raises TtsError before yielding anything if
the request itself fails, so callers can fall back cleanly; a mid-stream
failure just ends the generator."""
voice = voice_for(voice_id)
if not (config.is_configured() and voice):
raise TtsError("BOLT_SERVER_URL / DESK_API_KEY / ELEVENLABS_VOICE_ID not set")
try:
response = requests.post(
f"{config.SERVER_URL}/desk/tts",
headers=_headers(),
json={"session_id": config.SESSION_ID, "text": text, "voice_id": voice},
timeout=60,
stream=True,
)
response.raise_for_status()
except Exception as exc:
raise TtsError(f"server tts stream request failed: {exc}") from exc
return chunks_to_int16(response.iter_content(chunk_size=chunk_bytes))
def chunks_to_int16(byte_chunks: Iterable[bytes]) -> Iterator[np.ndarray]:
"""Reassemble a byte stream into int16 frames. HTTP chunk boundaries fall
wherever they like, including *inside* a 16-bit sample, so a trailing odd
byte has to be carried into the next chunk — otherwise every chunk after
the first is shifted by one byte and plays as static."""
carry = b""
for chunk in byte_chunks:
if not chunk:
continue
data = carry + chunk
usable = len(data) - (len(data) % 2)
carry = data[usable:]
if usable:
yield np.frombuffer(data[:usable], dtype=np.int16)
def synthesize_dialogue(
inputs: list, model_id: Optional[str] = None, stability: Optional[float] = None
) -> tuple[np.ndarray, int]:
"""Multi-voice scene via ElevenLabs Text to Dialogue.
One request, one take: the whole exchange is synthesized together, which
is the point — the model hears the previous line, so reactions and timing
land instead of sounding like separately-rendered clips.
Same PCM-over-`requests` posture as the rest of this module (no SDK, no
`play()` shelling out to ffplay), so playback is the same sounddevice path
everything else uses and barge-in works on it unchanged. There is no
documented streaming variant, and a scene is a short set piece anyway, so
this is whole-clip only.
"""
if not (config.ELEVENLABS_API_KEY and inputs):
raise TtsError("ELEVENLABS_API_KEY not set (or no dialogue lines)")
body: dict = {
"inputs": [
{"text": str(entry.get("text") or ""), "voice_id": str(entry.get("voice_id") or "")}
for entry in inputs
],
"model_id": model_id or config.DIALOGUE_MODEL_ID,
}
if stability is not None:
body["settings"] = {"stability": float(stability)}
try:
response = requests.post(
"https://api.elevenlabs.io/v1/text-to-dialogue",
headers={"xi-api-key": config.ELEVENLABS_API_KEY},
params={"output_format": f"pcm_{config.TTS_SAMPLE_RATE}"},
json=body,
timeout=120, # a multi-voice take is slower to render than one line
)
response.raise_for_status()
except Exception as exc:
detail = ""
# The API explains refusals (character limit, unknown voice) in the
# body; surfacing it is what lets Bolt fix the call and retry.
body_text = getattr(getattr(exc, "response", None), "text", "")
if body_text:
detail = f" — {body_text[:300]}"
raise TtsError(f"ElevenLabs dialogue request failed: {exc}{detail}") from exc
pcm = np.frombuffer(response.content, dtype=np.int16)
if pcm.size == 0:
raise TtsError("ElevenLabs returned no dialogue audio")
return pcm, config.TTS_SAMPLE_RATE
# ── how loud is it right now ────────────────────────────────────────────────
# The PCM is already decoded here on its way to the speakers, so the amplitude
# envelope is free — and it is exactly what a mouth needs to move in time with
# speech. Throwing it away and animating the mouth on a timer instead is why
# most talking sprites look dubbed.
# int16 RMS that counts as "mouth fully open". Speech peaks around 8-12k;
# 6000 keeps normal talking in the upper half of the range without clipping
# every syllable to wide-open.
_LOUD_RMS = 6000.0
def level_of(frame: np.ndarray) -> float:
"""0..1 loudness for one chunk of PCM.
Square-rooted because perceived loudness is not linear in amplitude — a
linear mapping leaves the mouth barely moving through ordinary speech."""
if frame is None or len(frame) == 0:
return 0.0
rms = float(np.sqrt(np.mean(np.square(frame.astype(np.float32)))))
return float(min(1.0, (rms / _LOUD_RMS) ** 0.5))
def envelope(pcm: np.ndarray, sample_rate: int, fps: int = 30) -> list:
"""Per-frame loudness for a whole clip, for playback that isn't streamed."""
if pcm is None or len(pcm) == 0:
return []
window = max(1, int(sample_rate / max(1, fps)))
return [level_of(pcm[start:start + window]) for start in range(0, len(pcm), window)]
def play_pcm(pcm: np.ndarray, sample_rate: int, blocking: bool = True, should_stop=None,
on_level=None) -> bool:
"""Play a whole clip. Returns True if it finished, False if *should_stop*
(barge-in) cut it short. *should_stop* is polled while audio plays — each
poll consumes one mic frame, which is what paces this loop."""
import sounddevice as sd
levels = envelope(pcm, sample_rate) if on_level is not None else []
started = time.monotonic()
sd.play(pcm, samplerate=sample_rate, device=config.SPEAKER_DEVICE)
if not blocking:
return True
if should_stop is None and on_level is None:
sd.wait()
return True
while True:
if levels:
# Indexed by elapsed time rather than by chunk, because this path
# hands the whole clip to the device at once and never sees it
# again — wall clock is the only position we have.
index = int((time.monotonic() - started) * 30)
if index < len(levels):
try:
on_level(levels[index])
except Exception:
levels = []
try:
if not sd.get_stream().active:
break
except Exception:
break # stream already torn down — playback is over
if should_stop is not None and should_stop():
sd.stop()
return False
return True
def play_stream(chunks: Iterable[np.ndarray], sample_rate: int, should_stop=None,
on_level=None) -> bool:
"""Play int16 chunks as they arrive. Returns False if interrupted.
*on_level* receives each chunk's loudness (0..1) just before it is written,
which is what drives the mouth: the sprite is animated by the same audio
the speakers are getting, not by a guess about how long a word takes."""
import sounddevice as sd
with sd.OutputStream(
samplerate=sample_rate, channels=1, dtype="int16", device=config.SPEAKER_DEVICE
) as out:
for chunk in chunks:
if should_stop is not None and should_stop():
# abort() rather than draining: barge-in should stop the voice
# now, not at the end of the buffered chunk.
out.abort()
return False
if on_level is not None:
try:
on_level(level_of(chunk))
except Exception:
on_level = None # a broken listener must not stop playback
out.write(chunk)
return True
def speak_offline(text: str) -> None:
try:
import pyttsx3
except ImportError:
return # no TTS available at all — caller already logs the text
engine = pyttsx3.init()
engine.say(text)
engine.runAndWait()
def speak(text: str, on_error=None, should_stop=None, voice_id: Optional[str] = None,
on_level=None) -> bool:
"""Speak *text*, preferring streaming ElevenLabs, then whole-clip
ElevenLabs, then offline TTS. *on_error*, if given, is called with the
exception when ElevenLabs fails (useful for logging) — a fallback still
runs either way. Returns False if barge-in interrupted playback.
*voice_id* overrides the configured voice for this line only.
The text is sanitized first (speech_text.for_speech): server replies are
written for a chat window, and a voice reads markdown/emoji literally
("asterisk asterisk"). Sanitizing here rather than at the call sites means
every path to the speakers — reply, heartbeat announcement — is covered."""
text = speech_text.for_speech(text)
if not text:
return True
if config.TTS_STREAMING:
try:
return play_stream(
stream_pcm(text, voice_id=voice_id),
config.TTS_SAMPLE_RATE,
should_stop=should_stop,
on_level=on_level,
)
except TtsError as exc:
if on_error is not None:
on_error(exc)
try:
pcm, sample_rate = synthesize_pcm(text, voice_id=voice_id)
return play_pcm(pcm, sample_rate, should_stop=should_stop, on_level=on_level)
except TtsError as exc:
if on_error is not None:
on_error(exc)
speak_offline(text)
return True
bolt_pet/audio/wake_word.py
-259
View File
@@ -1,259 +0,0 @@
"""Wake-word detection via a custom-trained openWakeWord model.
Uses `thunderbolt.onnx` — trained specifically for "thunderbolt", the same
way the main repo's `desk_client/bolt_desk.py` uses `bolt.onnx` for "hey
bolt". Same runtime (openWakeWord, ONNX inference), same per-frame
predict()/reset() pattern; the only difference is the model file
(WAKE_MODEL_FILE) and threshold (WAKE_WORD_THRESHOLD), both configurable via
.env if a differently-trained model is swapped in later.
"""
from __future__ import annotations
from collections import deque
from typing import Callable, Optional, Protocol, Union
import numpy as np
from .. import config
class WakeModel(Protocol):
def predict(self, frame: np.ndarray) -> dict: ...
def reset(self) -> None: ...
class NearMissLog:
"""Rolling record of frames that *almost* fired the wake word.
WAKE_WORD_THRESHOLD is otherwise tuned by guessing at a number in .env
and seeing whether the pet ignores you. Keeping the near misses (scores
within WAKE_NEAR_MISS_MARGIN below the threshold) turns that into
evidence: the tray's wake-word tuner shows what your actual "thunderbolt"
scores, so you can set the threshold just under it.
Pure bookkeeping — the caller supplies timestamps, so it's testable.
"""
def __init__(self, limit: int = None, margin: float = None):
self._entries: deque[tuple[float, float, float]] = deque( # (timestamp, score, threshold)
maxlen=max(1, config.WAKE_NEAR_MISS_LIMIT if limit is None else limit)
)
self._margin = config.WAKE_NEAR_MISS_MARGIN if margin is None else margin
self._peak = 0.0
@property
def peak(self) -> float:
"""Highest score seen since the last reset — the "how close did I
get?" readout while you test the wake phrase."""
return self._peak
def observe(self, score: float, threshold: float, timestamp: float) -> bool:
"""Record *score*; returns True if it counted as a near miss."""
self._peak = max(self._peak, score)
if score >= threshold or score < threshold - self._margin:
return False
self._entries.append((timestamp, score, threshold))
return True
def entries(self) -> list[tuple[float, float, float]]:
return list(self._entries)
def clear(self) -> None:
self._entries.clear()
self._peak = 0.0
# Where the cached blank state is stashed — on the preprocessor itself
# rather than in a dict keyed by id(), which CPython reuses after garbage
# collection and would hand one model another's buffers.
_BLANK_STATE_ATTR = "_bolt_blank_state"
def _blank_state(preprocessor) -> Optional[tuple]:
"""(feature_buffer, melspectrogram_buffer) as they are on a freshly
constructed model — i.e. "having heard nothing". Computing it costs an
ONNX pass over 10s of silence, so it's cached on the preprocessor and
copied from thereafter. Returns None if openwakeword's internals don't
look the way we expect, in which case callers leave the state alone
rather than corrupting it."""
cached = getattr(preprocessor, _BLANK_STATE_ATTR, None)
if cached is not None:
return cached
try:
# Same call the AudioFeatures constructor uses to prime the buffer.
state = (preprocessor._get_embeddings(np.zeros(160000).astype(np.int16)), np.ones((76, 32)))
setattr(preprocessor, _BLANK_STATE_ATTR, state)
except Exception:
return None
return state
def hard_reset(model) -> None:
"""Make the model forget the audio it has already heard — not just its
predictions.
openwakeword's ``Model.reset()`` clears the *prediction* buffer only.
The rolling audio window the classifier actually scores lives in
``model.preprocessor`` (raw_data_buffer / melspectrogram_buffer /
feature_buffer, ~10s of history) and has no reset method of its own. So
after a detection the wake word is still sitting in that window, and the
next frame fed to the model re-fires on it — which is exactly what made
the pet interrupt itself a word into every reply: the "thunderbolt" that
started the turn was still in the buffer when barge-in resumed feeding it.
Never raises. A model whose internals don't match (a fake in tests, a
future openwakeword release) just gets the plain reset()."""
try:
model.reset()
except Exception:
pass
preprocessor = getattr(model, "preprocessor", None)
if preprocessor is None:
return
blank = _blank_state(preprocessor)
try:
if blank is not None:
features, melspectrogram = blank
preprocessor.feature_buffer = features.copy()
preprocessor.melspectrogram_buffer = melspectrogram.copy()
preprocessor.raw_data_buffer.clear()
preprocessor.accumulated_samples = 0
except Exception:
pass
def _construct_model(model_cls, model_path: str):
"""openwakeword's Model() constructor keyword has drifted across
releases (wakeword_models -> wakeword_model_paths) and some builds
reject inference_framework entirely — the main repo's ai/wake_word.py
hit the same drift and works around it the same way: try each known
calling convention in turn."""
attempts = [
lambda: model_cls(wakeword_model_paths=[model_path], inference_framework="onnx"),
lambda: model_cls(wakeword_model_paths=[model_path]),
lambda: model_cls(wakeword_models=[model_path], inference_framework="onnx"),
lambda: model_cls(wakeword_models=[model_path]),
lambda: model_cls([model_path]),
]
last_exc: Optional[TypeError] = None
for attempt in attempts:
try:
return attempt()
except TypeError as exc:
last_exc = exc
raise RuntimeError(
f"Could not construct openwakeword.Model with any known calling convention "
f"(last error: {last_exc})"
)
class _OpenWakeWordModel:
"""Lazily loads the ONNX model on first use so importing this module
(and unit-testing listen_for_wake_word with a fake model) never requires
onnxruntime/openwakeword or the model file to be present."""
def __init__(self):
self._model = None
def _ensure_model(self):
if self._model is None:
from openwakeword.model import Model
# from openwakeword.utils import download_models
# # The pip package doesn't bundle its melspectrogram/embedding
# # feature-extraction sub-models — fetch them once on first use
# # (no-op if already cached in openwakeword's own resources dir).
# # A non-empty, non-matching model_names list keeps this from
# # also pulling every official pretrained wakeword model.
# download_models(model_names=["thunderbolt"])
self._model = _construct_model(Model, config.WAKE_MODEL_PATH)
return self._model
@property
def preprocessor(self):
"""Proxy the wrapped model's audio-feature buffers.
Without this, hard_reset() sees a wrapper with no `preprocessor` and
silently degrades to openwakeword's shallow reset() — which leaves the
previous detection sitting in the audio window, i.e. exactly the bug
hard_reset exists to fix. Returns None before the model is loaded, so
a reset that happens first is a no-op rather than a load."""
return getattr(self._model, "preprocessor", None)
def predict(self, frame: np.ndarray) -> dict:
return self._ensure_model().predict(frame)
def reset(self) -> None:
if self._model is not None:
self._model.reset()
_default_model = _OpenWakeWordModel()
def listen_for_wake_word(
stream,
should_continue=lambda: True,
model: Optional[WakeModel] = None,
threshold: Union[float, Callable[[], float], None] = None,
on_tick=None,
on_score: Optional[Callable[[float, float], None]] = None,
) -> bool:
"""Block until the wake word fires (returns True) or *should_continue*
goes false (returns False).
Feeds every frame to *model* (the thunderbolt openWakeWord model by
default) and treats any class score >= *threshold* as a detection,
clearing the model's internal state (audio window included, see
hard_reset) afterward so the next call starts clean — same pattern as
desk_client/bolt_desk.py's main loop.
*threshold* may be a number or a zero-argument callable. The callable
form exists because this function blocks for minutes at a time: the
tray's wake-word tuner slider has to be able to change sensitivity
*during* a listen, not only at the start of the next one.
*on_tick*, if given, is called once per ``WAKE_CHECK_INTERVAL_SECONDS``
(not every frame — prediction is cheap enough to run on every frame, but
this is the only point control returns to the caller while otherwise
blocked here for a possibly long time, so it's how a caller drives
periodic work, e.g. the heartbeat/announcement poll in controller.py,
during quiet stretches with no wake word).
*on_score*, if given, gets ``(best_score, threshold)`` every frame — used
to log near misses for threshold tuning.
"""
model = model or _default_model
if threshold is None:
threshold = config.WAKE_WORD_THRESHOLD
resolve_threshold = threshold if callable(threshold) else (lambda: threshold)
frame_len = config.FRAME_LEN
check_every_frames = max(1, int(config.WAKE_CHECK_INTERVAL_SECONDS * config.SAMPLE_RATE / frame_len))
frames_since_tick = 0
while should_continue():
chunk, _ = stream.read(frame_len)
frame = np.asarray(chunk)[:, 0]
scores = model.predict(frame)
current_threshold = resolve_threshold()
best = max(scores.values()) if scores else 0.0
frames_since_tick += 1
if frames_since_tick >= check_every_frames:
frames_since_tick = 0
if on_tick is not None:
on_tick()
if on_score is not None:
on_score(best, current_threshold)
if scores and best >= current_threshold:
# hard_reset, not reset: the phrase has to leave the model's audio
# window too, or the very next frame we feed it re-fires on the
# same "thunderbolt" (see hard_reset's docstring).
hard_reset(model)
return True
return False
bolt_pet/config.py
-386
View File
@@ -1,386 +0,0 @@
"""Environment configuration for the Bolt desktop pet.
Same lightweight ".env next to the script" pattern as desk_client/bolt_desk.py
in the main tmn-api repo, so this project can be copied anywhere (it does not
import anything from that repo) and configured the same way.
"""
from __future__ import annotations
import os
import platform
import re
from pathlib import Path
HERE = Path(__file__).resolve().parent.parent # project root (one above bolt_pet/)
def _load_env() -> None:
env_path = HERE / ".env"
if not env_path.exists():
return
for line in env_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
os.environ.setdefault(key.strip(), value.strip())
_load_env()
def set_env_value(key: str, value: str) -> None:
"""Write *key=value* into .env, in place — replacing (or uncommenting)
an existing line for it, appending a new one otherwise. Used by the tray's
microphone picker so a device pick survives a restart without hand-editing
the file. Settings read once at import time (like MIC_DEVICE) still need
that restart to take effect; this only updates the file and os.environ."""
env_path = HERE / ".env"
lines = env_path.read_text(encoding="utf-8").splitlines() if env_path.exists() else []
pattern = re.compile(rf"^\s*#?\s*{re.escape(key)}\s*=")
new_line = f"{key}={value}"
for i, line in enumerate(lines):
if pattern.match(line):
lines[i] = new_line
break
else:
lines.append(new_line)
env_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
os.environ[key] = value
def _node_name() -> str:
try:
return platform.node() or "desktop"
except Exception:
return "desktop"
# ── server / identity ───────────────────────────────────────────────────────
SERVER_URL = os.environ.get("BOLT_SERVER_URL", "").rstrip("/")
API_KEY = os.environ.get("DESK_API_KEY", "")
SESSION_ID = os.environ.get("DESK_SESSION_ID", "pet-" + _node_name())
# ── wake word ────────────────────────────────────────────────────────────────
# openWakeWord model trained specifically for "thunderbolt" — same pattern as
# desk_client/bolt_desk.py's WAKE_MODEL_FILE (bolt.onnx / "hey bolt") in the
# main repo, resolved relative to the project root so it sits next to .env.
WAKE_MODEL_PATH = str(HERE / os.environ.get("WAKE_MODEL_FILE", "thunderbolt.onnx"))
WAKE_WORD_THRESHOLD = float(os.environ.get("WAKE_WORD_THRESHOLD", "0.5"))
# How often (independent of prediction, which runs every frame) the wake
# listener yields control back to its caller via on_tick — e.g. the
# heartbeat poll in controller.py during quiet stretches with no wake word.
WAKE_CHECK_INTERVAL_SECONDS = float(os.environ.get("WAKE_CHECK_INTERVAL_SECONDS", "1.2"))
# ── STT (server-hosted, same /desk/stt relay the Android app uses) ─────────
# No local Deepgram account needed any more: audio/stt_stream.py opens a
# websocket to this pet's own BOLT_SERVER_URL/DESK_API_KEY, which the server
# relays to Deepgram and meters it against the same credit ledger as a chat turn.
# There is no separate one-shot REST path server-side, so audio/stt.py's
# "guaranteed" fallback uses this exact same connection too — just fed the
# whole utterance at once instead of frame-by-frame.
# Transcribe *while* you talk instead of waiting for the utterance to end:
# frames go up as they are captured, so the transcript is ready the moment the
# VAD says you stopped. Needs `websocket-client` (in requirements.txt); when
# it can't connect the one-shot path (audio/stt.py) still tries the same
# server relay itself, so turning this off only costs latency, not the
# ability to transcribe at all.
STT_STREAMING = os.environ.get("STT_STREAMING", "true").lower() in ("1", "true", "yes", "on")
# ── TTS (server-hosted, same /desk/tts endpoint the Android app uses) ──────
# No local ElevenLabs account needed for the normal reply voice any more —
# audio/tts.py posts to this pet's own BOLT_SERVER_URL/DESK_API_KEY and gets
# back raw 16 kHz mono PCM16, same as the phone. Falls back to offline
# pyttsx3 if the server call fails.
#
# ELEVENLABS_API_KEY is still read directly by this client for exactly one
# feature the server has no endpoint for: multi-voice `dialoguectl` scenes
# (audio/tts.synthesize_dialogue, see dialogue.py) — leave it blank and
# everything except that one feature works with zero local API keys.
ELEVENLABS_API_KEY = os.environ.get("ELEVENLABS_API_KEY", "")
# Which voice to ask the server for — an ElevenLabs voice id (or a "vb:"-
# prefixed cloned voice, if this desk key owns one). The server picks the
# synthesis model itself now; there is nothing left for this client to choose.
ELEVENLABS_VOICE_ID = os.environ.get("ELEVENLABS_VOICE_ID", "")
# Fixed by the server (ai/desk_media.py's PCM_SAMPLE_RATE) — not a free
# tunable any more, but still an env override in case that ever changes.
TTS_SAMPLE_RATE = int(os.environ.get("TTS_SAMPLE_RATE", "16000"))
# Does a voice the server picks (its speak_as marker — "talk like a pirate",
# "say that in Japanese") stay on for later replies, or last one reply only?
# Sticky by default: the server tags a single reply and does *not* keep the
# voice id in its history, so a one-reply-only voice can't be re-used when
# you say "keep talking like that" — it would have to search for a voice
# again. Reset it from the tray ("Use default voice") or by restarting.
VOICE_STICKY = os.environ.get("VOICE_STICKY", "true").lower() in ("1", "true", "yes", "on")
# ── multi-voice dialogue (ElevenLabs Text to Dialogue) ──────────────────────
# Lets Bolt play a short scene in several voices with delivery tags the v3
# model acts on ("[cheerfully] Hello"), instead of one voice reading a line.
# Driven by the server through the `dialoguectl` relayed command — see
# dialogue.py. Costs a separate (slower, whole-clip) request per scene, so
# it's a set piece, not the normal reply path.
#
# DIALOGUE_VOICES names the cast: "narrator:9BWtsMINqrJLrRacOk9x,villain:IKne3meq5aSn9XLyUdCD".
# The name "self" always resolves to the voice the pet is currently using,
# including one the server picked with speak_as.
DIALOGUE = os.environ.get("DIALOGUE", "true").lower() in ("1", "true", "yes", "on")
DIALOGUE_MODEL_ID = os.environ.get("DIALOGUE_MODEL_ID", "eleven_v3")
DIALOGUE_VOICES = os.environ.get("DIALOGUE_VOICES", "")
# ── mic / VAD (same tuning knobs as bolt_desk.py) ───────────────────────────
def _mic_device(raw: str):
"""sounddevice treats an int index and a string name very differently: a
string is matched as a (fragile, often ambiguous) substring against every
device name, while an int is used as-is. .env only ever gives us a
string, so a numeric one — what the tray's mic picker saves, since the
same physical device usually shows up once per host API (MME,
DirectSound, WASAPI, ...) under the same name — has to be cast back to
int here, or it goes through the substring path and matches nothing (or
everything) instead."""
raw = raw.strip()
if not raw:
return None
return int(raw) if raw.lstrip("-").isdigit() else raw
MIC_DEVICE = _mic_device(os.environ.get("MIC_DEVICE", ""))
SPEAKER_DEVICE = os.environ.get("SPEAKER_DEVICE", "") or None
RMS_THRESHOLD = int(os.environ.get("VAD_RMS_THRESHOLD", "300"))
SILENCE_END_SEC = float(os.environ.get("VAD_SILENCE_END_SEC", "1.2"))
MAX_UTTERANCE_S = float(os.environ.get("VAD_MAX_UTTERANCE_SECONDS", "15"))
MIN_UTTERANCE_S = float(os.environ.get("VAD_MIN_UTTERANCE_SECONDS", "0.4"))
# How long to wait for you to *start* talking before giving up on a turn.
GRACE_SECONDS = float(os.environ.get("VAD_GRACE_SECONDS", "4"))
# ── follow-up listening ─────────────────────────────────────────────────────
# When a reply ends on a question, the pet keeps listening for the answer
# instead of dropping back to idle and making you say the wake word again.
# The grace period is longer than a normal turn's because you were asked
# something and may need a beat to think. FOLLOW_UP_MAX_TURNS caps how many
# question-and-answer rounds can chain without you re-triggering it — a stop
# on runaway loops if the server ends every reply with a question and the mic
# keeps feeding it noise. 0 means no cap.
FOLLOW_UP_LISTEN = os.environ.get("FOLLOW_UP_LISTEN", "true").lower() in ("1", "true", "yes", "on")
FOLLOW_UP_MAX_TURNS = int(os.environ.get("FOLLOW_UP_MAX_TURNS", "10"))
FOLLOW_UP_GRACE_SECONDS = float(os.environ.get("FOLLOW_UP_GRACE_SECONDS", "7"))
COMMAND_TIMEOUT_SECONDS = int(os.environ.get("COMMAND_TIMEOUT_SECONDS", "30"))
# ── sudo password prompts ───────────────────────────────────────────────────
# The pet has no terminal, so a relayed `sudo` would block on a tty nobody is
# watching. With this on, bare `sudo` is rewritten to `sudo -A` and the
# password is collected in a desktop dialog (a real askpass binary if one is
# installed, otherwise a generated zenity/kdialog wrapper). Turn it off and
# sudo commands simply fail, which is the safer default if you'd rather Bolt
# never be able to ask for root at all.
SUDO_ASKPASS_PROMPT = os.environ.get("SUDO_ASKPASS_PROMPT", "true").lower() in ("1", "true", "yes", "on")
SUDO_ASKPASS_HELPER = os.environ.get("SUDO_ASKPASS_HELPER", "") # blank = auto-detect
# Longer than COMMAND_TIMEOUT_SECONDS because a person has to notice the
# dialog, read it, and type — 30s is nowhere near enough for that.
SUDO_COMMAND_TIMEOUT_SECONDS = int(os.environ.get("SUDO_COMMAND_TIMEOUT_SECONDS", "180"))
HEARTBEAT_INTERVAL_SECONDS = float(os.environ.get("HEARTBEAT_INTERVAL_SECONDS", "60"))
# ── self-restart ────────────────────────────────────────────────────────────
# `petctl self_restart` lets Bolt restart the pet after editing its code, so
# he can see his own change running instead of waiting for someone to restart
# it by hand. The code is import-checked in a subprocess first, and the reason
# is carried across the restart so the new process can report back — see
# self_restart.py. SELF_RESTART_MAX/_WINDOW_SECONDS bound the crash-loop case.
SELF_RESTART = os.environ.get("SELF_RESTART", "true").lower() in ("1", "true", "yes", "on")
# ── barge-in (interrupt playback while the pet is talking) ──────────────────
# The mic stays live while the pet talks. BARGE_IN_MODE decides what counts
# as an interruption:
# wake — only the wake word cuts playback (default). Immune to coughs,
# doors, and the TV, at the cost of ~a word of extra latency.
# energy — any sustained noise above BARGE_IN_RMS_THRESHOLD does. Faster,
# but interrupts on background noise. That threshold is well above
# VAD_RMS_THRESHOLD because the mic also hears the pet's own voice
# through the speakers — raise it further if playback keeps
# interrupting itself.
# Set BARGE_IN=false to make playback uninterruptible either way.
BARGE_IN = os.environ.get("BARGE_IN", "true").lower() in ("1", "true", "yes", "on")
BARGE_IN_MODE = os.environ.get("BARGE_IN_MODE", "wake")
BARGE_IN_RMS_THRESHOLD = int(os.environ.get("BARGE_IN_RMS_THRESHOLD", str(RMS_THRESHOLD * 4)))
BARGE_IN_FRAMES = int(os.environ.get("BARGE_IN_FRAMES", "4")) # consecutive loud frames (80ms each)
# Wake-mode sensitivity. Blank means "track the live WAKE_WORD_THRESHOLD from
# the tray tuner"; set a number to make interrupting deliberately harder than
# waking the pet from idle (useful if Bolt's own voice trips the model).
BARGE_IN_WAKE_THRESHOLD = float(os.environ.get("BARGE_IN_WAKE_THRESHOLD") or 0) or None
# ── streaming the reply ─────────────────────────────────────────────────────
# Speak each sentence as the server produces it, instead of waiting out the
# whole model call before the first word. Falls back to the ordinary
# request/response path automatically if the server has no streaming endpoint
# or the stream fails before anything has been spoken.
STREAMING_REPLIES = os.environ.get("STREAMING_REPLIES", "true").lower() in ("1", "true", "yes", "on")
# ── streaming TTS ───────────────────────────────────────────────────────────
# ElevenLabs' /stream endpoint + chunked playback: the pet starts talking
# after the first PCM chunk instead of after the whole clip is synthesized.
TTS_STREAMING = os.environ.get("TTS_STREAMING", "true").lower() in ("1", "true", "yes", "on")
# ── screen context ──────────────────────────────────────────────────────────
# Appends the active window's title to what you say, so "what's this error?"
# has a referent. The desk API takes text only, so this is a text annotation
# (no screenshot upload).
SCREEN_CONTEXT = os.environ.get("SCREEN_CONTEXT", "true").lower() in ("1", "true", "yes", "on")
# ── monitors ────────────────────────────────────────────────────────────────
# A one-line note about the screen layout (how many, their sizes, which one
# the pet is standing on) rides along with each utterance, so Bolt can decide
# to `petctl jump` somewhere without asking you what you've got plugged in.
# Cheap — the list comes from the UI, nothing is probed per turn.
MONITOR_CONTEXT = os.environ.get("MONITOR_CONTEXT", "true").lower() in (
"1", "true", "yes", "on"
)
# ── screen text (OCR) ───────────────────────────────────────────────────────
# Lets Bolt actually read a monitor, via `petctl read`. Pull-only: nothing is
# captured unless the server asks for it, and every read is logged. Needs the
# optional capture/OCR extras — see the comments in requirements.txt.
#
# This widens what can leave the machine more than any other switch here: the
# recognised text of a whole screen goes to the server. It is *not* a new
# capability (the shell relay could already run a screenshot tool and OCR it),
# but it is a much easier one to use by accident. Set SCREEN_TEXT=false to
# take it away entirely.
SCREEN_TEXT = os.environ.get("SCREEN_TEXT", "true").lower() in ("1", "true", "yes", "on")
SCREEN_TEXT_MAX_CHARS = int(os.environ.get("SCREEN_TEXT_MAX_CHARS", "4000"))
# ── quiet hours / do-not-disturb ────────────────────────────────────────────
# Comma-separated HH:MM-HH:MM ranges (wrapping midnight is fine). While
# napping the pet dims, stops wandering, and makes no proactive noise —
# wake word and click-to-talk still work.
QUIET_HOURS = os.environ.get("QUIET_HOURS", "")
DND_ON_FULLSCREEN = os.environ.get("DND_ON_FULLSCREEN", "true").lower() in ("1", "true", "yes", "on")
# ── desktop notification bridge ─────────────────────────────────────────────
# Mirrors desktop notifications to the server so Bolt can react to them.
# Off by default: every forwarded notification costs a converse() round trip.
NOTIFICATION_BRIDGE = os.environ.get("NOTIFICATION_BRIDGE", "false").lower() in ("1", "true", "yes", "on")
# Regex matched against "<app>: <summary> <body>"; empty means "everything".
NOTIFICATION_FILTER = os.environ.get("NOTIFICATION_FILTER", "")
NOTIFICATION_MIN_INTERVAL_SECONDS = float(os.environ.get("NOTIFICATION_MIN_INTERVAL_SECONDS", "60"))
# ── file delivery ────────────────────────────────────────────────────────
# The server's deliver_files tool (ai/desk_api.py in the main tmn-api repo)
# queues workspace files on this session — e.g. "send me that report" — for
# the client to fetch via GET /desk/files. Downloading a file dequeues it
# server-side, so each one lands here exactly once.
RECEIVE_FILES = os.environ.get("RECEIVE_FILES", "true").lower() in ("1", "true", "yes", "on")
DELIVERED_FILES_DIR = Path(
os.environ.get("DELIVERED_FILES_DIR") or str(Path.home() / "Downloads" / "Bolt")
).expanduser()
# ── conversation history ────────────────────────────────────────────────────
HISTORY_LIMIT = int(os.environ.get("HISTORY_LIMIT", "100"))
# ── push-to-talk ────────────────────────────────────────────────────────────
# Global hotkey (needs `pynput`; unavailable on most Wayland sessions, in
# which case it logs once and the wake word / tray still work). Empty to
# disable.
PUSH_TO_TALK_HOTKEY = os.environ.get("PUSH_TO_TALK_HOTKEY", "ctrl+alt+space")
# ── wake-word tuning ────────────────────────────────────────────────────────
# Scores this far below the threshold are recorded as "near misses" and shown
# in the tray's wake-word tuner, so the threshold can be set from evidence.
WAKE_NEAR_MISS_MARGIN = float(os.environ.get("WAKE_NEAR_MISS_MARGIN", "0.2"))
WAKE_NEAR_MISS_LIMIT = int(os.environ.get("WAKE_NEAR_MISS_LIMIT", "40"))
# ── auto-update ─────────────────────────────────────────────────────────────
# Watches the Gitea releases API for a tag newer than bolt_pet.__version__,
# then `git checkout`s it in place and restarts (see updater.py). The install
# has to be a git clone with a clean working tree — a dirty tree is skipped
# rather than stashed, so local edits are never thrown away. Any failure
# after checkout rolls back to the ref that was checked out before.
AUTO_UPDATE = os.environ.get("AUTO_UPDATE", "true").lower() in ("1", "true", "yes", "on")
UPDATE_REPO_API = os.environ.get(
"UPDATE_REPO_API",
"https://git.themajesticnetwork.com/api/v1/repos/TheMajesticNetwork/Bolt-Pet",
).rstrip("/")
UPDATE_CHECK_INTERVAL_SECONDS = float(os.environ.get("UPDATE_CHECK_INTERVAL_SECONDS", "3600"))
UPDATE_GIT_REMOTE = os.environ.get("UPDATE_GIT_REMOTE", "origin")
# Only needed if the repo is private — releases on a public repo read fine
# anonymously. A Gitea access token with read:repository.
UPDATE_TOKEN = os.environ.get("UPDATE_TOKEN", "")
# Reinstall requirements.txt when an update changes it. Off means a release
# that adds a dependency will roll straight back on the import smoke test.
UPDATE_INSTALL_DEPS = os.environ.get("UPDATE_INSTALL_DEPS", "true").lower() in ("1", "true", "yes", "on")
SAMPLE_RATE = 16000 # mic capture / STT rate
FRAME_LEN = 1280 # 80ms @ 16kHz — matches bolt_desk.py's chunking
# ── pet window ───────────────────────────────────────────────────────────────
PET_SIZE = int(os.environ.get("PET_SIZE", "160")) # on-screen pixel size (square)
PET_START_X = os.environ.get("PET_START_X", "") # blank = bottom-right of primary screen
PET_START_Y = os.environ.get("PET_START_Y", "")
PET_ALWAYS_ON_TOP = os.environ.get("PET_ALWAYS_ON_TOP", "true").lower() in ("1", "true", "yes", "on")
IDLE_ANIMATION_FPS = float(os.environ.get("IDLE_ANIMATION_FPS", "6"))
# ── wandering ────────────────────────────────────────────────────────────────
# The pet strolls to a random spot on its own while idle. Only ever moves when
# it's IDLE and not speaking/being dragged, so it never walks out from under a
# speech bubble mid-sentence.
PET_WANDER = os.environ.get("PET_WANDER", "true").lower() in ("1", "true", "yes", "on")
# Average seconds of standing still between strolls (each wait is randomized
# to 0.5x–1.5x this, so the pet doesn't move on an obvious metronome).
PET_WANDER_INTERVAL_SECONDS = float(os.environ.get("PET_WANDER_INTERVAL_SECONDS", "45"))
PET_WANDER_SPEED = float(os.environ.get("PET_WANDER_SPEED", "90")) # pixels/second
# Cap on how far one stroll can be, so it doesn't teleport across a 4K screen.
PET_WANDER_MAX_DISTANCE = float(os.environ.get("PET_WANDER_MAX_DISTANCE", "600"))
PET_WANDER_MARGIN = int(os.environ.get("PET_WANDER_MARGIN", "20")) # keep off screen edges
# ── mouse behaviour ─────────────────────────────────────────────────────────
# The sprite is drawn in a square translucent window, so its transparent
# corners would otherwise swallow clicks meant for whatever is underneath.
# PET_SHAPED_INPUT masks the window's input region to the sprite's own opaque
# pixels; PET_CLICK_THROUGH goes further and makes the whole pet ignore the
# mouse (tray-only control until you turn it back off).
PET_SHAPED_INPUT = os.environ.get("PET_SHAPED_INPUT", "true").lower() in ("1", "true", "yes", "on")
PET_CLICK_THROUGH = os.environ.get("PET_CLICK_THROUGH", "false").lower() in ("1", "true", "yes", "on")
# Snap flush to a screen edge when dropped/parked within this many pixels of it.
# Start where it was left rather than in the bottom-right corner. Ignored when
# PET_START_X/Y pin it explicitly, and a saved position on a monitor that is no
# longer plugged in is discarded rather than hiding the pet offscreen.
PET_REMEMBER_POSITION = os.environ.get("PET_REMEMBER_POSITION", "true").lower() in ("1", "true", "yes", "on")
PET_EDGE_SNAP = os.environ.get("PET_EDGE_SNAP", "true").lower() in ("1", "true", "yes", "on")
PET_SNAP_MARGIN = int(os.environ.get("PET_SNAP_MARGIN", "48"))
def is_configured() -> bool:
return bool(SERVER_URL and API_KEY)
def missing_config() -> list[str]:
missing = []
if not SERVER_URL:
missing.append("BOLT_SERVER_URL")
if not API_KEY:
missing.append("DESK_API_KEY")
return missing
bolt_pet/controller.py
-901
View File
@@ -1,901 +0,0 @@
"""Orchestrates the pet's mic -> wake word -> STT -> server -> TTS pipeline.
Runs on a background QThread (see ui/app.py) so the Qt event loop / window
painting is never blocked by audio I/O or network calls. Talks to the UI
only through Qt signals (state_changed / said / log / action / napping),
which Qt marshals safely across threads — this class never touches a QWidget
directly.
Beyond the core loop it owns the side channels that let the pet act on its
own: the heartbeat (proactive announcements), the desktop notification
bridge, quiet hours, barge-in, and the live wake-word threshold.
"""
from __future__ import annotations
import threading
import time
from typing import Optional
from PySide6.QtCore import QObject, Signal
from . import (
config, dialogue as dialogue_mod, speech, file_delivery, file_ops,
history as history_mod, monitors as monitors_mod, notifications,
pet_actions, quiet, screen_context, screen_text, self_restart,
server_client, speech_text, updater,
)
from . import __version__
from .audio import barge_in, mic, stt, stt_stream, tts, wake_word
from .state import PetState, PetStateMachine
# A burst while the pet is busy is batched into one turn, but the queue still
# needs a ceiling — a notification storm must not become an unbounded backlog
# that gets read out minutes later.
_MAX_PENDING_NOTIFICATIONS = 12
# How often to re-check whether the pet should be napping. The fullscreen
# probe shells out to xprop, so this deliberately isn't every heartbeat tick.
_NAP_CHECK_INTERVAL_SECONDS = 10.0
class PetController(QObject):
state_changed = Signal(str) # PetState.value
said = Signal(str) # text now showing in the speech bubble
log = Signal(str)
action = Signal(dict) # parsed petctl action for the UI to perform
napping = Signal(bool) # quiet hours / fullscreen do-not-disturb
voice_changed = Signal(str) # name of the server-picked voice ("" = default)
mouth = Signal(float) # 0..1 speech loudness, for lip-sync while talking
restart_requested = Signal(str) # version we just updated to
finished = Signal()
def __init__(self):
super().__init__()
self._running = True
self._muted = False
self._talk_now = threading.Event()
self._stream = None
self._last_heartbeat = 0.0
self._state = PetStateMachine(on_change=self._handle_state_change)
# Conversation scrollback, shared read-only with the UI's History
# window. Append-only from this thread; the UI only ever snapshots it.
self.history = history_mod.ConversationHistory(limit=config.HISTORY_LIMIT)
# The screen layout, as published by the UI (see set_monitors). Held
# here rather than probed, so "monitor 2" means the same thing to the
# controller and to the window that has to jump there — see
# monitors.py for why that matters.
self._monitors: list[monitors_mod.Monitor] = []
self._pet_monitor: Optional[int] = None
# Wake-word sensitivity is live-tunable (tray tuner), so it's read
# through a callable on every frame rather than captured per listen.
self._wake_threshold = config.WAKE_WORD_THRESHOLD
self._near_misses = wake_word.NearMissLog()
# The voice the server last picked for us with `speak_as` ("" = the
# configured default). Held here rather than passed straight through
# to one tts.speak() call because it's sticky by default — see
# _apply_voice for why.
self._voice_id = ""
self._voice_name = ""
self._barge_in: Optional[barge_in.BargeInDetector] = None
# Everything the pet says goes through here; see speech.py for why the
# four hand-rolled copies of this became one.
self._speaker = speech.Speaker(
state=self._state, tts=tts,
history=lambda text: self.history.add(history_mod.PET, text, time.time()),
on_said=self.said.emit, on_log=self.log.emit,
barge_in=lambda: self._barge_in,
detail_of=self._barge_in_detail,
voice_id=lambda: self._voice_id,
on_level=self.mouth.emit,
)
self._napping = False
self._nap_forced: Optional[bool] = None # petctl nap on/off overrides the schedule
self._last_nap_check = 0.0
# When a reply ends on a question the pet keeps listening for the
# answer. _follow_ups counts how many have chained without you
# re-triggering, so a server that ends every reply with "?" can't
# loop forever off mic noise.
self._pending_follow_up = False
self._follow_ups = 0
self._last_update_check = 0.0
self._update_pending = False # applied on disk, waiting for the restart
# Armed by `petctl self_restart`, fired after the turn it was asked in
# (see _arm_self_restart for why it can't happen inline).
self._restart_context = None
self._notification_watcher: Optional[notifications.NotificationWatcher] = None
self._notification_gate = notifications.NotificationGate(
config.NOTIFICATION_FILTER, config.NOTIFICATION_MIN_INTERVAL_SECONDS
)
self._pending_notifications: list[notifications.Notification] = []
self._notification_lock = threading.Lock()
# ── external controls (safe to call from the Qt/UI thread) ─────────
def request_talk_now(self) -> None:
self._talk_now.set()
def toggle_mute(self) -> bool:
self._muted = not self._muted
self.log.emit("Muted." if self._muted else "Unmuted.")
return self._muted
def set_napping(self, napping: Optional[bool]) -> None:
"""Force the nap state on/off, or pass None to hand control back to
the quiet-hours schedule."""
self._nap_forced = napping
if napping is not None:
self._apply_nap_state(napping)
def wake_threshold(self) -> float:
return self._wake_threshold
def set_wake_threshold(self, value: float) -> None:
self._wake_threshold = min(max(float(value), 0.01), 0.99)
def wake_stats(self) -> dict:
return {"peak": self._near_misses.peak, "near_misses": self._near_misses.entries()}
def reset_wake_stats(self) -> None:
self._near_misses.clear()
def current_voice(self) -> str:
"""Name (or id) of the server-picked voice in use, "" for the default."""
return self._voice_name or self._voice_id
def reset_voice(self) -> None:
"""Drop a server-picked voice and go back to Bolt's own. The tray's
way out of a voice you didn't want to keep — the server has no way to
ask for the default back, since it never learns what it is."""
if not self._voice_id:
return
self._voice_id = self._voice_name = ""
self.log.emit("Voice: back to the default.")
self.voice_changed.emit("")
def stop(self) -> None:
self._running = False
self._talk_now.set() # wake up anything blocked waiting on it
if self._notification_watcher is not None:
self._notification_watcher.stop()
# ── internal ─────────────────────────────────────────────────────────
def _handle_state_change(self, _old: PetState, new: PetState) -> None:
self.state_changed.emit(new.value)
def _should_continue(self) -> bool:
return self._running
def run(self) -> None:
"""Thread entry point (connected to QThread.started)."""
missing = config.missing_config()
if missing:
self.log.emit(f"Missing config: {', '.join(missing)} — set them in .env and restart.")
self.finished.emit()
return
try:
self._stream = mic.open_input_stream()
except Exception as exc:
self.log.emit(f"Could not open microphone: {exc}")
self.finished.emit()
return
if config.BARGE_IN:
# In wake mode the detector shares the idle listener's model and
# its live threshold, so the tray tuner's slider applies to
# interrupting as well as waking (unless BARGE_IN_WAKE_THRESHOLD
# pins it to a fixed, stricter number).
self._barge_in = barge_in.make_detector(
self._stream,
wake_threshold=(
config.BARGE_IN_WAKE_THRESHOLD
if config.BARGE_IN_WAKE_THRESHOLD is not None
else self.wake_threshold
),
)
self.log.emit(f"Barge-in: {config.BARGE_IN_MODE} mode.")
with self._stream:
try:
health = server_client.check_health()
self.log.emit(f"Connected to server: {health}")
except Exception as exc:
self.log.emit(f"Server not reachable yet ({exc}) — will keep trying per-request.")
self._start_notification_bridge()
self._report_self_restart()
self._loop()
if self._notification_watcher is not None:
self._notification_watcher.stop()
self.finished.emit()
def _loop(self) -> None:
while self._running:
if self._muted:
triggered = self._talk_now.wait(timeout=0.5)
if not self._running:
return
if not triggered:
continue
self._talk_now.clear()
else:
if not self._wait_for_wake_or_click():
if not self._running:
return
continue
self._handle_conversation_turn()
def _wait_for_wake_or_click(self) -> bool:
"""True once either the wake phrase was heard or a click-to-talk
request came in; False on a spurious wakeup (loop again)."""
def should_continue() -> bool:
return self._running and not self._talk_now.is_set()
detected = wake_word.listen_for_wake_word(
self._stream,
should_continue=should_continue,
threshold=self.wake_threshold, # callable: the tuner slider is live
on_tick=self._maybe_heartbeat,
on_score=self._observe_wake_score,
)
if not self._running:
return False
if detected:
self._talk_now.clear() # in case both fired around the same time
return True
if self._talk_now.is_set():
self._talk_now.clear()
return True
return False
def _observe_wake_score(self, score: float, threshold: float) -> None:
self._near_misses.observe(score, threshold, time.time())
def _handle_conversation_turn(self) -> None:
# A turn you started yourself ends any follow-up chain in progress.
following_up, self._pending_follow_up = self._pending_follow_up, False
if not following_up:
self._follow_ups = 0
self._state.transition(PetState.LISTENING)
# Transcribe while they talk rather than after: frames go to Deepgram
# as they are captured, so the text is ready the moment the VAD says
# they stopped. None here just means the one-shot path will do it.
streamed = stt_stream.StreamingTranscriber.open()
pcm = mic.record_utterance(
self._stream,
should_continue=self._should_continue,
# Answering a question deserves longer than saying the wake word
# on purpose does — you were just asked something.
grace_s=config.FOLLOW_UP_GRACE_SECONDS if following_up else None,
on_frame=streamed.feed if streamed is not None else None,
)
if pcm is None:
if streamed is not None:
streamed.finish()
self._follow_ups = 0 # silence ends the chain
self._state.transition(PetState.IDLE)
return
self._state.transition(PetState.THINKING)
try:
text = self._transcribe(pcm, streamed)
except stt.SttError as exc:
self.log.emit(f"STT failed: {exc}")
self._state.transition(PetState.ERROR)
self._state.transition(PetState.IDLE)
return
if not text:
self._state.transition(PetState.IDLE)
return
self.log.emit(f"You: {text}")
self.history.add(history_mod.USER, text, time.time())
try:
# What's focused right now rides along, so "what's this error?"
# has a referent without you having to describe the window.
reply = self._ask_server(self._with_context(text))
except server_client.ServerError as exc:
self.log.emit(f"Server error: {exc}")
self._state.transition(PetState.ERROR)
self._state.transition(PetState.IDLE)
return
self._check_deliveries()
self._apply_voice(reply)
if reply.spoken:
# Streamed: every sentence was spoken and logged as it arrived.
# The text still matters — a reply ending on a question should keep
# the mic open — but saying it again would repeat the whole answer.
self._after_speaking(reply.text, completed=True)
else:
self._speak(reply.text)
self._state.transition(PetState.IDLE)
self._maybe_self_restart()
def _transcribe(self, pcm, streamed) -> str:
"""The transcript, from the live stream if it produced one.
The fallback is not a rare path to be tolerated — it is the safety net
that lets streaming be switched on at all. Whatever happened to the
socket, the full audio is still buffered here, so a failed stream costs
one ordinary upload and nothing else."""
if streamed is not None:
try:
text = streamed.finish()
except Exception:
self.log.emit("Streaming STT failed — falling back.")
text = ""
if text:
self.log.emit("(transcribed while you spoke)")
return text
return stt.transcribe(pcm)
def _with_context(self, text: str) -> str:
"""Everything the server gets alongside what you actually said: the
focused window title, and a one-line note about the screen layout so
Bolt knows how many monitors there are and where he's standing
without having to ask. Only the *layout* rides along for free — the
text on those screens costs an OCR pass, so it stays behind
`petctl read`."""
text = screen_context.context_for(text)
if config.MONITOR_CONTEXT:
text = monitors_mod.annotate(text, self._monitors, self._pet_monitor)
return text
# ── screen layout, published by the UI ───────────────────────────────
def set_monitors(self, monitors: list) -> None:
"""Slot: the window telling us what screens exist (queued signal)."""
self._monitors = list(monitors)
self.log.emit(
"Screens: " + (monitors_mod.summary(self._monitors) or "none reported")
)
def set_pet_monitor(self, index: int) -> None:
"""Slot: the window telling us which screen the pet is standing on."""
self._pet_monitor = int(index)
def _handle_command(self, command: str) -> str:
"""Server-relayed command. `petctl ...` drives the pet's body and
`filectl ...` does local file read/write/edit — neither ever reaches
a shell; everything else is a real command, exactly as before (see
the security notes in the README)."""
try:
action = pet_actions.parse(command)
except pet_actions.ActionError as exc:
self.log.emit(f"petctl: {exc}")
return f"[pet] {exc}"
if action is not None:
self.log.emit(f"Pet action: {action}")
# Queries answer from here rather than from pet_actions.describe():
# their output *is* the useful part, and it's what the server reads
# back off the tool-result relay.
kind = action["action"]
if kind == "monitors":
return monitors_mod.describe(self._monitors, self._pet_monitor)
if kind == "self_restart":
return self._arm_self_restart(action.get("reason") or "")
if kind == "voice":
# Answered here, not by describe(): the UI has no part in it,
# and the server needs to hear whether there was anything to
# drop — it can't see which voice we're using.
previous = self.current_voice()
self.reset_voice()
return (
f"[pet] back to your own voice (was {previous})" if previous
else "[pet] already using your own voice"
)
if kind == "read":
return self._read_screen(action["target"])
if kind == "jump":
try:
target = monitors_mod.resolve(
self._monitors, action["target"], self._pet_monitor
)
except ValueError as exc:
self.log.emit(f"petctl jump: {exc}")
return f"[pet] {exc}"
# Hand the window a resolved index, so it can't re-resolve the
# spec against a different screen ordering.
self.action.emit({"action": "jump", "monitor": target.index})
return f"[pet] jumped to monitor {target.label}"
if kind == "nap":
self.set_napping(bool(action["enabled"]))
self.action.emit(action)
return pet_actions.describe(action)
try:
scene = dialogue_mod.parse(command)
except dialogue_mod.DialogueError as exc:
self.log.emit(f"dialoguectl: {exc}")
return f"[dialogue] {exc}"
if scene is not None:
return self._play_dialogue(scene)
try:
file_action = file_ops.parse(command)
except file_ops.FileOpError as exc:
self.log.emit(f"filectl: {exc}")
return f"[filectl] {exc}"
if file_action is not None:
self.log.emit(file_ops.describe(file_action))
try:
return file_ops.execute(file_action)
except file_ops.FileOpError as exc:
self.log.emit(f"filectl: {exc}")
return f"[filectl] {exc}"
return server_client.run_local_command(command)
def _read_screen(self, target: str) -> str:
"""`petctl read` — OCR a screen and hand the text back to the server."""
if not config.SCREEN_TEXT:
return "[pet] screen reading is disabled (set SCREEN_TEXT=true in .env)"
if not self._monitors:
return "[pet] no monitor information available"
limit = config.SCREEN_TEXT_MAX_CHARS
if target in ("all", "everything", "*"):
self.log.emit(f"Reading all {len(self._monitors)} screens…")
return screen_text.read_monitors(self._monitors, limit)
if target in ("here", "", "this", "current"):
index = self._pet_monitor if self._pet_monitor is not None else 0
monitor = self._monitors[min(index, len(self._monitors) - 1)]
else:
try:
monitor = monitors_mod.resolve(
self._monitors, target, self._pet_monitor
)
except ValueError as exc:
return f"[pet] {exc}"
self.log.emit(f"Reading monitor {monitor.number} ({monitor.name})…")
return screen_text.read_monitor(monitor, limit)
def _arm_self_restart(self, reason: str) -> str:
"""`petctl self_restart` — check the code, then arm a restart.
Nothing restarts here. The tool result has to get back up the relay
before this process can die (otherwise the server waits out its
timeout on a turn that will never finish), so the restart is armed and
`_maybe_self_restart` fires it once the turn has been spoken. The
preflight import runs *now*, in this turn, so a syntax error Bolt just
introduced comes back as something he can read and fix rather than as
a pet that never comes back."""
if not config.SELF_RESTART:
return "[pet] self-restart is disabled on this device (SELF_RESTART=false)"
if self._restart_context is not None:
return "[pet] a restart is already armed for the end of this turn"
try:
self_restart.check_loop_guard(self_restart.load())
self.log.emit("Self-restart requested — checking the code imports first…")
self_restart.preflight()
except self_restart.RestartError as exc:
self.log.emit(f"Self-restart refused: {exc}")
return f"[pet] restart refused — {exc}"
recent = [entry.text[:120] for entry in self.history.entries()[-4:]]
self._restart_context = self_restart.arm(
reason or "no reason given",
verify=reason,
version=__version__,
session=config.SESSION_ID,
recent=recent,
)
self.log.emit("Self-restart armed; it happens after this turn.")
return (
"[pet] code imports cleanly; restarting as soon as this turn finishes. "
"I'll come back and tell you what version I'm on and what I found — "
"wrap up your reply now, the next thing you hear from me is the report."
)
def _maybe_self_restart(self) -> bool:
"""Fire an armed restart, once the turn is over and the reply spoken.
Returns True if a restart was requested, so the caller can stop
driving the pipeline — the process is on its way out."""
if self._restart_context is None:
return False
self._update_pending = True # same latch the updater uses: no double restart
self.log.emit("Restarting now.")
self._state.force(PetState.IDLE)
self.restart_requested.emit(f"self-restart: {self._restart_context.reason[:60]}")
return True
def _report_self_restart(self) -> None:
"""On the way up: tell the server we're back, and why we left.
Runs once, before the listen loop starts, and only when a context file
was left behind. The report goes through the ordinary conversation
path, so Bolt's answer is spoken out loud like any other turn — which
is what makes "restart and check the sprites load" finish as a
sentence instead of a silence."""
context = self_restart.load()
if context is None:
return
self_restart.clear()
message = self_restart.report(context, version=__version__)
self.log.emit(f"Back from a self-restart ({context.reason[:80]}).")
self.history.add(history_mod.SYSTEM, message, time.time())
try:
reply = server_client.converse(message, on_command=self._handle_command)
except server_client.ServerError as exc:
# The restart still worked; only the report failed. Say so locally
# rather than pretending nothing happened.
self.log.emit(f"Couldn't report the restart to the server: {exc}")
return
self._apply_voice(reply)
if reply.text.strip() and not self._napping:
self._speak(reply.text)
self._state.force(PetState.IDLE)
def _ask_server(self, text: str):
"""One turn with the server, streamed when possible.
Streaming speaks each sentence as it is generated, so the wait is
time-to-first-sentence rather than the whole model call. It falls back
to the ordinary request/response path when the server has no streaming
endpoint, or when a stream dies *before* anything was spoken — after
that, retrying would say the first half twice."""
if config.STREAMING_REPLIES:
try:
return server_client.converse_stream(
text, on_say=self._speak_stream_chunk,
on_command=self._handle_command,
)
except server_client.ServerError as exc:
self.log.emit(f"Streaming unavailable ({exc}) — using the plain path.")
return server_client.converse(
text, on_command=self._handle_command, on_say=self._speak_holding,
)
def _play_dialogue(self, scene: dict) -> str:
"""Play a `dialoguectl` scene and report back up the relay.
This runs *mid-turn* (the server is still waiting on the tool result),
so the pet has to look like it's talking and then go back to waiting —
hence the TALKING → THINKING leg rather than the usual return to IDLE.
Everything a normal reply gets, a scene gets too: the bubble, the
transcript, and barge-in, so a long scene can be talked over exactly
like a long answer."""
if not config.DIALOGUE:
return "[dialogue] disabled on this device (DIALOGUE=false)"
try:
inputs = dialogue_mod.resolve(
scene,
voices=dialogue_mod.parse_voice_map(config.DIALOGUE_VOICES),
self_voice=self._voice_id or config.ELEVENLABS_VOICE_ID,
)
except dialogue_mod.DialogueError as exc:
self.log.emit(f"dialoguectl: {exc}")
return f"[dialogue] {exc}"
text = dialogue_mod.spoken_text(scene)
self.log.emit(f"Dialogue ({len(inputs)} lines): {text[:120]}")
try:
pcm, sample_rate = tts.synthesize_dialogue(
inputs, model_id=scene.get("model"), stability=scene.get("stability")
)
except tts.TtsError as exc:
self.log.emit(f"Dialogue failed: {exc}")
# Reported, not raised: the server can read this, shorten the
# scene or fix the voice, and try again inside the same turn.
return f"[dialogue] couldn't synthesize it: {exc}"
completed = self._speaker.say_pcm(
speech.Utterance.scene(text), pcm=pcm, sample_rate=sample_rate)
if not completed:
return dialogue_mod.describe(scene) + " (interrupted — they talked over it)"
return dialogue_mod.describe(scene)
def _apply_voice(self, reply) -> None:
"""Adopt (or drop) the voice the server tagged this reply with.
The server's `speak_as` marker names an ElevenLabs voice it just
picked — and it adds a Voice Library pick to the account first, so by
the time the id gets here it's usable for TTS. It tags *one* reply,
but the voice sticks by default: the server strips the marker before
storing the turn, so it can't recall the id later, and "keep talking
like that" would otherwise send it searching for a voice all over
again. `VOICE_STICKY=false` makes each pick last exactly one reply.
Untagged replies never *change* the voice — with stickiness on they
just keep whatever's in use, which is what makes the rest of the
conversation stay in the requested voice."""
voice_id = getattr(reply, "voice_id", "")
if voice_id:
if voice_id != self._voice_id:
self._voice_id = voice_id
self._voice_name = getattr(reply, "voice_name", "") or ""
self.log.emit(f"Voice: {self.current_voice()}")
self.voice_changed.emit(self.current_voice())
elif not config.VOICE_STICKY:
self.reset_voice()
def _speak(self, text: str) -> None:
"""The answer: transcript, bubble, and the follow-up rule."""
completed = self._speaker.say(speech.Utterance.reply(text))
self._after_speaking(text, completed=completed,
detail=self._speaker.last_detail)
def _speak_holding(self, text: str) -> None:
""""Give me a sec" while a tool runs — filler, so no transcript, and it
resumes the state it interrupted because the turn isn't over."""
self._speaker.say(speech.Utterance.holding(text))
def _speak_stream_chunk(self, text: str) -> None:
"""One sentence of a streamed answer, spoken the moment it arrives."""
completed = self._speaker.say(speech.Utterance.stream_chunk(text))
if not completed:
self.log.emit("Interrupted — listening.")
self._follow_ups = 0
self._talk_now.set()
def _after_speaking(self, text: str, *, completed: bool, detail: str = "") -> None:
"""What happens once an answer has been said, however it was said.
Shared by the plain and streamed paths: a streamed reply is spoken
sentence by sentence, but it still has to obey the same rules about
keeping the mic open when it ended on a question."""
if not completed:
# You talked over it — take that as the start of the next turn
# rather than making you say the wake word again.
self.log.emit(f"Interrupted — listening. {detail}")
self._follow_ups = 0 # you're clearly engaged; start the count over
self._talk_now.set()
elif self._should_follow_up(text):
self._follow_ups += 1
cap = config.FOLLOW_UP_MAX_TURNS
self.log.emit(
f"Asked a question — listening for your answer "
f"({self._follow_ups}{'/' + str(cap) if cap > 0 else ''})."
)
self._pending_follow_up = True
self._talk_now.set()
def _should_follow_up(self, text: str) -> bool:
"""Whether *text* leaves the pet waiting on an answer.
The rule itself — question, cap, off switch — is
`speech.follow_up_decision`, because it is a rule with an off-by-one in
it and deserves a test that needs no audio. What stays here is the part
that is genuinely the controller's: mute, and the log line. Muted is
excluded because mute means "don't listen to me" and an automatic turn
would walk straight past it. Napping isn't: quiet hours suppress the
pet *starting* something, and a question is only ever asked in reply to
you."""
if self._muted:
return False
keep, why = speech.follow_up_decision(text, completed=True, follow_ups=self._follow_ups)
if not keep and "cap" in why:
# Only worth mentioning the cap on a reply that would otherwise have
# kept listening, or it fires on every statement the pet makes.
self.log.emit("Follow-up limit reached — say the wake word to keep going.")
return keep
def _barge_in_detail(self) -> str:
"""Why the interruption fired, for the log. How far into playback it
happened is the tell: frame 1 means the detector was still holding
audio from before this reply started, whereas a hit several seconds
in is something the mic actually heard."""
detector = self._barge_in
if isinstance(detector, barge_in.WakeWordBargeIn):
return (
f"(wake score {detector.last_score:.3f} >= {detector.last_threshold:.2f}, "
f"peak {detector.peak_score:.3f}, at frame {detector.frames_checked} / "
f"{detector.seconds_checked:.1f}s into playback)"
)
if isinstance(detector, barge_in.BargeInDetector):
return f"(loud frames {detector.loud_frames}, threshold {config.BARGE_IN_RMS_THRESHOLD})"
return ""
# ── quiet hours / do-not-disturb ─────────────────────────────────────
def _apply_nap_state(self, napping: bool) -> None:
if napping == self._napping:
return
self._napping = napping
self.log.emit("Napping — no proactive noise." if napping else "Awake.")
self.napping.emit(napping)
def _refresh_nap_state(self) -> None:
if self._nap_forced is not None:
self._apply_nap_state(self._nap_forced)
return
now = time.monotonic()
if now - self._last_nap_check < _NAP_CHECK_INTERVAL_SECONDS:
return
self._last_nap_check = now
napping = quiet.is_quiet(
config.QUIET_HOURS,
on_error=lambda exc: self.log.emit(f"QUIET_HOURS is malformed ({exc}) — ignoring it."),
)
if not napping and config.DND_ON_FULLSCREEN:
napping = screen_context.is_fullscreen_active()
self._apply_nap_state(napping)
# ── desktop notification bridge ──────────────────────────────────────
def _start_notification_bridge(self) -> None:
if not config.NOTIFICATION_BRIDGE:
return
watcher = notifications.NotificationWatcher(self._queue_notification)
problem = watcher.start()
if problem:
self.log.emit(problem)
return
self._notification_watcher = watcher
self.log.emit("Notification bridge on.")
def _queue_notification(self, notification: notifications.Notification) -> None:
"""Called on the watcher thread — just queue it; forwarding happens on
the pipeline thread where it can't collide with a live conversation.
Only the *filter* applies here. The rate limit used to as well, which
meant a second message arriving inside the window was silently thrown
away — with NOTIFICATION_MIN_INTERVAL_SECONDS=60, two texts a minute
apart and you only ever heard about one of them. Losing a message from
a person to save a round trip is the wrong trade; they are batched at
the far end instead, which costs the same one round trip and keeps
them all."""
if not self._notification_gate.matches(notification):
return
with self._notification_lock:
if len(self._pending_notifications) >= _MAX_PENDING_NOTIFICATIONS:
self._pending_notifications.pop(0) # bound it; oldest goes first
self._pending_notifications.append(notification)
def _drain_notifications(self) -> None:
"""Forward everything waiting as ONE turn.
Batching is what makes it safe to keep every notification: five that
arrived while the pet was mid-conversation become one message and one
round trip, instead of five separate interruptions queued up to fire
back to back."""
with self._notification_lock:
pending, self._pending_notifications = self._pending_notifications, []
if not pending or not self._running or self._napping:
return
for notification in pending:
self.log.emit(f"Notification: {notification.as_text()}")
self.history.add(history_mod.SYSTEM, notification.as_text(), time.time())
if len(pending) == 1:
message = f"[desktop notification] {pending[0].as_text()}"
else:
lines = "\n".join(f"- {n.as_text()}" for n in pending)
message = f"[{len(pending)} desktop notifications]\n{lines}"
try:
reply = server_client.converse(
message, on_command=self._handle_command, on_say=self._speak_holding,
)
except server_client.ServerError as exc:
self.log.emit(f"Couldn't forward notification: {exc}")
return
self._check_deliveries()
self._apply_voice(reply)
if reply.text.strip() and not reply.spoken:
self._speak(reply.text)
self._state.transition(PetState.IDLE)
# ── file delivery ────────────────────────────────────────────────────
def _check_deliveries(self) -> None:
"""Download anything the server has queued via deliver_files —
called right after a conversation/notification turn (the common
case: "send me that file") and once per heartbeat for anything
queued out-of-band. Best-effort: a failure here is logged, not
raised, so it can't sour a turn that already got its spoken reply."""
if not config.RECEIVE_FILES:
return
try:
queued = server_client.list_outbox_files()
except server_client.ServerError as exc:
self.log.emit(f"Couldn't check for delivered files: {exc}")
return
for entry in queued:
file_id = entry.get("id")
name = entry.get("name") or file_id
if not file_id:
continue
try:
data = server_client.download_outbox_file(file_id)
except server_client.ServerError as exc:
self.log.emit(f"Couldn't download {name}: {exc}")
continue
path = file_delivery.save(config.DELIVERED_FILES_DIR, name, data)
self.log.emit(f"Received file: {path}")
# ── auto-update ──────────────────────────────────────────────────────
def _maybe_update(self) -> None:
"""Poll the Gitea releases page and, if there's a newer tag, apply it
and ask the UI to restart.
Only ever runs from the wake-listener's tick, so the pet is IDLE and
between turns by construction — an update can't land mid-sentence.
Failures are logged and the interval resets, so a server that's down
(or a release that rolls back) costs one log line an hour, not a
retry storm."""
if not config.AUTO_UPDATE or self._update_pending:
return
now = time.monotonic()
if now - self._last_update_check < config.UPDATE_CHECK_INTERVAL_SECONDS:
return
self._last_update_check = now
try:
release = updater.check_for_update()
except updater.UpdateError as exc:
self.log.emit(f"Update check failed: {exc}")
return
if release is None:
return
self.log.emit(f"Update available: {release.tag} — applying.")
try:
previous = updater.apply_update(release.tag, on_log=self.log.emit)
except updater.UpdateError as exc:
self.log.emit(f"Update to {release.tag} failed: {exc}")
return
self._update_pending = True
self.log.emit(f"Updated {previous} -> {release.tag}; restarting.")
if not self._napping:
# Napping means no proactive noise, so a silent restart it is.
self._speak(f"Updating to {release.tag}. Back in a second.")
self._state.transition(PetState.IDLE)
self.restart_requested.emit(release.tag)
# ── heartbeat ────────────────────────────────────────────────────────
def _maybe_heartbeat(self) -> None:
"""Called from the wake listener's tick (~every WAKE_CHECK_INTERVAL_SECONDS).
Two different cadences live here, and conflating them was costing
minutes. A *notification* is an event that already happened — it should
go out as soon as the pet is free, which is the next tick. The
*heartbeat* is a poll, and polling the server every 1.2s would be
absurd, so it stays on its own interval."""
self._refresh_nap_state()
self._maybe_update()
if self._update_pending:
return # on the way out — don't start a conversation now
# Notifications: every tick, not every heartbeat.
if self._state.state == PetState.IDLE and not self._napping:
self._drain_notifications()
now = time.monotonic()
if now - self._last_heartbeat < config.HEARTBEAT_INTERVAL_SECONDS:
return
if self._state.state != PetState.IDLE:
# Mid-conversation. Do NOT stamp the clock — an earlier version
# did, so a heartbeat that landed while the pet was talking burned
# its slot and waited another full interval. With follow-up
# listening that could repeat for several cycles, which is why a
# notification could sit unspoken for five or ten minutes.
return
if self._napping:
return # quiet hours: still answers when spoken to, just doesn't start
self._last_heartbeat = now
self._check_deliveries()
if self._state.state != PetState.IDLE:
return
try:
announcement = server_client.report_status()
except server_client.ServerError as exc:
self.log.emit(f"Heartbeat failed: {exc}")
return
if announcement:
self._speak(announcement)
self._state.transition(PetState.IDLE)
bolt_pet/dialogue.py
-235
View File
@@ -1,235 +0,0 @@
"""`dialoguectl` — multi-voice dialogue playback (ElevenLabs Text to Dialogue).
Normal replies are one voice saying one thing (audio/tts.py). This is the
other mode: a short *scene* — two or more voices, with delivery tags the v3
model acts on (`[cheerfully]`, `[stuttering]`, `[whispering]`) — synthesized
as a single take so the timing and reactions between lines actually sound
like a conversation rather than clips glued together.
Wire format, the same discipline as file_ops.py and for the same reason: it
rides the server's ordinary `command` tool marker, whose extractor only
captures up to the next newline, so the payload is a **single-line compact
JSON object**.
dialoguectl {"lines": [{"voice": "self", "text": "[cheerfully] Morning!"},
{"voice": "narrator", "text": "[whispering] He lies."}]}
The ElevenLabs field names are accepted too (`inputs` / `voice_id`), because
the model has read that API and copying its shape is the obvious thing to
try:
dialoguectl {"inputs": [{"voice_id": "9BWtsMINqrJLrRacOk9x", "text": "hi"}]}
Voices are *named*, not pasted as ids. `DIALOGUE_VOICES` in .env maps names
to ids (`narrator:9BWts…,villain:IKne3…`), and `self` always means the voice
the pet is speaking with right now — including a voice the server picked
mid-conversation with `speak_as`, so a scene featuring Bolt sounds like
whoever Bolt currently is.
Pure parsing and validation here; the HTTP call is
`audio/tts.synthesize_dialogue` and the playback/state handling is
`controller._play_dialogue`, matching the parse/execute split used by
pet_actions.py and file_ops.py.
The API's own limits are enforced *here*, before the request goes out, so a
mistake comes back through the tool-result relay as a sentence Bolt can act
on ("too many characters, split it") rather than as an HTTP 422 he can't see.
"""
from __future__ import annotations
import json
import re
from typing import Iterable, Optional
from . import relay_json
_PREFIXES = ("dialoguectl", "dialogue", "scene")
# ElevenLabs Text to Dialogue limits (docs, 2026-07): at most 10 distinct
# voice ids per request and ~2000 characters across all inputs.
MAX_VOICES = 10
MAX_CHARS = 2000
# Names that always mean "the voice the pet is using right now".
SELF_NAMES = ("self", "bolt", "me", "pet")
# A raw ElevenLabs voice id: 20 URL-safe characters, no separators. Used to
# tell "the model pasted an id" from "the model used a name".
_VOICE_ID_RE = re.compile(r"^[A-Za-z0-9]{20}$")
class DialogueError(Exception):
"""Bad dialoguectl syntax or an unusable request — reported back to the
server as this command's output."""
def is_dialogue_command(command: str) -> bool:
parts = (command or "").strip().split(None, 1)
return bool(parts) and parts[0].lower() in _PREFIXES
def parse(command: str) -> Optional[dict]:
"""Parse `dialoguectl <json>` into {"lines": [{"voice", "text"}], ...}.
Returns None if this isn't a dialogue command at all (the caller then
tries filectl, then a real shell command). Raises DialogueError on a
dialogue command that doesn't make sense."""
if not is_dialogue_command(command):
return None
_, _, payload = (command or "").strip().partition(" ")
payload = payload.strip()
if not payload:
raise DialogueError(
'dialoguectl needs a JSON argument, e.g. dialoguectl {"lines": '
'[{"voice": "self", "text": "[cheerfully] hello"}]}'
)
# Same lenient parse as filectl (see relay_json): machine-written JSON
# fails in a handful of repeatable ways, and a stray quote shouldn't cost
# a turn — but the repair is reported back rather than hidden.
try:
data, repairs = relay_json.loads(payload)
except relay_json.RelayJsonError as exc:
raise DialogueError(
f"couldn't parse the JSON — {exc}\nIt must be one line of compact "
"JSON — put line breaks inside text as \\n, never as real newlines."
) from exc
if not isinstance(data, dict):
raise DialogueError("the argument must be a JSON object, not a list or a bare value")
raw_lines = data.get("lines")
if raw_lines is None:
raw_lines = data.get("inputs") # the ElevenLabs field name
if not isinstance(raw_lines, list) or not raw_lines:
raise DialogueError('needs a non-empty "lines" array of {"voice", "text"} objects')
lines: list[dict] = []
for index, entry in enumerate(raw_lines, start=1):
if not isinstance(entry, dict):
raise DialogueError(f"line {index} must be an object with 'voice' and 'text'")
text = str(entry.get("text") or "").strip()
if not text:
raise DialogueError(f"line {index} has no text")
voice = str(entry.get("voice") or entry.get("voice_id") or "self").strip()
lines.append({"voice": voice, "text": text})
action = {"action": "dialogue", "lines": lines}
if repairs:
action["_repairs"] = repairs
model = str(data.get("model") or data.get("model_id") or "").strip()
if model:
action["model"] = model
stability = data.get("stability")
if stability is not None:
try:
action["stability"] = min(1.0, max(0.0, float(stability)))
except (TypeError, ValueError):
raise DialogueError("stability must be a number between 0 and 1") from None
return action
def parse_voice_map(spec: str) -> dict[str, str]:
"""Parse DIALOGUE_VOICES ("narrator:9BWts…, villain:IKne3…") into a map.
Malformed entries are skipped rather than raising: a typo in .env should
cost that one voice, not the whole feature."""
voices: dict[str, str] = {}
for chunk in str(spec or "").split(","):
name, separator, voice_id = chunk.partition(":")
name, voice_id = name.strip().lower(), voice_id.strip()
if separator and name and voice_id:
voices[name] = voice_id
return voices
def resolve(
action: dict,
*,
voices: Optional[dict] = None,
self_voice: str = "",
) -> list[dict]:
"""Turn parsed lines into the API's `inputs`, resolving names to ids.
*self_voice* is the pet's current voice (which may be a `speak_as` pick,
not the configured default), so "self" tracks whoever Bolt sounds like
right now."""
known = dict(voices or {})
resolved: list[dict] = []
for index, line in enumerate(action.get("lines") or [], start=1):
name = str(line.get("voice") or "self")
key = name.lower()
if key in SELF_NAMES:
voice_id = self_voice
if not voice_id:
raise DialogueError(
"no voice is configured for the pet itself — set "
"ELEVENLABS_VOICE_ID, or name a voice from DIALOGUE_VOICES"
)
elif key in known:
voice_id = known[key]
elif _VOICE_ID_RE.match(name):
voice_id = name # a raw id pasted straight from the voice library
else:
available = ", ".join(sorted(known) + list(SELF_NAMES[:1])) or "self"
raise DialogueError(
f"line {index}: unknown voice {name!r}. Known names: {available}. "
"Use one of those, 'self' for your own voice, or a raw voice id."
)
resolved.append({"text": str(line.get("text") or ""), "voice_id": voice_id})
check_limits(resolved)
return resolved
def check_limits(inputs: Iterable[dict], *, max_voices: int = MAX_VOICES,
max_chars: int = MAX_CHARS) -> None:
"""Enforce the API's own limits before spending a request on a 422."""
entries = list(inputs)
if not entries:
raise DialogueError("no lines to speak")
distinct = {entry["voice_id"] for entry in entries}
if len(distinct) > max_voices:
raise DialogueError(
f"{len(distinct)} different voices — the limit is {max_voices} per scene"
)
total = sum(len(entry["text"]) for entry in entries)
if total > max_chars:
raise DialogueError(
f"{total} characters — the limit is {max_chars} per scene. "
"Split it into two dialoguectl calls."
)
def spoken_text(action: dict) -> str:
"""The scene as readable text, for the speech bubble and the transcript.
Delivery tags are stripped: `[cheerfully]` is a stage direction for the
model, not something to show (or, via tts.speak's sanitizer, to read out)."""
parts = []
for line in action.get("lines") or []:
text = re.sub(r"\[[^\]]{1,40}\]", " ", str(line.get("text") or ""))
text = " ".join(text.split())
if text:
parts.append(text)
return " ".join(parts)
def describe(action: dict, *, played: bool = True) -> str:
"""The tool-result string handed back to the server."""
lines = action.get("lines") or []
voices = sorted({str(line.get("voice") or "self") for line in lines})
note = relay_json.repair_note(action.get("_repairs") or [])
if not played:
return f"[dialogue] not played ({len(lines)} lines){note}"
return (
f"[dialogue] played {len(lines)} line{'s' if len(lines) != 1 else ''} "
f"in {len(voices)} voice{'s' if len(voices) != 1 else ''}: {', '.join(voices)}{note}"
# The scene was spoken out loud before this result got back to the
# server, and the model has no other way to know that. Without saying
# so it writes a final reply summarising what the user just heard, and
# the pet says the same thing twice in a row (observed 2026-07-31).
# An instruction delivered here, at the moment it applies, lands far
# better than a rule buried in a long system prompt.
"\nThe user HEARD this already. Do not repeat, summarise or narrate it "
"in your reply — answer with at most one short line, or nothing new."
)
bolt_pet/doctor.py
-323
View File
@@ -1,323 +0,0 @@
"""`python -m bolt_pet --doctor` — is this install actually going to work?
Written after a week of debugging things a preflight would have shown in one
command: a missing port publish, a wrong reverse-proxy header, a venv whose
python was a zero-byte file, an OCR engine that was never installed. Every one
of those presented as "the pet is being weird" and took a conversation to find.
Each check is independent and reports one of three things — ok, a warning
(works, but degraded), or a failure (this will not do what you expect) — and
says *what to do about it* rather than just what is wrong. Nothing here raises:
a doctor that crashes on a broken install is diagnosing the wrong patient.
Deliberately does not open the mic or call a paid API by default. It checks
that the door is unlocked, not that the room is furnished; `--deep` is there
when you want it to actually knock.
"""
from __future__ import annotations
import importlib
import shutil
import sys
from dataclasses import dataclass
from typing import Callable, Optional
from . import config
OK, WARN, FAIL = "ok", "warn", "fail"
_MARKS = {OK: " ok ", WARN: " warn ", FAIL: " FAIL "}
@dataclass
class Check:
name: str
status: str
detail: str = ""
fix: str = ""
def line(self) -> str:
text = f"[{_MARKS[self.status]}] {self.name:22} {self.detail}"
if self.fix and self.status != OK:
text += f"\n{'':32}→ {self.fix}"
return text
def _module(name: str) -> bool:
try:
importlib.import_module(name)
return True
except Exception:
return False
# ── the checks ──────────────────────────────────────────────────────────────
def check_config() -> Check:
missing = config.missing_config()
if missing:
return Check("server config", FAIL, f"missing {', '.join(missing)}",
"set them in .env — without these the controller exits at startup")
return Check("server config", OK, f"{config.SERVER_URL} as {config.SESSION_ID}")
def check_server(deep: bool = False) -> Check:
if not config.SERVER_URL:
return Check("server", FAIL, "no BOLT_SERVER_URL",
"cp .env.example .env and set BOLT_SERVER_URL to your Bolt server")
if not deep:
return Check("server", OK, f"{config.SERVER_URL} (not contacted; --deep to try)")
try:
from . import server_client
health = server_client.check_health(timeout=8)
return Check("server", OK, f"reachable — {health}")
except Exception as exc:
return Check("server", FAIL, f"unreachable: {exc}",
"check the URL, the key, and that the container is up")
def check_streaming_endpoint(deep: bool = False) -> Check:
"""The streamed-reply endpoint is newer than some deployed servers."""
if not config.STREAMING_REPLIES:
return Check("streamed replies", WARN, "disabled (STREAMING_REPLIES=false)")
if not deep:
return Check("streamed replies", OK, "enabled (endpoint not probed)")
try:
import requests
response = requests.post(
f"{config.SERVER_URL}/desk/converse_stream",
json={"session_id": config.SESSION_ID, "text": ""},
headers={"X-Desk-Api-Key": config.API_KEY}, timeout=8, stream=True,
)
if response.status_code == 404:
return Check("streamed replies", WARN, "server has no /desk/converse_stream",
"update the server, or set STREAMING_REPLIES=false to skip the probe")
return Check("streamed replies", OK, f"endpoint answered {response.status_code}")
except Exception as exc:
return Check("streamed replies", WARN, f"probe failed: {exc}")
def check_microphone(deep: bool = False) -> Check:
if not _module("sounddevice"):
return Check("microphone", FAIL, "sounddevice is not installed",
"pip install -r requirements.txt")
try:
import sounddevice as sd
devices = [d for d in sd.query_devices() if d.get("max_input_channels", 0) > 0]
if not devices:
return Check("microphone", FAIL, "no input devices",
"on Linux check PipeWire/PulseAudio is running as your user")
chosen = config.MIC_DEVICE or "system default"
if not deep:
return Check("microphone", OK, f"{len(devices)} input device(s), using {chosen}")
with sd.InputStream(samplerate=config.SAMPLE_RATE, channels=1, dtype="int16",
device=config.MIC_DEVICE, blocksize=config.FRAME_LEN):
pass
return Check("microphone", OK, f"opened at {config.SAMPLE_RATE} Hz ({chosen})")
except Exception as exc:
return Check("microphone", FAIL, f"could not open: {exc}",
"don't run the pet as root — PortAudio can't reach your PipeWire socket")
def check_wake_model() -> Check:
from pathlib import Path
path = Path(config.WAKE_MODEL_PATH)
if not path.exists():
return Check("wake word", FAIL, f"{path.name} is missing",
"it ships in the project root; check WAKE_MODEL_FILE")
if not _module("openwakeword"):
return Check("wake word", FAIL, "openwakeword is not installed",
"pip install -r requirements.txt")
return Check("wake word", OK, f"{path.name}, threshold {config.WAKE_WORD_THRESHOLD}")
def check_stt() -> Check:
"""STT is a websocket relay to the server (/desk/stt) — no local Deepgram
account, but websocket-client is now load-bearing for transcription to
work at all, not just the streaming optimisation (there's no separate
REST fallback any more)."""
if not config.is_configured():
return Check("speech-to-text", FAIL, "no BOLT_SERVER_URL/DESK_API_KEY",
"set them in .env — nothing you say can be transcribed without the server")
if not _module("websocket"):
return Check("speech-to-text", FAIL, "websocket-client is missing",
"pip install -r requirements.txt — /desk/stt is a websocket relay "
"with no REST fallback")
mode = "streaming" if config.STT_STREAMING else "one-shot (still via the server relay)"
return Check("speech-to-text", OK, f"server relay, {mode}")
def check_tts(deep: bool = False) -> Check:
"""TTS is the server's /desk/tts — no local ElevenLabs account needed for
the normal reply voice, just a voice id for it to request.
Shallow only checks config presence, same as the other shallow checks —
it can't see a live failure (bad voice id, an expired key, a Voicebox
clone owned by someone else). --deep actually calls /desk/tts with a
couple of words and reports the real HTTP status/error, which is the
only way to tell "misconfigured" apart from "configured but broken"
(e.g. speak() silently falling back to the offline voice)."""
if not (config.is_configured() and config.ELEVENLABS_VOICE_ID):
if _module("pyttsx3"):
return Check("text-to-speech", WARN, "server/voice not configured — offline voice only",
"set BOLT_SERVER_URL/DESK_API_KEY and ELEVENLABS_VOICE_ID for the real voice")
return Check("text-to-speech", FAIL, "no server/voice config and no pyttsx3 fallback",
"set BOLT_SERVER_URL/DESK_API_KEY/ELEVENLABS_VOICE_ID, "
"or pip install pyttsx3 for an offline voice")
if not deep:
return Check("text-to-speech", OK,
f"server relay, voice …{config.ELEVENLABS_VOICE_ID[-6:]} "
"(not probed; --deep to actually synthesize)")
try:
import requests
response = requests.post(
f"{config.SERVER_URL}/desk/tts",
headers={"X-Desk-Api-Key": config.API_KEY},
json={"session_id": config.SESSION_ID, "text": "testing",
"voice_id": config.ELEVENLABS_VOICE_ID},
timeout=20,
)
except Exception as exc:
return Check("text-to-speech", FAIL, f"probe failed: {exc}",
"check BOLT_SERVER_URL is reachable from this machine")
if response.status_code == 200 and response.content:
return Check("text-to-speech", OK,
f"server synthesized {len(response.content)} bytes "
f"for voice …{config.ELEVENLABS_VOICE_ID[-6:]}")
detail = ""
try:
detail = str(response.json().get("error") or "")
except Exception:
pass
return Check(
"text-to-speech", FAIL,
f"server returned {response.status_code}{': ' + detail if detail else ''} "
f"for voice …{config.ELEVENLABS_VOICE_ID[-6:]}",
"a 404 means this voice id isn't yours (or doesn't exist); 402 means "
"the account is out of credits; 401 means DESK_API_KEY is wrong",
)
def check_dialogue() -> Check:
if not config.DIALOGUE:
return Check("multi-voice scenes", WARN, "disabled (DIALOGUE=false)")
if not config.ELEVENLABS_API_KEY:
return Check("multi-voice scenes", WARN, "no ELEVENLABS_API_KEY",
"set it in .env — dialogue scenes are the one feature still calling "
"ElevenLabs directly, since the server has no equivalent endpoint")
from . import dialogue
cast = dialogue.parse_voice_map(config.DIALOGUE_VOICES)
if not cast:
return Check("multi-voice scenes", WARN, "no cast configured — only 'self' works",
'set DIALOGUE_VOICES=narrator:<id>,villain:<id>')
return Check("multi-voice scenes", OK, f"{len(cast)} voice(s): {', '.join(sorted(cast))}")
def check_screen_text() -> Check:
if not config.SCREEN_TEXT:
return Check("screen reading", WARN, "disabled (SCREEN_TEXT=false)")
if not _module("mss"):
return Check("screen reading", WARN, "mss is not installed — petctl read will decline",
"pip install mss (and note it cannot capture on Wayland)")
engine = "pytesseract" if _module("pytesseract") else (
"rapidocr" if _module("rapidocr_onnxruntime") else "")
if not engine:
return Check("screen reading", WARN, "no OCR engine",
"pip install pytesseract && apt install tesseract-ocr, "
"or pip install rapidocr-onnxruntime")
if engine == "pytesseract" and not shutil.which("tesseract"):
return Check("screen reading", WARN, "pytesseract is installed but tesseract is not",
"apt install tesseract-ocr")
return Check("screen reading", OK, f"mss + {engine}")
def check_hotkey() -> Check:
if not _module("pynput"):
return Check("push-to-talk", WARN, "pynput is not installed",
"the wake word still works; pip install pynput for the hotkey")
import os
if os.environ.get("WAYLAND_DISPLAY") and not os.environ.get("DISPLAY"):
return Check("push-to-talk", WARN, "Wayland session — global hotkeys usually blocked",
"use the wake word, or click the pet")
return Check("push-to-talk", OK, config.PUSH_TO_TALK_HOTKEY)
def check_sprites() -> Check:
from pathlib import Path
root = Path(__file__).resolve().parent / "assets" / "sprites"
if not root.exists():
return Check("sprites", WARN, "no art — the placeholder blob will be drawn",
"python scripts/generate_bolt_sprites.py")
counts = {d.name: len(list(d.glob("*.png"))) for d in sorted(root.iterdir()) if d.is_dir()}
empty = [name for name, count in counts.items() if count == 0]
if empty:
return Check("sprites", WARN, f"no frames for: {', '.join(empty)}",
"python scripts/generate_bolt_sprites.py")
return Check("sprites", OK, ", ".join(f"{n} {c}" for n, c in counts.items()))
def check_latency() -> Check:
"""The setting most likely to make it feel slow, and the least obvious."""
silence = config.SILENCE_END_SEC
if silence >= 1.5:
return Check("turn latency", WARN, f"VAD_SILENCE_END_SEC={silence:g}s of dead air per turn",
"0.8-1.0 feels markedly snappier; it is pure wait before anything starts")
return Check("turn latency", OK, f"silence timeout {silence:g}s, "
f"streaming {'on' if config.STREAMING_REPLIES else 'off'}")
CHECKS: tuple[tuple[str, Callable], ...] = (
("config", check_config),
("server", check_server),
("stream", check_streaming_endpoint),
("mic", check_microphone),
("wake", check_wake_model),
("stt", check_stt),
("tts", check_tts),
("dialogue", check_dialogue),
("screen", check_screen_text),
("hotkey", check_hotkey),
("sprites", check_sprites),
("latency", check_latency),
)
def run(deep: bool = False) -> list[Check]:
results = []
for _name, check in CHECKS:
try:
try:
results.append(check(deep))
except TypeError:
results.append(check())
except Exception as exc: # a broken check must not hide the others
results.append(Check(_name, FAIL, f"the check itself failed: {exc}"))
return results
def main(argv: Optional[list] = None) -> int:
argv = list(argv if argv is not None else sys.argv[1:])
deep = "--deep" in argv
print(f"Bolt pet preflight{' (deep: contacting the server and opening the mic)' if deep else ''}\n")
results = run(deep=deep)
for check in results:
print(check.line())
failures = [c for c in results if c.status == FAIL]
warnings = [c for c in results if c.status == WARN]
print()
if failures:
print(f"{len(failures)} problem(s) will stop this working. Fix those first.")
elif warnings:
print(f"Ready. {len(warnings)} thing(s) degraded but working.")
else:
print("Everything checks out.")
return 1 if failures else 0
bolt_pet/history.py
-62
View File
@@ -1,62 +0,0 @@
"""Rolling transcript of the conversation.
The speech bubble hides itself after a few seconds, which is fine for chat
but bad for anything you actually needed to read (a command's output, a
number, a URL). This keeps the last HISTORY_LIMIT turns so the tray's
History window — and click-to-copy on the bubble — have something to show.
Pure logic, no Qt: the UI half is ui/history_window.py.
"""
from __future__ import annotations
from collections import deque
from dataclasses import dataclass
from typing import Iterable, Optional
USER = "you"
PET = "bolt"
SYSTEM = "system"
@dataclass(frozen=True)
class Entry:
role: str
text: str
timestamp: Optional[float] = None # time.time(); None when not recorded
def formatted(self, clock=None) -> str:
label = {USER: "You", PET: "Bolt", SYSTEM: "—"}.get(self.role, self.role)
stamp = clock(self.timestamp) if (clock and self.timestamp) else None
return f"[{stamp}] {label}: {self.text}" if stamp else f"{label}: {self.text}"
class ConversationHistory:
def __init__(self, limit: int = 100):
self._entries: deque[Entry] = deque(maxlen=max(1, limit))
def add(self, role: str, text: str, timestamp: Optional[float] = None) -> Optional[Entry]:
text = (text or "").strip()
if not text:
return None
entry = Entry(role=role, text=text, timestamp=timestamp)
self._entries.append(entry)
return entry
def entries(self) -> list[Entry]:
return list(self._entries)
def last(self, role: Optional[str] = None) -> Optional[Entry]:
for entry in reversed(self._entries):
if role is None or entry.role == role:
return entry
return None
def clear(self) -> None:
self._entries.clear()
def as_text(self, clock=None, entries: Optional[Iterable[Entry]] = None) -> str:
return "\n".join(e.formatted(clock) for e in (entries if entries is not None else self._entries))
def __len__(self) -> int:
return len(self._entries)
bolt_pet/hotkey.py
-105
View File
@@ -1,105 +0,0 @@
"""Global push-to-talk hotkey.
The wake word is the primary trigger, but it misfires in a noisy room and
won't fire at all if you're on a call — so there's a keyboard fallback that
works even when the pet has no focus (it's a frameless Qt.Tool window with no
taskbar entry, so an ordinary QShortcut would never see the key).
Needs `pynput`, which needs an X11/Win32/macOS input hook: on most Wayland
sessions it can't grab global keys, and on macOS it needs Accessibility
permission. All of that is a soft failure — start() reports the reason and
the wake word / tray keep working.
"""
from __future__ import annotations
from typing import Callable, Optional
# Aliases for the names people actually type in a .env file.
_ALIASES = {
"control": "ctrl",
"ctl": "ctrl",
"option": "alt",
"opt": "alt",
"win": "cmd",
"windows": "cmd",
"super": "cmd",
"meta": "cmd",
"command": "cmd",
"return": "enter",
"escape": "esc",
"del": "delete",
"ins": "insert",
"pgup": "page_up",
"pgdn": "page_down",
}
class HotkeyError(Exception):
pass
def to_pynput_spec(spec: str) -> str:
""""ctrl+alt+space" -> "<ctrl>+<alt>+<space>" (pynput's GlobalHotKeys
syntax: named keys in angle brackets, literal characters bare)."""
tokens = [t.strip().lower() for t in (spec or "").split("+")]
tokens = [t for t in tokens if t]
if not tokens:
raise HotkeyError("empty hotkey")
parts = []
for token in tokens:
token = _ALIASES.get(token, token)
parts.append(token if len(token) == 1 else f"<{token}>")
return "+".join(parts)
class GlobalHotkey:
"""Fires *callback* whenever the hotkey is pressed, anywhere. Safe to
construct unconditionally — nothing happens until start(), and start()
reports failure instead of raising into the UI thread."""
def __init__(self, spec: str, callback: Callable[[], None]):
self.spec = spec
self._callback = callback
self._listener = None
@property
def running(self) -> bool:
return self._listener is not None
def start(self) -> Optional[str]:
"""None on success, otherwise a human-readable reason it's off."""
if not (self.spec or "").strip():
return None # explicitly disabled — not an error worth reporting
try:
pynput_spec = to_pynput_spec(self.spec)
except HotkeyError as exc:
return f"push-to-talk hotkey {self.spec!r} is invalid: {exc}"
try:
from pynput import keyboard
except Exception as exc: # ImportError, or a backend that won't load
return f"push-to-talk needs pynput ({exc}) — wake word still works"
try:
listener = keyboard.GlobalHotKeys({pynput_spec: self._safe_callback})
listener.daemon = True
listener.start()
except Exception as exc:
return f"push-to-talk unavailable on this session ({exc}) — wake word still works"
self._listener = listener
return None
def _safe_callback(self) -> None:
# This runs on pynput's listener thread; an exception there would
# silently kill the listener for the rest of the session.
try:
self._callback()
except Exception:
pass
def stop(self) -> None:
if self._listener is not None:
try:
self._listener.stop()
except Exception:
pass
self._listener = None
bolt_pet/monitors.py
-213
View File
@@ -1,213 +0,0 @@
"""Which screens exist, and which one the pet is standing on.
Deliberately free of Qt *and* of any subprocess probing: the monitor list is
published by the UI (`ui/pet_window.py` builds it from
`QGuiApplication.screens()`) and handed to the controller over a queued
signal, the same way every other UI↔controller message travels.
That indirection is the whole point. The pet has to agree with itself about
what "monitor 2" means — if the controller enumerated screens with `xrandr`
while the window jumped using Qt's screen list, the two orderings could
disagree and Bolt would announce one screen and land on another. Making Qt the
single source of truth removes that class of bug, and leaves everything here
pure enough to unit test without a display.
Indices are **1-based in every string a human or the model ever sees**, and
0-based in the list itself. `Monitor.index` is the 0-based one; `.number` is
what gets printed.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Iterable, Optional
# Directional specs understood by resolve(), mapped to a (dx, dy) heading.
_DIRECTIONS = {
"left": (-1, 0),
"right": (1, 0),
"up": (0, -1),
"above": (0, -1),
"down": (0, 1),
"below": (0, 1),
}
@dataclass(frozen=True)
class Monitor:
"""One screen, in the global desktop coordinate space."""
index: int # 0-based position in the published list
name: str
x: int
y: int
width: int
height: int
primary: bool = False
@property
def number(self) -> int:
"""1-based, for anything a person or the model reads."""
return self.index + 1
@property
def right(self) -> int:
return self.x + self.width
@property
def bottom(self) -> int:
return self.y + self.height
@property
def center(self) -> tuple[int, int]:
return self.x + self.width // 2, self.y + self.height // 2
def contains(self, x: int, y: int) -> bool:
return self.x <= x < self.right and self.y <= y < self.bottom
@property
def label(self) -> str:
bits = f"{self.number}: {self.name} {self.width}x{self.height}"
return bits + " (primary)" if self.primary else bits
def monitor_containing(
monitors: Iterable[Monitor], x: int, y: int
) -> Optional[Monitor]:
"""The screen holding point (x, y), or None if it's off every screen."""
for monitor in monitors:
if monitor.contains(x, y):
return monitor
return None
def nearest_monitor(monitors: Iterable[Monitor], x: int, y: int) -> Optional[Monitor]:
"""Screen whose centre is closest to (x, y) — the fallback when a point
lands in the dead space between mismatched screens."""
monitors = list(monitors)
if not monitors:
return None
return min(
monitors,
key=lambda m: (m.center[0] - x) ** 2 + (m.center[1] - y) ** 2,
)
def resolve(
monitors: list[Monitor], spec: str, current: Optional[int] = None
) -> Monitor:
"""Turn a `petctl jump` target into a screen.
Accepts a 1-based number, a name (case-insensitive substring, so "hdmi"
finds "HDMI-0"), `next`/`prev`, `primary`, `other`, or a direction
(`left`/`right`/`up`/`down`) relative to *current*. Raises ValueError with
a message meant to be read by the model, since it goes back as tool
output.
"""
if not monitors:
raise ValueError("no monitors have been reported yet")
spec = (spec or "").strip().lower()
if not spec:
raise ValueError("jump needs a target monitor")
count = len(monitors)
if current is None or not (0 <= current < count):
current = next((m.index for m in monitors if m.primary), 0)
if spec.isdigit():
number = int(spec)
if not (1 <= number <= count):
raise ValueError(
f"there is no monitor {number}; you have {count} "
f"(1-{count})"
)
return monitors[number - 1]
if spec in ("next", "forward"):
return monitors[(current + 1) % count]
if spec in ("prev", "previous", "back"):
return monitors[(current - 1) % count]
if spec == "primary":
return next((m for m in monitors if m.primary), monitors[0])
if spec == "other":
# With two screens "the other one" is unambiguous; with more it's just
# the next one round, which is at least always a *different* screen.
return monitors[(current + 1) % count]
if spec == "random":
# Deterministic-free choice is the caller's business; pick the screen
# furthest from the current one so "random" always visibly moves.
here = monitors[current].center
return max(
monitors,
key=lambda m: (m.center[0] - here[0]) ** 2 + (m.center[1] - here[1]) ** 2,
)
if spec in _DIRECTIONS:
dx, dy = _DIRECTIONS[spec]
here = monitors[current].center
candidates = []
for monitor in monitors:
if monitor.index == current:
continue
ox, oy = monitor.center
along = (ox - here[0]) * dx + (oy - here[1]) * dy
if along <= 0:
continue # not in that direction at all
drift = abs((ox - here[0]) * dy + (oy - here[1]) * dx)
candidates.append((drift, along, monitor))
if not candidates:
raise ValueError(
f"there's no monitor to the {spec} of monitor "
f"{monitors[current].number}"
)
# Prefer the best-aligned screen, then the closest of those.
candidates.sort(key=lambda item: (item[0], item[1]))
return candidates[0][2]
matches = [m for m in monitors if spec in m.name.lower()]
if len(matches) == 1:
return matches[0]
if len(matches) > 1:
raise ValueError(
f"{spec!r} matches several monitors: "
+ ", ".join(m.label for m in matches)
)
raise ValueError(
f"unknown monitor {spec!r}; you have: "
+ "; ".join(m.label for m in monitors)
+ " — or use next/prev/primary/left/right/up/down"
)
def describe(monitors: list[Monitor], current: Optional[int] = None) -> str:
"""Full listing, used as `petctl monitors` output."""
if not monitors:
return "[pet] no monitor information available"
lines = [f"[pet] {len(monitors)} monitor(s):"]
for monitor in monitors:
here = " <- Bolt is here" if monitor.index == current else ""
lines.append(
f" {monitor.label} at +{monitor.x}+{monitor.y}{here}"
)
return "\n".join(lines)
def summary(monitors: list[Monitor], current: Optional[int] = None) -> Optional[str]:
"""One-line version tacked onto each utterance — short on purpose, since
it rides along with every single thing you say."""
if not monitors:
return None
parts = ", ".join(f"{m.number}) {m.name} {m.width}x{m.height}" for m in monitors)
line = f"{len(monitors)} monitors: {parts}"
if current is not None and 0 <= current < len(monitors):
line += f"; Bolt is on {monitors[current].number}"
return line
def annotate(text: str, monitors: list[Monitor], current: Optional[int] = None) -> str:
"""Attach the screen summary as a separate aside, matching the style of
screen_context.annotate() so the model can ignore it when irrelevant."""
text = (text or "").strip()
line = summary(monitors, current)
if not text or not line:
return text
return f"{text}\n\n[{line}]"
bolt_pet/notifications.py
-166
View File
@@ -1,166 +0,0 @@
"""Desktop notification bridge (Linux/D-Bus).
Lets Bolt react to things that happen without you: a build finishing, a
calendar alert, a message arriving. Notifications are tailed from
`dbus-monitor`, filtered, rate-limited, and handed to the controller, which
forwards them through the normal converse() path — so the pet can say
"your deploy just went green" instead of only ever answering questions.
Off by default (NOTIFICATION_BRIDGE): every forwarded notification is a
round trip to the server, and an unfiltered desktop can be very chatty.
NOTIFICATION_FILTER (a regex) is the main knob for keeping it useful.
The dbus-monitor *parsing* is pure and unit tested; only the subprocess
plumbing needs a real session bus.
"""
from __future__ import annotations
import platform
import re
import shutil
import subprocess
import threading
from dataclasses import dataclass
from typing import Callable, Iterable, Iterator, Optional
_STRING_LINE = re.compile(r'^\s*string\s+"(.*)"\s*$')
_BLOCK_START = re.compile(r"^(method call|signal|method return|error)\b")
@dataclass(frozen=True)
class Notification:
app: str
summary: str
body: str
def as_text(self) -> str:
parts = [p for p in (self.summary, self.body) if p]
joined = " — ".join(parts)
return f"{self.app}: {joined}" if self.app else joined
def iter_notifications(lines: Iterable[str]) -> Iterator[Notification]:
"""Pull Notification records out of a `dbus-monitor` line stream.
A Notify call prints its arguments one per line after the header; the
string arguments arrive in the order app_name, app_icon, summary, body
(replaces_id is a uint32, so it isn't in the string list). Anything
that doesn't look like that is skipped rather than guessed at.
"""
collecting = False
strings: list[str] = []
def _emit() -> Optional[Notification]:
if len(strings) < 3:
return None
return Notification(app=strings[0].strip(), summary=strings[2].strip(),
body=(strings[3].strip() if len(strings) > 3 else ""))
for line in lines:
if _BLOCK_START.match(line):
if collecting:
notification = _emit()
if notification is not None:
yield notification
collecting = "member=Notify" in line
strings = []
continue
if not collecting:
continue
match = _STRING_LINE.match(line)
if match:
strings.append(match.group(1))
if collecting:
notification = _emit()
if notification is not None:
yield notification
class NotificationGate:
"""Filter + rate limit. Clock is passed in (monotonic seconds) so the
rate limiting is testable without sleeping."""
def __init__(self, pattern: str = "", min_interval: float = 60.0):
self._min_interval = max(0.0, min_interval)
self._last_forwarded = None
self._pattern = None
if (pattern or "").strip():
try:
self._pattern = re.compile(pattern, re.IGNORECASE)
except re.error:
self._pattern = None # a broken regex shouldn't mute everything
def matches(self, notification: Notification) -> bool:
if self._pattern is None:
return True
return bool(self._pattern.search(notification.as_text()))
def should_forward(self, notification: Notification, now: float) -> bool:
if not notification.as_text().strip():
return False
if not self.matches(notification):
return False
if self._last_forwarded is not None and now - self._last_forwarded < self._min_interval:
return False
self._last_forwarded = now
return True
def available() -> bool:
return platform.system() == "Linux" and shutil.which("dbus-monitor") is not None
class NotificationWatcher:
"""Tails dbus-monitor on a daemon thread, calling *callback* per
notification. Best-effort: if the session bus isn't reachable it reports
why via start() and stays off."""
_ARGS = [
"dbus-monitor", "--session",
"interface='org.freedesktop.Notifications',member='Notify'",
]
def __init__(self, callback: Callable[[Notification], None]):
self._callback = callback
self._process: Optional[subprocess.Popen] = None
self._thread: Optional[threading.Thread] = None
self._running = False
def start(self) -> Optional[str]:
"""None on success, otherwise the reason the bridge is off."""
if not available():
return "notification bridge needs Linux + dbus-monitor — skipping"
try:
self._process = subprocess.Popen(
self._ARGS, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL,
text=True, bufsize=1,
)
except Exception as exc:
return f"couldn't start dbus-monitor ({exc}) — notification bridge off"
self._running = True
self._thread = threading.Thread(target=self._pump, name="notification-bridge", daemon=True)
self._thread.start()
return None
def _pump(self) -> None:
assert self._process is not None and self._process.stdout is not None
try:
for notification in iter_notifications(self._process.stdout):
if not self._running:
return
try:
self._callback(notification)
except Exception:
pass # one bad notification shouldn't end the bridge
except Exception:
pass
def stop(self) -> None:
self._running = False
if self._process is not None:
try:
self._process.terminate()
except Exception:
pass
self._process = None
bolt_pet/pet_actions.py
-191
View File
@@ -1,191 +0,0 @@
"""Commands that drive the pet's *body* instead of the shell.
The server relays shell commands to this machine (see server_client.
run_local_command). Rather than inventing a new payload type the desk API
doesn't speak — this client can't change the server — a small `petctl`
pseudo-command is intercepted before it ever reaches `subprocess`: if Bolt
emits `petctl move top-left` or `petctl emote wave`, the pet does it and
returns a normal-looking command output string, so from the server's side
it's just another tool call that worked.
Pure parsing logic — no Qt, no subprocess — so it's cheap to unit test. The
UI half lives in ui/pet_window.py (apply_action).
"""
from __future__ import annotations
import shlex
from typing import Optional
# What Bolt is allowed to type. Anything else falls through to a real shell.
_PREFIXES = ("petctl", "bolt-pet", "pet")
ANCHORS = (
"top-left", "top", "top-right",
"left", "center", "right",
"bottom-left", "bottom", "bottom-right",
"cursor", "random",
)
EMOTES = ("wave", "hop", "spin", "nod", "shake", "bounce", "wiggle")
# Where `petctl jump` can be aimed. A bare number (1-based) works too, as does
# any unique part of a monitor's name — resolution lives in monitors.resolve().
MONITOR_SPECS = (
"next", "prev", "primary", "other", "random",
"left", "right", "up", "down",
)
HELP = (
"petctl move <x> <y> | <" + "|".join(ANCHORS) + ">\n"
"petctl jump <monitor number|" + "|".join(MONITOR_SPECS) + "|name>\n"
"petctl monitors\n"
"petctl read [monitor number|here|all]\n"
"petctl emote <" + "|".join(EMOTES) + ">\n"
"petctl say <text>\n"
"petctl wander on|off\n"
"petctl nap on|off\n"
"petctl voice reset\n"
"petctl self_restart [why]"
)
# `petctl voice` only ever goes one way: back to the configured voice. Picking
# a *different* one is the server's job (its speak_as reply marker), and it
# already knows how — what it has no way to say is "never mind, be yourself
# again", because it was never told which voice that is.
VOICE_RESETS = ("reset", "default", "normal", "own", "back", "mine", "yours")
class ActionError(Exception):
"""Bad petctl syntax — reported back to the server as command output."""
def is_pet_command(command: str) -> bool:
parts = (command or "").strip().split()
return bool(parts) and parts[0].lower() in _PREFIXES
def _bool_arg(value: str) -> bool:
value = value.lower()
if value in ("on", "true", "yes", "1", "start", "enable"):
return True
if value in ("off", "false", "no", "0", "stop", "disable"):
return False
raise ActionError(f"expected on/off, got {value!r}")
def parse(command: str) -> Optional[dict]:
"""Parse a `petctl ...` string into an action dict, or None if this isn't
a pet command at all (caller should run it as a real shell command).
Raises ActionError on a pet command that doesn't make sense."""
if not is_pet_command(command):
return None
try:
parts = shlex.split(command.strip())
except ValueError as exc: # unbalanced quotes
raise ActionError(f"couldn't parse arguments: {exc}") from exc
verb = (parts[1].lower() if len(parts) > 1 else "help")
args = parts[2:]
if verb in ("help", "-h", "--help"):
return {"action": "help"}
if verb in ("move", "goto", "walk"):
if not args:
raise ActionError("move needs a target: " + ", ".join(ANCHORS) + ", or x y")
if len(args) >= 2 and _looks_numeric(args[0]) and _looks_numeric(args[1]):
return {"action": "move", "x": int(float(args[0])), "y": int(float(args[1]))}
anchor = args[0].lower().replace("_", "-")
if anchor not in ANCHORS:
raise ActionError(f"unknown position {args[0]!r}; try one of: " + ", ".join(ANCHORS))
return {"action": "move", "anchor": anchor}
if verb in ("jump", "monitor", "screen"):
if not args:
raise ActionError(
"jump needs a monitor: a number, a name, or one of "
+ ", ".join(MONITOR_SPECS)
)
# The spec isn't validated here on purpose: which monitors exist is a
# runtime fact this pure module doesn't have. monitors.resolve() does
# it once the published screen list is in hand.
return {"action": "jump", "target": " ".join(args).strip()}
if verb in ("monitors", "screens", "displays"):
return {"action": "monitors"}
if verb in ("read", "look", "ocr", "see"):
target = (" ".join(args).strip() or "here").lower()
return {"action": "read", "target": target}
if verb in ("emote", "do"):
if not args:
raise ActionError("emote needs a name: " + ", ".join(EMOTES))
emote = args[0].lower()
if emote not in EMOTES:
raise ActionError(f"unknown emote {args[0]!r}; try one of: " + ", ".join(EMOTES))
return {"action": "emote", "emote": emote}
if verb == "say":
text = " ".join(args).strip()
if not text:
raise ActionError("say needs something to say")
return {"action": "say", "text": text}
if verb == "wander":
if not args:
raise ActionError("wander needs on or off")
return {"action": "wander", "enabled": _bool_arg(args[0])}
if verb == "voice":
target = (args[0].lower() if args else "reset").lstrip("-")
if target not in VOICE_RESETS:
raise ActionError(
f"can't set a voice from petctl (got {args[0]!r}); "
"use the speak_as reply marker to pick one. "
"petctl voice reset goes back to the default voice."
)
return {"action": "voice", "voice": "default"}
if verb in ("self_restart", "restart", "reboot"):
# Free text, not a fixed grammar: the argument is a note to the pet's
# *next* process about why it died and what to look at when it comes
# back, so anything the model wants to tell future-itself is valid.
return {"action": "self_restart", "reason": " ".join(args).strip()}
if verb in ("nap", "sleep", "dnd"):
if not args:
raise ActionError("nap needs on or off")
return {"action": "nap", "enabled": _bool_arg(args[0])}
raise ActionError(f"unknown petctl verb {verb!r}\n{HELP}")
def _looks_numeric(value: str) -> bool:
try:
float(value)
return True
except ValueError:
return False
def describe(action: dict) -> str:
"""The text handed back to the server as this "command"'s output. Phrased
as a completed fact so the model doesn't narrate the mechanics of it."""
kind = action.get("action")
if kind == "move":
where = action.get("anchor") or f"({action.get('x')}, {action.get('y')})"
return f"[pet] walking to {where}"
if kind == "jump":
return f"[pet] jumping to monitor {action['target']}"
if kind == "emote":
return f"[pet] {action['emote']}"
if kind == "say":
return "[pet] showing that in the speech bubble"
if kind == "wander":
return "[pet] wandering " + ("enabled" if action["enabled"] else "disabled")
if kind == "nap":
return "[pet] " + ("napping" if action["enabled"] else "awake")
if kind == "help":
return HELP
return "[pet] ok"
bolt_pet/quiet.py
-78
View File
@@ -1,78 +0,0 @@
"""Quiet hours — when the pet is allowed to make noise on its own.
Napping only ever suppresses *proactive* noise (heartbeat announcements,
forwarded notifications) and wandering. The wake word, click-to-talk and
push-to-talk still work: telling it to be quiet shouldn't mean it stops
answering when spoken to.
Pure logic (parsing + a time comparison) so it's testable without waiting
for 11pm.
"""
from __future__ import annotations
from datetime import time as dtime
from typing import Iterable, Optional
class QuietHoursError(ValueError):
pass
def _parse_clock(value: str) -> dtime:
parts = value.strip().split(":")
if len(parts) != 2:
raise QuietHoursError(f"expected HH:MM, got {value!r}")
try:
hour, minute = int(parts[0]), int(parts[1])
except ValueError as exc:
raise QuietHoursError(f"expected HH:MM, got {value!r}") from exc
if not (0 <= hour <= 23 and 0 <= minute <= 59):
raise QuietHoursError(f"{value!r} is not a real time of day")
return dtime(hour, minute)
def parse_ranges(spec: str) -> list[tuple[dtime, dtime]]:
""""23:00-08:00, 13:00-14:00" -> [(23:00, 08:00), (13:00, 14:00)].
Empty/blank spec means "no quiet hours"."""
ranges: list[tuple[dtime, dtime]] = []
for chunk in (spec or "").split(","):
chunk = chunk.strip()
if not chunk:
continue
start, sep, end = chunk.partition("-")
if not sep:
raise QuietHoursError(f"expected HH:MM-HH:MM, got {chunk!r}")
ranges.append((_parse_clock(start), _parse_clock(end)))
return ranges
def in_ranges(now: dtime, ranges: Iterable[tuple[dtime, dtime]]) -> bool:
for start, end in ranges:
if start == end:
continue # a zero-length range is a typo, not "all day"
if start < end:
if start <= now < end:
return True
elif now >= start or now < end: # wraps past midnight
return True
return False
def is_quiet(spec: str, now: Optional[dtime] = None, on_error=None) -> bool:
"""True if *now* (defaults to the local wall clock) falls inside *spec*.
A malformed spec is reported via *on_error* and treated as "not quiet" —
a config typo shouldn't silently mute the pet forever."""
if not (spec or "").strip():
return False
try:
ranges = parse_ranges(spec)
except QuietHoursError as exc:
if on_error is not None:
on_error(exc)
return False
if now is None:
from datetime import datetime
now = datetime.now().time()
return in_ranges(now, ranges)
bolt_pet/screen_context.py
-184
View File
@@ -1,184 +0,0 @@
"""What's on screen right now — the active window's title, and whether
something is running fullscreen.
Two consumers:
* annotate() tacks the focused window's title onto what you said, so
"what's this error?" has a referent without you describing it (the desk
API takes text only, so this is a text annotation — no screenshot upload).
* is_fullscreen_active() feeds do-not-disturb: the pet shouldn't announce
anything over a call or a fullscreen game.
Everything here is best-effort and must never raise: on a locked-down Wayland
session none of it is available, and the correct behaviour is simply "no
context", not a crashed pipeline. The subprocess *parsing* is split into pure
functions so it can be tested without a display server.
"""
from __future__ import annotations
import platform
import re
import shutil
import subprocess
from typing import Optional
_TIMEOUT = 2.0
_MAX_TITLE_CHARS = 160
# Titles that are just the desktop itself — annotating with these is noise.
_BORING_TITLES = {"", "desktop", "@!0,0;bdib", "plasmashell", "gnome-shell", "xfdesktop"}
def _run(args: list[str]) -> Optional[str]:
try:
completed = subprocess.run(args, capture_output=True, text=True, timeout=_TIMEOUT)
except Exception:
return None
if completed.returncode != 0:
return None
return completed.stdout
# ── pure parsing helpers (unit tested; no display server needed) ─────────────
def parse_xprop_window_id(output: str) -> Optional[str]:
"""`xprop -root _NET_ACTIVE_WINDOW` -> '_NET_ACTIVE_WINDOW(WINDOW): window id # 0x3c00007'"""
match = re.search(r"(0x[0-9a-fA-F]+)", output or "")
if not match or int(match.group(1), 16) == 0:
return None
return match.group(1)
def parse_xprop_window_name(output: str) -> Optional[str]:
"""`xprop -id <id> _NET_WM_NAME` -> '_NET_WM_NAME(UTF8_STRING) = "Firefox"'"""
match = re.search(r'=\s*"(.*)"\s*$', (output or "").strip(), re.DOTALL)
if not match:
return None
return match.group(1).strip()
def parse_xprop_fullscreen(output: str) -> bool:
return "_NET_WM_STATE_FULLSCREEN" in (output or "")
def clean_title(title: Optional[str]) -> Optional[str]:
title = (title or "").strip().replace("\n", " ")
if title.lower() in _BORING_TITLES:
return None
if len(title) > _MAX_TITLE_CHARS:
title = title[: _MAX_TITLE_CHARS - 1].rstrip() + "…"
return title or None
def annotate(text: str, title: Optional[str]) -> str:
"""Attach the window title as an explicit aside rather than splicing it
into the sentence, so the model can ignore it when it's irrelevant."""
text = (text or "").strip()
title = clean_title(title)
if not text or not title:
return text
return f"{text}\n\n[on screen right now: {title}]"
# ── platform probes ──────────────────────────────────────────────────────────
def _linux_active_window_id() -> Optional[str]:
if not shutil.which("xprop"):
return None
output = _run(["xprop", "-root", "_NET_ACTIVE_WINDOW"])
return parse_xprop_window_id(output or "")
def _linux_title() -> Optional[str]:
if shutil.which("xdotool"):
output = _run(["xdotool", "getactivewindow", "getwindowname"])
if output and output.strip():
return output.strip()
window_id = _linux_active_window_id()
if window_id is None:
return None
for prop in ("_NET_WM_NAME", "WM_NAME"):
output = _run(["xprop", "-id", window_id, prop])
title = parse_xprop_window_name(output or "")
if title:
return title
return None
def _windows_title() -> Optional[str]:
try:
import ctypes
user32 = ctypes.windll.user32
handle = user32.GetForegroundWindow()
if not handle:
return None
length = user32.GetWindowTextLengthW(handle)
buffer = ctypes.create_unicode_buffer(length + 1)
user32.GetWindowTextW(handle, buffer, length + 1)
return buffer.value or None
except Exception:
return None
def _macos_title() -> Optional[str]:
script = (
'tell application "System Events" to get name of first application process '
"whose frontmost is true"
)
output = _run(["osascript", "-e", script])
return (output or "").strip() or None
def active_window_title() -> Optional[str]:
"""Focused window's title, or None if the platform won't tell us."""
try:
system = platform.system()
if system == "Linux":
return clean_title(_linux_title())
if system == "Windows":
return clean_title(_windows_title())
if system == "Darwin":
return clean_title(_macos_title())
except Exception:
pass
return None
def is_fullscreen_active() -> bool:
"""True when the focused window is fullscreen (call, game, presentation).
False whenever we can't tell — do-not-disturb should be something you opt
into, not something a failed probe turns on."""
try:
system = platform.system()
if system == "Linux":
window_id = _linux_active_window_id()
if window_id is None:
return False
return parse_xprop_fullscreen(_run(["xprop", "-id", window_id, "_NET_WM_STATE"]) or "")
if system == "Windows":
import ctypes
from ctypes import wintypes
user32 = ctypes.windll.user32
handle = user32.GetForegroundWindow()
if not handle:
return False
rect = wintypes.RECT()
user32.GetWindowRect(handle, ctypes.byref(rect))
screen_w = user32.GetSystemMetrics(0)
screen_h = user32.GetSystemMetrics(1)
return (rect.right - rect.left) >= screen_w and (rect.bottom - rect.top) >= screen_h
except Exception:
pass
return False
def context_for(text: str) -> str:
"""What the controller sends: *text* plus the active window title, when
SCREEN_CONTEXT is on and there's a title worth mentioning."""
from . import config
if not config.SCREEN_CONTEXT:
return text
return annotate(text, active_window_title())
bolt_pet/screen_text.py
-187
View File
@@ -1,187 +0,0 @@
"""Reading the text that's actually on a monitor, via screenshot + OCR.
This is the "Bolt can see what's on screen" half of the screen features. It is
**pull, not push**: nothing here runs on its own. The server has to ask, by
relaying `petctl read`, and the recognised text goes back as that command's
output through the existing tool-result relay (see server_client.converse).
That's deliberate on two counts — OCR of a 4K screen costs a second or two,
which would be tacked onto every single utterance if it ran automatically, and
"screen contents leave this machine" should be a thing Bolt decides to do and
you can see in the log, not a silent constant.
Both halves are optional and soft-fail with a reason, the way hotkey.py does:
capture needs `mss`, recognition needs a Tesseract or RapidOCR install. With
neither, `petctl read` reports what's missing instead of raising, and the rest
of the pet carries on.
The pure parts (cleaning OCR output, formatting the reply, deciding which
engine to use given what's installed) are split out and unit tested; only
capture and the OCR call itself need a real screen.
"""
from __future__ import annotations
import re
import shutil
from typing import Callable, Optional
from .monitors import Monitor
DEFAULT_MAX_CHARS = 4000
INSTALL_HINT = (
"install one of: `pip install mss pytesseract` + `sudo apt install "
"tesseract-ocr` (fastest), or `pip install mss rapidocr-onnxruntime` "
"(no system package needed)"
)
# Lines that are almost certainly OCR noise rather than text: window chrome
# fragments, isolated punctuation, single stray characters.
_MIN_MEANINGFUL = 2
def _module_available(name: str) -> bool:
import importlib.util
try:
return importlib.util.find_spec(name) is not None
except (ImportError, ValueError):
return False
# ── pure helpers (unit tested; no screen, no OCR engine needed) ──────────────
def resolve_engine(
has_module: Callable[[str], bool] = _module_available,
which: Callable[[str], Optional[str]] = shutil.which,
) -> tuple[Optional[str], str]:
"""Pick an OCR engine from what's installed.
Returns `(engine, reason)`. *engine* is None when nothing usable is
present, and *reason* then explains what to install. Probes are injected
so this is testable on a machine with a different set of things installed.
"""
if has_module("pytesseract") and which("tesseract"):
return "pytesseract", ""
if has_module("rapidocr_onnxruntime"):
return "rapidocr", ""
if has_module("pytesseract") and not which("tesseract"):
return None, (
"pytesseract is installed but the tesseract binary isn't on PATH "
"(try: sudo apt install tesseract-ocr)"
)
return None, f"no OCR engine available — {INSTALL_HINT}"
def capture_available(has_module: Callable[[str], bool] = _module_available) -> bool:
return has_module("mss")
def clean_ocr_text(raw: str, max_chars: int = DEFAULT_MAX_CHARS) -> str:
"""Squeeze raw OCR output into something worth sending.
Screen OCR produces a lot of junk — single stray glyphs off window
borders, runs of blank lines, the same toolbar label recognised twice. All
of that costs tokens and tells the model nothing, so it goes.
"""
if not raw:
return ""
lines: list[str] = []
for line in raw.splitlines():
line = re.sub(r"[^\S\n]+", " ", line).strip()
if not line:
continue
if len(re.sub(r"[^0-9A-Za-z]", "", line)) < _MIN_MEANINGFUL:
continue
if lines and line == lines[-1]:
continue # consecutive duplicate
lines.append(line)
text = "\n".join(lines)
if max_chars and len(text) > max_chars:
text = text[: max_chars - 1].rstrip() + "…"
text += "\n[truncated]"
return text
def format_reading(monitor: Optional[Monitor], text: str) -> str:
"""The tool output handed back for `petctl read`."""
where = f"monitor {monitor.number} ({monitor.name})" if monitor else "screen"
if not text.strip():
return f"[pet] read {where}: no text recognised"
return f"[pet] text on {where}:\n{text}"
# ── capture + recognition (needs a real screen) ──────────────────────────────
def capture(monitor: Monitor):
"""Grab *monitor* as a PIL image, or None if capture isn't available."""
try:
import mss
from PIL import Image
except ImportError:
return None
try:
box = {
"left": monitor.x,
"top": monitor.y,
"width": monitor.width,
"height": monitor.height,
}
with mss.mss() as sct:
shot = sct.grab(box)
return Image.frombytes("RGB", shot.size, shot.bgra, "raw", "BGRX")
except Exception:
return None
def _ocr(image, engine: str) -> str:
if engine == "pytesseract":
import pytesseract
# Grayscale first: tesseract is measurably better on it than on the
# colour desktop, and it's a cheap conversion.
return pytesseract.image_to_string(image.convert("L"))
if engine == "rapidocr":
import numpy as np
from rapidocr_onnxruntime import RapidOCR
result, _ = RapidOCR()(np.array(image))
if not result:
return ""
return "\n".join(line[1] for line in result)
return ""
def read_monitor(monitor: Monitor, max_chars: int = DEFAULT_MAX_CHARS) -> str:
"""OCR one screen and return the formatted tool output.
Never raises: every failure path returns a sentence explaining itself,
because the return value goes straight back to the server as the result of
a command Bolt chose to run.
"""
if not capture_available():
return f"[pet] can't capture the screen — {INSTALL_HINT}"
engine, reason = resolve_engine()
if engine is None:
return f"[pet] can't read the screen — {reason}"
image = capture(monitor)
if image is None:
return (
f"[pet] couldn't capture monitor {monitor.number} "
"(is this a Wayland session? mss needs X11)"
)
try:
raw = _ocr(image, engine)
except Exception as exc:
return f"[pet] OCR failed on monitor {monitor.number}: {exc}"
return format_reading(monitor, clean_ocr_text(raw, max_chars))
def read_monitors(monitors: list[Monitor], max_chars: int = DEFAULT_MAX_CHARS) -> str:
"""OCR several screens, splitting the character budget between them."""
if not monitors:
return "[pet] no monitor information available"
if len(monitors) == 1:
return read_monitor(monitors[0], max_chars)
share = max(400, max_chars // len(monitors))
return "\n\n".join(read_monitor(m, share) for m in monitors)
bolt_pet/self_restart.py
-235
View File
@@ -1,235 +0,0 @@
"""`petctl self_restart` — the pet restarting itself, and remembering why.
Bolt can already edit this repo through `filectl` and run commands through the
shell relay, which means he can change the pet's own code. What he could not
do is *see the result*: the running process keeps the old modules in memory,
so an edit is invisible until somebody restarts the pet by hand, and by then
the conversation that motivated it is over. That makes the edit-test-review
loop a human errand.
This closes the loop. The tricky part is that the thing being asked to report
back is the thing that dies, so the mechanism is built around three problems:
1. **The turn must survive.** A restart mid-turn would kill the HTTP tool
relay before the result was posted, and the server would sit waiting until
it timed out — the conversation lost, with no explanation. So the command
only *arms* the restart: it returns immediately, the turn finishes and Bolt
speaks his reply, and the restart happens after (see
`controller._maybe_self_restart`), exactly like the updater's "only between
turns" rule.
2. **A broken edit must not be fatal.** Before anything is armed, the new code
is imported in a *subprocess* (`preflight`) — this process still holds the
old modules, so importing here would prove nothing. A syntax error comes
back as the command's output, in the same turn, and nothing restarts. That
is the difference between "Bolt broke the pet and lost his own way to fix
it" and "Bolt got a traceback and tried again".
3. **The reason must outlive the process.** The context (why, what to check,
which version, when) is written to disk before exec and read on the way
back up, so the new process can open with "I'm back — you asked me to check
X" instead of amnesia. That report goes to the server as a normal turn, so
Bolt sees the result of his own change and can carry on.
A loop guard bounds the worst case: `MAX_RESTARTS` inside `WINDOW_SECONDS`
and further self-restarts are refused with a reason, so an edit-restart-crash
cycle stops on its own rather than spinning the process forever.
Pure-ish and injectable throughout (paths, clock, subprocess runner) so the
whole thing is testable without ever restarting anything.
"""
from __future__ import annotations
import json
import os
import subprocess
import sys
import tempfile
import time
from dataclasses import asdict, dataclass, field
from pathlib import Path
from typing import Any, Callable, Optional
from . import config
# Lives in the cache dir, not the repo: it is transient state about *this*
# machine's process, and it must never end up in a git diff of the checkout
# Bolt is editing.
DEFAULT_STATE_PATH = Path.home() / ".cache" / "bolt-pet" / "restart_context.json"
# Loop guard. Deliberately small: a healthy edit-check cycle is one restart
# per change, and anything hammering past this is a crash loop, not work.
MAX_RESTARTS = int(os.environ.get("SELF_RESTART_MAX", "5"))
WINDOW_SECONDS = float(os.environ.get("SELF_RESTART_WINDOW_SECONDS", "900"))
# What the preflight subprocess imports. `ui.app` pulls in the widest slice of
# the package (Qt, controller, audio, every helper), so if this imports, a
# restart will at least reach the event loop.
_PREFLIGHT_IMPORT = "import bolt_pet, bolt_pet.controller, bolt_pet.ui.app"
class RestartError(Exception):
"""A refused restart — reported back to the server as command output."""
@dataclass
class RestartContext:
"""What the dying process wants the next one to know."""
reason: str = ""
verify: str = ""
armed_at: float = 0.0
version: str = ""
session: str = ""
recent: list = field(default_factory=list)
restarts: list = field(default_factory=list) # timestamps, for the loop guard
def as_dict(self) -> dict[str, Any]:
return asdict(self)
def _now() -> float:
return time.time()
def load(path: Optional[Path] = None) -> Optional[RestartContext]:
"""Read the context left by a previous process, or None."""
target = Path(path or DEFAULT_STATE_PATH)
try:
data = json.loads(target.read_text(encoding="utf-8"))
except (FileNotFoundError, json.JSONDecodeError, OSError):
return None
if not isinstance(data, dict):
return None
known = {field_name for field_name in RestartContext().as_dict()}
return RestartContext(**{k: v for k, v in data.items() if k in known})
def save(context: RestartContext, path: Optional[Path] = None) -> None:
"""Persist the context atomically — a half-written file on the way out
would make the next process start confused instead of oriented."""
target = Path(path or DEFAULT_STATE_PATH)
target.parent.mkdir(parents=True, exist_ok=True)
descriptor, temp_path = tempfile.mkstemp(dir=target.parent, prefix=".restart_", suffix=".tmp")
try:
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump(context.as_dict(), handle, ensure_ascii=False, indent=1)
os.replace(temp_path, target)
except BaseException:
try:
os.unlink(temp_path)
except OSError:
pass
raise
def clear(path: Optional[Path] = None) -> None:
"""Consume the context. Called once it has been reported, so the pet
doesn't announce the same restart every time it starts."""
try:
Path(path or DEFAULT_STATE_PATH).unlink()
except (FileNotFoundError, OSError):
pass
def recent_restarts(context: Optional[RestartContext], *, now: Optional[float] = None) -> list:
current = now if now is not None else _now()
stamps = list((context.restarts if context else []) or [])
return [stamp for stamp in stamps if current - float(stamp) <= WINDOW_SECONDS]
def check_loop_guard(context: Optional[RestartContext], *, now: Optional[float] = None) -> None:
"""Refuse to restart if we've already done it too many times recently."""
stamps = recent_restarts(context, now=now)
if len(stamps) >= MAX_RESTARTS:
raise RestartError(
f"refusing: {len(stamps)} self-restarts in the last "
f"{int(WINDOW_SECONDS / 60)} minutes. Something is looping — fix the "
"cause, or wait for the window to clear before trying again."
)
def preflight(
repo: Optional[Path] = None,
run: Optional[Callable[..., Any]] = None,
timeout: float = 120.0,
) -> None:
"""Import the current source in a subprocess; raise if it's broken.
This process has the *old* modules loaded, so importing in-process would
happily succeed on a file that no longer parses. Mirrors
`updater._smoke_test`, and exists for the same reason: never hand the
session to code that can't start."""
runner = run or subprocess.run
root = Path(repo or config.HERE)
try:
completed = runner(
[sys.executable, "-c", _PREFLIGHT_IMPORT],
cwd=str(root), capture_output=True, text=True, timeout=timeout,
env={**os.environ, "QT_QPA_PLATFORM": "offscreen"}, # no display needed to import
)
except Exception as exc: # subprocess itself failed to run
raise RestartError(f"couldn't run the preflight import check: {exc}") from exc
if completed.returncode != 0:
detail = (completed.stderr or completed.stdout or "").strip()
raise RestartError(
"the current code does not import, so restarting would leave you with "
f"nothing running. Fix this first:\n{detail[-800:]}"
)
def arm(
reason: str,
*,
verify: str = "",
version: str = "",
session: str = "",
recent: Optional[list] = None,
path: Optional[Path] = None,
now: Optional[float] = None,
) -> RestartContext:
"""Record why we're about to die, carrying the restart history forward."""
current = now if now is not None else _now()
previous = load(path)
context = RestartContext(
reason=" ".join(str(reason or "").split())[:400],
verify=" ".join(str(verify or "").split())[:400],
armed_at=current,
version=str(version or ""),
session=str(session or ""),
recent=list(recent or [])[-6:],
restarts=recent_restarts(previous, now=current) + [current],
)
save(context, path)
return context
def report(
context: RestartContext,
*,
version: str = "",
now: Optional[float] = None,
) -> str:
"""The message the new process sends the server on the way up.
Phrased as Bolt reporting to himself, because that is what it is: the
server sees it as an ordinary turn, and the reply comes back through the
normal pipeline — which is what lets "restart and check X" finish as a
sentence spoken out loud."""
current = now if now is not None else _now()
took = max(0.0, current - float(context.armed_at or current))
lines = [
"[pet self-restart] I restarted myself and I'm back up.",
f"- reason: {context.reason or 'not recorded'}",
f"- took: {took:.1f}s",
f"- version now running: {version or 'unknown'}"
+ (f" (was {context.version})" if context.version and context.version != version else ""),
]
if context.verify:
lines.append(f"- you wanted to check: {context.verify}")
if context.recent:
lines.append("- what we were doing before: " + " | ".join(str(x)[:120] for x in context.recent))
lines.append(
"The new code is loaded and running. If you wanted to verify something, "
"check it now (filectl to read, command to test) and tell the user what you found."
)
return "\n".join(lines)
bolt_pet/server_client.py
-307
View File
@@ -1,307 +0,0 @@
"""HTTP client for Bolt's desk API (ai/desk_api.py on the server).
Protocol is identical to desk_client/bolt_desk.py in the main tmn-api repo —
this pet is just another desk client, so it gets the exact same brain,
memory, tools, and persona as Discord chat and the Linux voice client:
text -> POST /desk/converse
[server may relay a shell command back to run on THIS machine]
... -> POST /desk/tool_result (repeat until the server sends a reply)
reply <- returned to caller
A reply can also carry a voice (`voice_id`/`voice_name`), which is how the
server's `speak_as` marker reaches us: Bolt searched the ElevenLabs voice
library, picked one, and tagged the reply with it — the client is what
actually speaks in it. See `Reply` and controller._apply_voice.
Kept dependency-free beyond `requests` so it's easy to unit test with mocks.
"""
from __future__ import annotations
import json
import subprocess
from pathlib import Path
from typing import Callable, NamedTuple, Optional
import requests
from . import config, sudo_askpass
_MAX_RELAY_HOPS = 16
class ServerError(Exception):
"""Raised when the server responds with an error payload or unreachable."""
class Reply(NamedTuple):
"""One final reply from the desk API. *voice_id* is set only when the
server tagged this reply with a `speak_as` voice; *voice_name* is the
human-readable name that came with it (may be empty even when the id
isn't). Both empty means "say it in the usual voice"."""
text: str
voice_id: str = ""
voice_name: str = ""
# True when the sentences were already spoken as they streamed in. The
# text is still carried — the follow-up rule needs to see whether the
# answer ended on a question — it just must not be read out again.
spoken: bool = False
def _headers() -> dict:
return {"X-Desk-Api-Key": config.API_KEY}
def check_health(timeout: float = 10.0) -> dict:
response = requests.get(f"{config.SERVER_URL}/desk/health", headers=_headers(), timeout=timeout)
response.raise_for_status()
return response.json()
def run_local_command(command: str, timeout: int = None) -> str:
"""Execute a command relayed by the server, exactly as bolt_desk.py does —
"full desktop control" for things like "open firefox" or "how full is my
disk". Runs as the current desktop user. See README security notes.
`sudo` gets special handling: the pet has no terminal, so sudo would sit
waiting on a tty that nobody is looking at. With SUDO_ASKPASS_PROMPT on,
bare `sudo` becomes `sudo -A` and the password is collected in a desktop
dialog you have to answer — which also gives those commands a longer
timeout, since a human has to notice the window and type."""
env = None
if config.SUDO_ASKPASS_PROMPT and sudo_askpass.needs_password_prompt(command):
helper = sudo_askpass.find_helper()
if helper:
env = sudo_askpass.environment(helper)
command = sudo_askpass.add_askpass_flag(command)
timeout = timeout or config.SUDO_COMMAND_TIMEOUT_SECONDS
timeout = timeout or config.COMMAND_TIMEOUT_SECONDS
try:
completed = subprocess.run(
command, shell=True, capture_output=True, text=True,
timeout=timeout, cwd=str(Path.home()), env=env,
)
output = (completed.stdout or "") + (completed.stderr or "")
return f"[exit {completed.returncode}]\n{output}"[:6000]
except subprocess.TimeoutExpired:
return f"[command timed out after {timeout}s]"
except Exception as exc:
return f"[command failed: {exc}]"
def converse(
text: str,
on_command: Callable[[str], str] = run_local_command,
timeout: float = 120.0,
on_say: Optional[Callable[[str], None]] = None,
) -> Reply:
"""Send one turn of conversation to the desk API, relaying any commands
the server sends back until it produces a final reply.
*on_command* is injectable for tests; defaults to actually running the
command locally (matching bolt_desk.py's behavior).
*on_say* is called with a short holding line ("give me a sec") when the
server sends one alongside a command. It is the difference between silence
and an answer while a tool runs: the model's acknowledgement used to be
discarded server-side, so the whole round trip was dead air and the model
then repeated itself in the final reply. Optional, so an older pet against
a newer server simply stays quiet as before.
"""
headers = _headers()
try:
response = requests.post(
f"{config.SERVER_URL}/desk/converse",
json={"session_id": config.SESSION_ID, "text": text},
headers=headers, timeout=timeout,
)
payload = response.json()
except Exception as exc:
raise ServerError(f"couldn't reach the server: {exc}") from exc
for _ in range(_MAX_RELAY_HOPS):
if payload.get("type") != "command":
break
holding = str(payload.get("say") or "").strip()
if holding and on_say is not None:
# Spoken *before* the command runs — that is the whole point.
try:
on_say(holding)
except Exception:
pass # a failed acknowledgement must not cost the tool call
output = on_command(str(payload.get("command") or ""))
try:
response = requests.post(
f"{config.SERVER_URL}/desk/tool_result",
json={
"session_id": config.SESSION_ID,
"token": payload.get("token"),
"output": output,
},
headers=headers, timeout=180,
)
payload = response.json()
except Exception as exc:
raise ServerError(f"couldn't reach the server during tool relay: {exc}") from exc
if payload.get("type") == "reply":
return Reply(
text=str(payload.get("text") or ""),
voice_id=str(payload.get("voice_id") or ""),
voice_name=str(payload.get("voice_name") or ""),
)
raise ServerError(str(payload.get("error") or "unknown server response"))
def converse_stream(
text: str,
on_say: Callable[[str], None],
on_command: Callable[[str], str] = run_local_command,
timeout: float = 180.0,
) -> Reply:
"""Same turn as converse(), but speaking each sentence as it arrives.
Without this the pet waits out the *entire* model call before a single
word is heard; with it the wait is time-to-first-sentence, which on a
multi-sentence answer is most of the difference.
Falls back by raising ServerError before anything has been spoken — the
caller then retries the ordinary path and the user never finds out. Once a
sentence *has* been spoken there is no going back, so late failures end the
turn with whatever was said rather than repeating it.
"""
spoke_anything = False
try:
response = requests.post(
f"{config.SERVER_URL}/desk/converse_stream",
json={"session_id": config.SESSION_ID, "text": text},
headers=_headers(), timeout=timeout, stream=True,
)
response.raise_for_status()
for raw in response.iter_lines(decode_unicode=True):
if not raw:
continue
try:
event = json.loads(raw)
except (TypeError, ValueError):
continue
kind = str(event.get("type") or "")
if kind == "say":
line = str(event.get("text") or "").strip()
if line:
spoke_anything = True
on_say(line)
elif kind == "command":
# The tool loop is request/response, so the rest of the turn
# finishes through the ordinary relay rather than inside the
# stream — one protocol for tools, not two.
response.close()
return _finish_relay(event, on_command, on_say)
elif kind == "reply":
return Reply(
text=str(event.get("text") or ""),
voice_id=str(event.get("voice_id") or ""),
voice_name=str(event.get("voice_name") or ""),
spoken=bool(event.get("already_spoken")),
)
elif kind == "error":
raise ServerError(str(event.get("error") or "stream failed"))
except ServerError:
raise
except Exception as exc:
if spoke_anything:
# Half a reply is out loud already; ending quietly beats saying it
# all again through the fallback path.
return Reply(text="", spoken=True)
raise ServerError(f"streaming failed: {exc}") from exc
raise ServerError("stream ended without a reply")
def _finish_relay(event: dict, on_command, on_say) -> Reply:
"""Run the tool the stream handed over, then continue the classic relay."""
payload = dict(event)
headers = _headers()
for _ in range(_MAX_RELAY_HOPS):
if payload.get("type") != "command":
break
holding = str(payload.get("say") or "").strip()
if holding and on_say is not None:
try:
on_say(holding)
except Exception:
pass
output = on_command(str(payload.get("command") or ""))
try:
response = requests.post(
f"{config.SERVER_URL}/desk/tool_result",
json={"session_id": config.SESSION_ID,
"token": payload.get("token"), "output": output},
headers=headers, timeout=180,
)
payload = response.json()
except Exception as exc:
raise ServerError(f"couldn't reach the server during tool relay: {exc}") from exc
if payload.get("type") == "reply":
return Reply(
text=str(payload.get("text") or ""),
voice_id=str(payload.get("voice_id") or ""),
voice_name=str(payload.get("voice_name") or ""),
)
raise ServerError(str(payload.get("error") or "unknown server response"))
def list_outbox_files(timeout: float = 15.0) -> list:
"""Files the server has queued for this session via its deliver_files
tool (e.g. "send me that report" during a conversation) — each entry has
id/name/size. Downloading one (download_outbox_file) dequeues it
server-side, so a file is only ever handed out once."""
try:
response = requests.get(
f"{config.SERVER_URL}/desk/files",
params={"session_id": config.SESSION_ID},
headers=_headers(), timeout=timeout,
)
response.raise_for_status()
return list(response.json().get("files") or [])
except Exception as exc:
raise ServerError(f"couldn't list delivered files: {exc}") from exc
def download_outbox_file(file_id: str, timeout: float = 60.0) -> bytes:
"""Fetches and dequeues one file listed by list_outbox_files()."""
try:
response = requests.get(
f"{config.SERVER_URL}/desk/files/{file_id}",
params={"session_id": config.SESSION_ID},
headers=_headers(), timeout=timeout,
)
response.raise_for_status()
return response.content
except Exception as exc:
raise ServerError(f"couldn't download delivered file {file_id!r}: {exc}") from exc
def report_status(timeout: float = 15.0) -> Optional[str]:
"""Heartbeat — lets the desk API attach a pending spoken announcement
(proactive nudges, reminders fired since the last heartbeat) that the pet
can speak unprompted, exactly like the phone/desk clients. A pet has no
battery/GPS to report, so the device-status fields
(battery/is_charging/latitude/longitude/address, all optional server-side)
are simply omitted.
Returns the announcement text to speak, or None if there's nothing pending.
"""
try:
response = requests.post(
f"{config.SERVER_URL}/desk/report_status",
json={"session_id": config.SESSION_ID}, headers=_headers(), timeout=timeout,
)
response.raise_for_status()
data = response.json()
except Exception as exc:
raise ServerError(f"heartbeat failed: {exc}") from exc
reply = data.get("reply")
return str(reply) if reply else None
bolt_pet/speech.py
-194
View File
@@ -1,194 +0,0 @@
"""Everything the pet says, and the policy differences between kinds of saying.
There used to be four of these in controller.py — a reply, a holding line, a
streamed sentence, a dialogue scene — each written when its feature was built,
each repeating the same dance: transition state, show the bubble, maybe record
history, reset barge-in, call TTS, reset barge-in again, resume state, decide
whether to keep the mic open. Only the *policy* differed, and the copies had
already started to drift: one forgot to arm barge-in, another logged a
different prefix, a third skipped the follow-up rule.
So the dance lives here once, and the differences are data:
reply the answer. Transcript, bubble, follow-up rule, ends IDLE.
holding "give me a sec" while a tool runs. No transcript — it is
filler, and the transcript should keep the answer. Resumes
whatever state it interrupted, because the turn isn't over.
stream one sentence of a streamed answer. Transcript and bubble like
a reply, but stays TALKING so the sprite doesn't flicker
between sentences, and the follow-up rule waits for the last.
scene a dialoguectl take. Transcript (the user heard it), resumes
mid-turn like a holding line.
The controller keeps the pipeline; this keeps the rules about talking.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Optional
from . import config, speech_text
from .state import PetState
@dataclass(frozen=True)
class Utterance:
"""One thing to say, and how saying it should behave."""
text: str
record: bool = True # goes in the transcript, or is it filler?
resume: bool = False # return to the state it interrupted (mid-turn)
hold_talking: bool = False # stay TALKING afterwards (more is coming)
follow_up: bool = True # may leave the mic open if it ends on a question
interruptible: bool = True # arm barge-in for this one
log_prefix: str = "Bolt"
@classmethod
def reply(cls, text: str) -> "Utterance":
return cls(text)
@classmethod
def holding(cls, text: str) -> "Utterance":
# Filler: no transcript, no follow-up, and not interruptible — cutting
# off "give me a sec" would strand the tool that is already running.
return cls(text, record=False, resume=True, follow_up=False,
interruptible=False, log_prefix="Bolt (holding)")
@classmethod
def stream_chunk(cls, text: str) -> "Utterance":
return cls(text, hold_talking=True, follow_up=False)
@classmethod
def scene(cls, text: str) -> "Utterance":
return cls(text, resume=True, follow_up=False)
class Speaker:
"""Says things on behalf of the controller.
Collaborators are passed in rather than reached for, so the whole of
speaking is testable without Qt, audio hardware or a state machine: hand it
fakes and assert on what came out."""
def __init__(self, *, state, tts, history=None, on_said=None, on_log=None,
barge_in: Callable[[], object] = lambda: None,
voice_id: Callable[[], str] = lambda: "",
detail_of: Callable[[], str] = lambda: "",
on_level: Optional[Callable[[float], None]] = None):
self._state = state
self._tts = tts
self._history = history
self._on_said = on_said or (lambda _text: None)
self._on_log = on_log or (lambda _msg: None)
# Read through a callable rather than held: the detector is built after
# the speaker (it needs the mic stream), replaced when barge-in mode
# changes, and swapped by tests. Two copies of it drifting apart is a
# bug nobody notices until the wake model starts hearing the pet.
self._barge_in_of = barge_in
self._voice_id = voice_id
# What actually fired, captured BEFORE the post-playback reset. Read it
# afterwards and every interruption reports 0.000 at frame 0 — which
# looks like hard evidence and is nothing of the sort.
self._detail_of = detail_of
self.last_detail = ""
self._on_level = on_level
@property
def _barge_in(self):
try:
return self._barge_in_of()
except Exception:
return None
def say(self, utterance: Utterance) -> bool:
"""Speak it. Returns False if it was interrupted.
The one place that knows the order these steps go in — which is the
point, because getting that order wrong is invisible until the wake
model starts hearing the pet's own voice."""
line = speech_text.for_display(utterance.text)
if not line:
return True
resume_state = self._state.state
if self._state.state != PetState.TALKING:
self._state.transition(PetState.TALKING)
self._on_said(line)
self._on_log(f"{utterance.log_prefix}: {line}")
if utterance.record and self._history is not None:
self._history(utterance.text)
should_stop = None
if self._barge_in is not None and utterance.interruptible:
self._barge_in.reset()
should_stop = self._barge_in.check
completed = self._tts.speak(
utterance.text,
on_error=lambda exc: self._on_log(f"TTS failed: {exc}"),
should_stop=should_stop,
voice_id=self._voice_id() or None,
on_level=self._on_level,
)
self.last_detail = self._detail_of()
if self._barge_in is not None:
# Playback fed the pet's own voice into the wake model's rolling
# window. Clear it before the idle listener scores again, or the
# last sentence is still in there being re-heard.
self._barge_in.reset()
if self._on_level is not None:
self._on_level(0.0) # mouth closed; nothing is playing now
if utterance.resume and resume_state in (PetState.THINKING, PetState.IDLE):
self._state.transition(resume_state)
return bool(completed)
def say_pcm(self, utterance: Utterance, *, pcm, sample_rate: int) -> bool:
"""Speak audio that is already synthesized — a dialoguectl scene.
Same policy, same barge-in handling, same mouth; only the source of
the samples differs. Sharing this is why a scene can be talked over
exactly like an ordinary reply."""
line = speech_text.for_display(utterance.text)
resume_state = self._state.state
if self._state.state != PetState.TALKING:
self._state.transition(PetState.TALKING)
if line:
self._on_said(line)
self._on_log(f"{utterance.log_prefix}: {line}")
if utterance.record and self._history is not None:
self._history(utterance.text)
should_stop = None
if self._barge_in is not None and utterance.interruptible:
self._barge_in.reset()
should_stop = self._barge_in.check
completed = self._tts.play_pcm(
pcm, sample_rate, should_stop=should_stop, on_level=self._on_level)
self.last_detail = self._detail_of()
if self._barge_in is not None:
self._barge_in.reset()
if self._on_level is not None:
self._on_level(0.0)
if utterance.resume and resume_state in (PetState.THINKING, PetState.IDLE):
self._state.transition(resume_state)
return bool(completed)
def follow_up_decision(text: str, *, completed: bool, follow_ups: int) -> tuple[bool, str]:
"""Whether to keep listening after speaking, and why.
Split out as a pure function because it is a *rule* with an off-by-one cap
in it, and rules with counters deserve a test that doesn't need audio."""
if not completed:
return True, "interrupted"
if not speech_text.is_question(text):
return False, ""
cap = config.FOLLOW_UP_MAX_TURNS
if not config.FOLLOW_UP_LISTEN:
return False, ""
if cap > 0 and follow_ups >= cap:
return False, f"follow-up cap ({cap}) reached"
return True, "question"
bolt_pet/speech_text.py
-173
View File
@@ -1,173 +0,0 @@
"""Turn a server reply into something worth *hearing*.
The server's persona writes for a chat window: markdown emphasis, bullet
lists, emoji, bare URLs. A TTS voice reads those literally ("asterisk
asterisk OS colon", "https colon slash slash..."), so everything spoken goes
through for_speech() first. Pure string logic, no Qt/audio imports — cheap to
unit test (see tests/test_speech_text.py).
for_display() is the lighter sibling used for the speech bubble: it drops the
markdown *syntax* but keeps emoji and punctuation, since those render fine.
"""
from __future__ import annotations
import re
# Pictographs, symbols, flags, dingbats, arrows, box drawing, variation
# selectors, ZWJ — anything a voice would either skip or read as a name
# ("black right-pointing triangle").
_EMOJI = re.compile(
"["
"\U0001F000-\U0001FAFF" # emoji / pictographs / symbols blocks
"\U00002190-\U000021FF" # arrows
"\U00002300-\U000023FF" # misc technical (⌘ ⏱ …)
"\U000025A0-\U000027BF" # geometric shapes, misc symbols, dingbats
"\U00002B00-\U00002BFF" # extra arrows / shapes
"\U0000FE00-\U0000FE0F" # variation selectors
"\U0001F1E6-\U0001F1FF" # regional indicators (flags)
"\U0000200D" # zero-width joiner
"]+",
flags=re.UNICODE,
)
# Characters that are markup or decoration rather than speech. Kept out of
# the spoken text entirely; ordinary punctuation (. , ! ? ; : ' " ( ) -) is
# preserved because it shapes prosody.
_UNSPEAKABLE = re.compile(r"[*_#`~^|<>\\{}\[\]/=+@©®™•·–—]+")
_FENCED_CODE = re.compile(r"```.*?```", re.DOTALL)
_INLINE_CODE = re.compile(r"`([^`]*)`")
_MD_IMAGE = re.compile(r"!\[([^\]]*)\]\([^)]*\)")
_MD_LINK = re.compile(r"\[([^\]]+)\]\([^)]*\)")
_URL = re.compile(r"\b(?:https?://|www\.)\S+")
_HEADING = re.compile(r"^\s{0,3}#{1,6}\s*", re.MULTILINE)
_BLOCKQUOTE = re.compile(r"^\s{0,3}>\s?", re.MULTILINE)
_RULE = re.compile(r"^\s*([-*_=])(?:\s*\1){2,}\s*$", re.MULTILINE)
_BULLET = re.compile(r"^\s*(?:[-*+•·]|\d+[.)])\s+", re.MULTILINE)
_EMPHASIS = re.compile(r"(\*{1,3}|_{1,3})(\S(?:.*?\S)?)\1", re.DOTALL)
_TABLE_PIPE = re.compile(r"[ \t]*\|[ \t]*")
# Symbols worth saying out loud rather than dropping — a bare "&" read as
# nothing turns "R&D" into "RD". Split in two because the non-ASCII ones sit
# inside the arrow/symbol blocks _EMOJI strips, so they have to be worded
# before that pass; the ASCII ones must wait until *after* markdown parsing
# (an "=" turned into " equals " would stop a "====" rule matching _RULE).
_PRE_SPOKEN_SYMBOLS = {
"→": " to ",
"×": " by ",
"°": " degrees ",
"≈": " about ",
}
_SPOKEN_SYMBOLS = {
"&": " and ",
"%": " percent ",
"@": " at ",
"+": " plus ",
"=": " equals ",
}
_MULTI_SPACE = re.compile(r"[ \t]+")
_MULTI_PUNCT = re.compile(r"(?:\s*\.){2,}")
def _strip_markdown(text: str, *, keep_emoji: bool) -> str:
text = _FENCED_CODE.sub(" (code) ", text)
text = _INLINE_CODE.sub(r"\1", text)
text = _MD_IMAGE.sub(r"\1", text)
text = _MD_LINK.sub(r"\1", text)
text = _RULE.sub("", text)
text = _HEADING.sub("", text)
text = _BLOCKQUOTE.sub("", text)
text = _EMPHASIS.sub(r"\2", text)
if not keep_emoji:
text = _EMOJI.sub(" ", text)
return text
def _bullets_to_sentences(text: str) -> str:
"""A read-aloud list needs pauses where the eye would see line breaks,
otherwise "OS Linux Uptime 1 day CPU load moderate" runs together."""
lines = [_BULLET.sub("", line).strip() for line in text.splitlines()]
lines = [line for line in lines if line]
if len(lines) < 2:
return lines[0] if lines else ""
# A bullet like "OS: Linux" ends mid-thought — give the voice a full stop
# so consecutive items don't slur into one run-on sentence.
return " ".join(line if line[-1] in ".!?:,;" else line + "." for line in lines)
# ElevenLabs v3 delivery tags — "[laughing]", "[whispering]", "[sighs]". They
# are instructions to the *dialogue* model (see dialogue.py), and only inside a
# dialoguectl scene. Left in an ordinary reply they reach eleven_flash_v2,
# which has no idea what they mean and simply reads the word: observed live
# 2026-07-31, the pet announcing "laughing That one came through clean".
#
# Matched narrowly — a bracketed adverb/gerund, or one of the noise words that
# aren't either — so real bracketed text ("[1]", "[see the docs]") survives.
_AUDIO_TAG_RE = re.compile(
r"\[\s*(?:"
r"[a-z]+(?:ly|ing)"
r"|laughs?|chuckles?|sighs?|exhales?|inhales?|gulps?|pause|beat"
r"|clears throat|shouts?|whispers?|cries|sings?|gasps?|snorts?"
r"|sarcastic|excited|curious|nervous|angry|sad|happy|deadpan|flat"
r")\s*\]",
re.IGNORECASE,
)
def strip_audio_tags(text: str) -> str:
"""Remove v3 delivery tags from text destined for the ordinary voice."""
return _AUDIO_TAG_RE.sub(" ", str(text or ""))
def for_speech(text: str) -> str:
"""Plain prose for the TTS engine: no markdown, no emoji, no bare URLs,
no stray symbols that would be read out character by character."""
text = strip_audio_tags((text or "").strip())
if not text:
return ""
for symbol, spoken in _PRE_SPOKEN_SYMBOLS.items():
text = text.replace(symbol, spoken)
text = _strip_markdown(text, keep_emoji=False)
text = _URL.sub(" link ", text)
text = _TABLE_PIPE.sub(", ", text)
text = _bullets_to_sentences(text)
for symbol, spoken in _SPOKEN_SYMBOLS.items():
text = text.replace(symbol, spoken)
text = _UNSPEAKABLE.sub(" ", text)
text = _MULTI_PUNCT.sub(".", text)
text = _MULTI_SPACE.sub(" ", text)
text = re.sub(r"\s+([.,!?;:])", r"\1", text)
return text.strip()
def is_question(text: str) -> bool:
"""True if the reply *ends* by asking the user something — the cue for
the pet to keep listening instead of making you say the wake word again.
Deliberately only looks at the end. A reply that asks something in
passing ("What time is it? It's 7:15.") isn't waiting on an answer,
whereas one that finishes on a question mark is. The test runs on the
spoken form, so a '?' that only exists inside a stripped code block or a
URL doesn't count, and trailing decoration (emoji, quotes, brackets) is
peeled off first so "Ready to go? 🚀" still reads as a question."""
spoken = for_speech(text)
while spoken and not (spoken[-1].isalnum() or spoken[-1] == "?"):
spoken = spoken[:-1]
return spoken.endswith("?")
def for_display(text: str) -> str:
"""What the speech bubble shows: markdown syntax removed (the bubble
can't render it) but emoji and layout-ish punctuation left alone.
Delivery tags go too — the voice no longer says them, so showing them
would caption a laugh nobody heard."""
text = strip_audio_tags((text or "").strip())
if not text:
return ""
text = _strip_markdown(text, keep_emoji=True)
lines = [_BULLET.sub("• ", line).strip() for line in text.splitlines()]
text = " ".join(line for line in lines if line)
return _MULTI_SPACE.sub(" ", text).strip()
bolt_pet/state.py
-76
View File
@@ -1,76 +0,0 @@
"""Pet state machine — pure logic, no Qt/audio dependencies, so it's cheap
to unit test. The UI layer (ui/pet_window.py) reacts to state changes by
swapping the active sprite animation; the worker thread (ui/app.py) drives
transitions as the mic/wake/converse/tts pipeline progresses.
"""
from __future__ import annotations
from enum import Enum
from typing import Callable, Optional
class PetState(str, Enum):
IDLE = "idle" # waiting for the wake phrase (or a click)
LISTENING = "listening" # actively recording an utterance
THINKING = "thinking" # waiting on the server (STT done, converse in flight)
TALKING = "talking" # playing back the TTS reply
ERROR = "error" # brief flash state on failure, then back to idle
# States it's valid to move to from each state. Keeps ad-hoc bugs (e.g.
# firing TALKING before a reply exists) from silently passing through.
#
# IDLE -> TALKING is legal (not just IDLE -> LISTENING) because of proactive
# announcements: the heartbeat poll (controller.py's _maybe_heartbeat) can
# make the pet speak unprompted — a reminder firing, a nudge from the server
# — without the user having said anything first, so there's no preceding
# LISTENING/THINKING leg for that turn.
#
# TALKING -> THINKING is the mirror case: a `dialoguectl` scene is played
# *mid-turn*, while the server is still waiting on the tool result, so the pet
# talks and then goes back to waiting rather than falling to IDLE (which would
# make it look like the turn had ended).
_TRANSITIONS: dict[PetState, set[PetState]] = {
PetState.IDLE: {PetState.LISTENING, PetState.TALKING, PetState.ERROR},
PetState.LISTENING: {PetState.THINKING, PetState.IDLE, PetState.ERROR},
PetState.THINKING: {PetState.TALKING, PetState.IDLE, PetState.ERROR},
PetState.TALKING: {PetState.IDLE, PetState.THINKING, PetState.ERROR},
PetState.ERROR: {PetState.IDLE},
}
class InvalidTransition(Exception):
pass
class PetStateMachine:
def __init__(self, on_change: Optional[Callable[[PetState, PetState], None]] = None):
self._state = PetState.IDLE
self._on_change = on_change
@property
def state(self) -> PetState:
return self._state
def transition(self, new_state: PetState) -> None:
if new_state == self._state:
return
allowed = _TRANSITIONS.get(self._state, set())
if new_state not in allowed:
raise InvalidTransition(f"{self._state} -> {new_state} is not allowed")
old_state = self._state
self._state = new_state
if self._on_change is not None:
self._on_change(old_state, new_state)
def force(self, new_state: PetState) -> None:
"""Bypass the transition table — used only for recovering to IDLE
from an unexpected/edge-case state (e.g. after an exception mid
pipeline). Prefer transition() everywhere else."""
old_state = self._state
if new_state == old_state:
return
self._state = new_state
if self._on_change is not None:
self._on_change(old_state, new_state)
bolt_pet/sudo_askpass.py
-145
View File
@@ -1,145 +0,0 @@
"""Graphical password prompts for server-relayed `sudo` commands.
The pet has no terminal. When the server relays something like
`sudo apt update`, sudo tries to read a password from a tty, finds none (or
finds the terminal the pet was launched from, which you're not looking at),
and the command fails with no way to answer it.
sudo's own answer to this is SUDO_ASKPASS: with `-A`, it runs a helper
program and reads the password from the helper's stdout instead of a tty.
Any GUI prompt that prints what was typed works, so this module finds a real
askpass binary if one is installed and otherwise generates a one-line wrapper
around zenity/kdialog, which every desktop has one of.
Worth being clear about what this changes: the prompt is a *feature*, not
just plumbing. Server-relayed commands already run as your desktop user (see
server_client.run_local_command); this lets them ask to run as root, and the
dialog is the only thing standing between "Bolt decided to run sudo" and it
happening. Leave SUDO_ASKPASS_PROMPT on, and read the dialogs.
The parts that decide *what* to run are pure functions so they're tested
without a display, a password, or a working sudo.
"""
from __future__ import annotations
import os
import re
import shutil
import stat
from pathlib import Path
from typing import Callable, Optional
from . import config
# Real askpass binaries, in preference order. These are purpose-built for
# this (they grab the keyboard, hide the input, and don't leave the password
# in a process argument), so they win over a generated wrapper.
_KNOWN_HELPERS = (
"/usr/bin/ssh-askpass",
"/usr/lib/ssh/ssh-askpass",
"/usr/lib/openssh/gnome-ssh-askpass3",
"/usr/lib/openssh/gnome-ssh-askpass",
"/usr/libexec/openssh/ssh-askpass",
"/usr/bin/ksshaskpass",
"/usr/bin/lxqt-openssh-askpass",
)
# Dialog tools we can wrap when no askpass binary exists. Each must print the
# typed password to stdout and nothing else.
_WRAPPABLE = {
"zenity": '{tool} --password --title="Bolt" --text="$1" 2>/dev/null',
"kdialog": '{tool} --password "$1" --title "Bolt" 2>/dev/null',
}
# `sudo` at the start of the command or right after a shell separator, not
# already carrying a flag. Deliberately conservative: a `sudo` inside a
# quoted string or a heredoc is left alone, because rewriting it could change
# what the command means.
_SUDO = re.compile(r"(^|[;&|]\s*|\n\s*)(sudo)(\s+)(?!-)")
def add_askpass_flag(command: str) -> str:
"""Insert `-A` after each bare `sudo`, so it prompts through the helper
instead of a tty. Commands that already pass a flag (`sudo -n`, `sudo -A`,
`sudo -u bob`) are left exactly as they are — the caller was explicit."""
return _SUDO.sub(r"\1\2 -A\3", command or "")
def needs_password_prompt(command: str) -> bool:
"""True if *command* has a `sudo` that might sit waiting on a dialog.
Used to give those commands a longer timeout — 30 seconds is fine for a
shell command and nowhere near enough for a human to notice a window,
read it, and type a password."""
return bool(_SUDO.search(command or ""))
def helper_script(tool_path: str) -> str:
"""The wrapper script for a dialog *tool_path*. sudo passes its prompt
("[sudo] password for maji:") as $1, which is worth showing — it names
the user the password is for."""
name = Path(tool_path).name
body = _WRAPPABLE[name].format(tool=tool_path)
return f"#!/bin/sh\n# Generated by bolt-pet. Prints the typed password on stdout for sudo -A.\n{body}\n"
def _default_cache_dir() -> Path:
base = os.environ.get("XDG_CACHE_HOME") or (Path.home() / ".cache")
return Path(base) / "bolt-pet"
def find_helper(
configured: str = None,
is_executable: Callable[[str], bool] = None,
which: Callable[[str], Optional[str]] = None,
cache_dir: Path = None,
write: bool = True,
) -> Optional[str]:
"""Path to an askpass helper, or None if the desktop has nothing we can
use. Order: whatever SUDO_ASKPASS_HELPER names, then a real askpass
binary, then a generated wrapper around zenity/kdialog.
The lookups are injectable so the resolution order is testable on a box
with a different set of these installed than yours."""
configured = config.SUDO_ASKPASS_HELPER if configured is None else configured
is_executable = is_executable or (lambda path: os.path.isfile(path) and os.access(path, os.X_OK))
which = which or shutil.which
if configured:
return configured if is_executable(configured) else None
for candidate in _KNOWN_HELPERS:
if is_executable(candidate):
return candidate
for tool in _WRAPPABLE:
tool_path = which(tool)
if not tool_path:
continue
if not write:
return tool_path
return _write_wrapper(tool_path, cache_dir or _default_cache_dir())
return None
def _write_wrapper(tool_path: str, cache_dir: Path) -> Optional[str]:
"""Drop the wrapper script somewhere sudo can execute it. Mode 0700: it
isn't secret, but it's a thing that pops up a password box, so nobody
else on the machine gets to edit it."""
try:
cache_dir.mkdir(parents=True, exist_ok=True)
script = cache_dir / "askpass.sh"
source = helper_script(tool_path)
if not script.exists() or script.read_text(encoding="utf-8") != source:
script.write_text(source, encoding="utf-8")
script.chmod(stat.S_IRWXU)
return str(script)
except Exception:
return None # no prompt is better than a crashed command relay
def environment(helper: str, base: dict = None) -> dict:
"""The subprocess environment with SUDO_ASKPASS pointed at *helper*."""
env = dict(os.environ if base is None else base)
env["SUDO_ASKPASS"] = helper
return env
bolt_pet/ui/__init__.py
View File
bolt_pet/ui/app.py
-158
View File
@@ -1,158 +0,0 @@
"""Wires everything together: QApplication, the pet window, the tray icon,
the history / wake-tuner windows, the global push-to-talk hotkey, and the
background PetController thread that owns the mic/wake/server/TTS pipeline.
Everything the controller wants the UI to do arrives as a Qt signal, so the
worker thread never touches a widget.
"""
from __future__ import annotations
import sys
from PySide6.QtCore import QThread
from PySide6.QtWidgets import QApplication, QMessageBox
from .. import config, updater
from ..controller import PetController
from ..hotkey import GlobalHotkey
from ..state import PetState
from .history_window import HistoryWindow
from .mic_picker import MicPickerDialog
from .pet_window import PetWindow
from .tray import PetTray
from .wake_tuner import WakeTunerWindow
def _log(message: str) -> None:
print(message, flush=True)
def run() -> int:
app = QApplication(sys.argv)
app.setQuitOnLastWindowClosed(False) # tray-driven app; closing the pet isn't "quit"
window = PetWindow()
window.show()
controller = PetController()
thread = QThread()
controller.moveToThread(thread)
thread.started.connect(controller.run)
controller.state_changed.connect(lambda value: window.set_state(PetState(value)))
controller.said.connect(window.say)
# Lip-sync: the loudness of the audio actually going to the speakers.
controller.mouth.connect(window.set_mouth)
controller.log.connect(_log)
controller.action.connect(window.apply_action) # petctl move/emote/say/...
controller.finished.connect(thread.quit)
window.talk_requested.connect(controller.request_talk_now)
# The window owns the screen list and tells the controller about it, so
# both ends agree on what "monitor 2" means (see monitors.py).
window.monitors_changed.connect(controller.set_monitors)
window.pet_monitor_changed.connect(controller.set_pet_monitor)
# PetWindow publishes once in its constructor, which ran before those
# connections existed — so say it again now that anyone is listening.
window.publish_monitors()
window.copied.connect(lambda text: _log(f"Copied to clipboard: {text[:60]}"))
# The updater has already moved the checkout by the time this fires; all
# that's left is to let Qt tear down cleanly (so the mic and the tray
# icon are released) and then exec the new code. Doing the exec after
# app.exec() returns, rather than from the controller thread, is what
# guarantees the audio device is free before the new process opens it.
# Defined early so the mic picker (below) can trigger the same restart
# a self-update or self-restart does.
pending_restart = {"tag": None}
def _handle_restart(tag: str) -> None:
pending_restart["tag"] = tag
app.quit()
history_window = HistoryWindow(controller.history)
tuner_window = WakeTunerWindow(
get_threshold=controller.wake_threshold,
set_threshold=controller.set_wake_threshold,
get_stats=controller.wake_stats,
on_reset=controller.reset_wake_stats,
)
def _save_mic_device(value) -> None:
config.set_env_value("MIC_DEVICE", str(value) if value is not None else "")
_log(f"Microphone set to device #{value}." if value is not None else
"Microphone set to system default.")
answer = QMessageBox.question(
None, "Restart Bolt?",
"Restart now so Bolt starts listening on the new microphone?",
QMessageBox.Yes | QMessageBox.No,
)
if answer == QMessageBox.Yes:
_handle_restart("microphone change")
mic_picker = MicPickerDialog(
get_current=lambda: config.MIC_DEVICE,
on_save=_save_mic_device,
)
def _set_nap(napping: bool) -> None:
# Clicking the tray item pins the state; the schedule takes over again
# only after a restart or a `petctl nap off`.
controller.set_napping(napping)
window.set_napping(napping)
tray = PetTray(
on_talk_now=controller.request_talk_now,
on_toggle_mute=controller.toggle_mute,
on_quit=app.quit,
on_set_wander=window.set_wander_enabled,
wander_enabled=config.PET_WANDER,
on_set_click_through=window.set_click_through,
click_through_enabled=config.PET_CLICK_THROUGH,
on_set_nap=_set_nap,
on_show_history=history_window.show_refreshed,
on_show_wake_tuner=tuner_window.show_refreshed,
on_show_mic_picker=mic_picker.show_refreshed,
on_reset_voice=controller.reset_voice,
)
def _handle_napping(napping: bool) -> None:
window.set_napping(napping)
tray.set_napping(napping)
controller.napping.connect(_handle_napping)
# The server can hand Bolt a different voice mid-conversation (speak_as);
# the tray is where you get his own back.
controller.voice_changed.connect(tray.set_voice)
# Push-to-talk: a global hook, because the pet window never has focus.
# request_talk_now() only sets a threading.Event, so it's safe to call
# from pynput's listener thread.
hotkey = GlobalHotkey(config.PUSH_TO_TALK_HOTKEY, controller.request_talk_now)
problem = hotkey.start()
if problem:
_log(problem)
elif hotkey.running:
_log(f"Push-to-talk: {config.PUSH_TO_TALK_HOTKEY}")
controller.restart_requested.connect(_handle_restart)
def _shutdown() -> None:
hotkey.stop()
controller.stop()
thread.quit()
thread.wait(5000)
app.aboutToQuit.connect(_shutdown)
thread.start()
status = app.exec()
if pending_restart["tag"]:
_log(f"Restarting into {pending_restart['tag']}…")
try:
updater.restart() # never returns
except Exception as exc:
_log(f"Couldn't restart automatically ({exc}) — start the pet again by hand.")
return status
bolt_pet/ui/history_window.py
-83
View File
@@ -1,83 +0,0 @@
"""Scrollback for the speech bubble.
The bubble is transient by design, so anything Bolt said more than a few
seconds ago is gone. This is the "wait, what was that path again?" window:
the last HISTORY_LIMIT turns, copyable. Opened from the tray.
Reads a bolt_pet.history.ConversationHistory (pure logic, tested separately);
this file is only presentation.
"""
from __future__ import annotations
import time
from typing import Callable, Optional
from PySide6.QtCore import Qt
from PySide6.QtWidgets import (
QApplication, QDialog, QHBoxLayout, QPlainTextEdit, QPushButton, QVBoxLayout,
)
from ..history import ConversationHistory
def _clock(timestamp: float) -> str:
return time.strftime("%H:%M:%S", time.localtime(timestamp))
class HistoryWindow(QDialog):
def __init__(self, history: ConversationHistory, on_clear: Optional[Callable[[], None]] = None):
super().__init__()
self._history = history
self._on_clear = on_clear
self.setWindowTitle("Bolt — conversation history")
self.resize(620, 420)
self._view = QPlainTextEdit()
self._view.setReadOnly(True)
self._view.setLineWrapMode(QPlainTextEdit.WidgetWidth)
copy_button = QPushButton("Copy all")
copy_button.clicked.connect(self._copy_all)
clear_button = QPushButton("Clear")
clear_button.clicked.connect(self._clear)
close_button = QPushButton("Close")
close_button.clicked.connect(self.close)
close_button.setDefault(True)
buttons = QHBoxLayout()
buttons.addWidget(copy_button)
buttons.addWidget(clear_button)
buttons.addStretch(1)
buttons.addWidget(close_button)
layout = QVBoxLayout(self)
layout.addWidget(self._view)
layout.addLayout(buttons)
def refresh(self) -> None:
self._view.setPlainText(self._history.as_text(clock=_clock))
# Jump to the newest line — that's what you opened this for.
scrollbar = self._view.verticalScrollBar()
scrollbar.setValue(scrollbar.maximum())
def show_refreshed(self) -> None:
self.refresh()
self.show()
self.raise_()
self.activateWindow()
def _copy_all(self) -> None:
QApplication.clipboard().setText(self._history.as_text(clock=_clock))
def _clear(self) -> None:
self._history.clear()
if self._on_clear is not None:
self._on_clear()
self.refresh()
def keyPressEvent(self, event) -> None:
if event.key() == Qt.Key_Escape:
self.close()
return
super().keyPressEvent(event)
bolt_pet/ui/mic_picker.py
-98
View File
@@ -1,98 +0,0 @@
"""Microphone picker — MIC_DEVICE in .env is a device index most people have
no way to look up short of running a script. This dialog lists whatever
sounddevice can actually see, lets you click one, and writes its index back
to .env so it survives a restart.
The index, not the name, is what gets saved: the same device is commonly
listed once per host API (MME, DirectSound, WASAPI, WDM-KS, ...) under the
same or a near-identical name, which makes a bare name an ambiguous pick that
sounddevice will flatly refuse to resolve (see audio/mic.list_input_devices).
The index is exact.
The mic is only ever opened once, at startup (see controller.run()), so a
pick made here still needs a restart before Bolt actually uses it — this
dialog just makes finding and saving the right device painless, and offers
to do the restart for you.
"""
from __future__ import annotations
from typing import Callable, Optional
from PySide6.QtCore import Qt
from PySide6.QtWidgets import (
QDialog, QDialogButtonBox, QHBoxLayout, QLabel, QListWidget,
QListWidgetItem, QPushButton, QVBoxLayout,
)
from ..audio.mic import list_input_devices
_DEFAULT_LABEL = "System default"
class MicPickerDialog(QDialog):
def __init__(
self,
get_current: Callable[[], Optional[int]],
on_save: Callable[[Optional[int]], None],
):
super().__init__()
self._get_current = get_current
self._on_save = on_save
self.setWindowTitle("Bolt — select microphone")
self.resize(440, 320)
self._list = QListWidget()
refresh_button = QPushButton("Refresh")
refresh_button.clicked.connect(self.refresh)
top = QHBoxLayout()
top.addWidget(QLabel("Pick the microphone Bolt should listen on:"))
top.addStretch(1)
top.addWidget(refresh_button)
buttons = QDialogButtonBox(QDialogButtonBox.Save | QDialogButtonBox.Cancel)
buttons.accepted.connect(self._save)
buttons.rejected.connect(self.reject)
layout = QVBoxLayout(self)
layout.addLayout(top)
layout.addWidget(self._list)
layout.addWidget(buttons)
def refresh(self) -> None:
self._list.clear()
default_item = QListWidgetItem(_DEFAULT_LABEL)
default_item.setData(Qt.UserRole, None)
self._list.addItem(default_item)
current = self._get_current()
selected = default_item
try:
devices = list_input_devices()
except Exception as exc:
self._list.addItem(f"Could not list devices: {exc}")
devices = []
for device in devices:
item = QListWidgetItem(
f"{device['name']} [{device['hostapi']}] (#{device['index']})"
)
item.setData(Qt.UserRole, device["index"])
self._list.addItem(item)
if current == device["index"]:
selected = item
self._list.setCurrentItem(selected)
def _save(self) -> None:
item = self._list.currentItem()
value = item.data(Qt.UserRole) if item is not None else None
self._on_save(value)
self.accept()
def show_refreshed(self) -> None:
self.refresh()
self.show()
self.raise_()
self.activateWindow()
bolt_pet/ui/pet_window.py
-822
View File
@@ -1,822 +0,0 @@
"""The pet itself: a frameless, translucent, always-on-top window that
renders the current sprite animation, wanders the desktop on its own while
idle, walks/emotes on command from the server (see pet_actions.py), can be
dragged around, dims when napping, and turns a plain (non-drag) click into a
"talk now" request.
"""
from __future__ import annotations
import math
import random
import time
from pathlib import Path
from typing import Optional
from PySide6.QtCore import QPoint, Qt, QTimer, Signal
from PySide6.QtGui import (
QColor, QCursor, QFont, QFontMetrics, QPainter, QPainterPath, QPixmap, QRegion, QTransform,
)
from PySide6.QtWidgets import QApplication, QWidget
from .. import config, window_state
from ..monitors import Monitor
from ..state import PetState
from .sprite import WALK, SpriteSet
_DRAG_THRESHOLD_PX = 4
# Movement runs on its own ~30fps timer, independent of the (slower) sprite
# animation timer, so a stroll looks smooth even at IDLE_ANIMATION_FPS=6.
_WANDER_TICK_MS = 33
_EMOTE_TICKS = 36 # ~1.2s per emote at the tick rate above
_NAP_OPACITY = 0.35
# How far the pet travels per walk-cycle frame. The cycle is advanced by
# distance rather than by the animation clock so a planted paw tracks backwards
# at exactly the speed the window moves forwards — drive it off a timer instead
# and the feet skate whenever PET_WANDER_SPEED doesn't happen to match the fps.
# Eight frames at 13px is a ~104px stride cycle, a bit under the pet's width.
_WALK_PIXELS_PER_FRAME = 13.0
# How long a loudness level stays believable. The audio thread sends one per
# ~30ms while a clip plays; if they stop arriving (offline TTS has no envelope,
# or playback died) the mouth must not stay frozen mid-syllable, so after this
# long the ordinary looping animation takes back over.
_MOUTH_STALE_SECONDS = 0.35
def emote_transform(emote: str, progress: float) -> tuple[float, float, float, float]:
"""(dx, dy, rotation_degrees, scale) for an emote at *progress* 0..1.
Pure maths, deliberately separate from paintEvent so the motion curves can
be unit tested (and so adding an emote doesn't mean touching painting
code). Every emote must return to (0, 0, 0, 1) at progress 1.0, otherwise
the pet ends up permanently askew.
"""
progress = min(max(progress, 0.0), 1.0)
fade = math.sin(math.pi * progress) # 0 -> 1 -> 0, so it always lands home
tau = 2 * math.pi
if emote == "wave":
return 0.0, 0.0, 14.0 * fade * math.sin(tau * 2 * progress), 1.0
if emote in ("hop", "bounce"):
hops = 2 if emote == "hop" else 3
return 0.0, -22.0 * fade * abs(math.sin(math.pi * hops * progress)), 0.0, 1.0
if emote == "spin":
return 0.0, 0.0, 360.0 * progress % 360.0, 1.0
if emote == "nod":
return 0.0, 10.0 * fade * math.sin(tau * 2 * progress), 0.0, 1.0 - 0.05 * fade
if emote in ("shake", "wiggle"):
return 14.0 * fade * math.sin(tau * 3 * progress), 0.0, 0.0, 1.0
return 0.0, 0.0, 0.0, 1.0
class SpeechBubble(QWidget):
"""Small translucent word-bubble shown above the pet while it talks."""
_MAX_WIDTH = 260
_PADDING = 10
copied = Signal(str)
def __init__(self, parent: Optional[QWidget] = None):
super().__init__(parent, Qt.FramelessWindowHint | Qt.Tool)
self.setAttribute(Qt.WA_TranslucentBackground)
self.setAttribute(Qt.WA_ShowWithoutActivating)
self._text = ""
self._font = QFont()
self._font.setPointSize(10)
self._flash = "" # transient overlay ("Copied") drawn over the text
self._hide_timer = QTimer(self)
self._hide_timer.setSingleShot(True)
self._hide_timer.timeout.connect(self.hide)
self._flash_timer = QTimer(self)
self._flash_timer.setSingleShot(True)
self._flash_timer.timeout.connect(self._clear_flash)
self.setToolTip("Click to copy")
self.setCursor(Qt.PointingHandCursor)
self.hide()
@property
def text(self) -> str:
return self._text
def show_text(self, text: str, duration_ms: int = 6000) -> None:
text = (text or "").strip()
if not text:
self.hide()
return
self._text = text
self._relayout()
self.show()
self.raise_()
self._hide_timer.start(duration_ms)
# ── click to copy ────────────────────────────────────────────────────
# The bubble hides itself after a few seconds, which is fine for chat and
# awful for anything you needed to keep (a path, a number, a command's
# output). One click puts it on the clipboard; the tray's History window
# has the rest.
def mousePressEvent(self, event) -> None:
if event.button() != Qt.LeftButton or not self._text:
return
QApplication.clipboard().setText(self._text)
self.copied.emit(self._text)
self._flash = "Copied to clipboard"
self.update()
self._flash_timer.start(900)
self._hide_timer.start(2500) # linger a moment so the flash is visible
def _clear_flash(self) -> None:
self._flash = ""
self.update()
def _wrapped_lines(self) -> list[str]:
metrics = QFontMetrics(self._font)
words = self._text.split()
lines: list[str] = []
current = ""
max_text_width = self._MAX_WIDTH - 2 * self._PADDING
for word in words:
candidate = f"{current} {word}".strip()
if metrics.horizontalAdvance(candidate) <= max_text_width or not current:
current = candidate
else:
lines.append(current)
current = word
if current:
lines.append(current)
return lines[:6] # don't let a huge reply turn into a wall of bubble
def _relayout(self) -> None:
metrics = QFontMetrics(self._font)
lines = self._wrapped_lines()
text_width = max((metrics.horizontalAdvance(line) for line in lines), default=0)
width = min(self._MAX_WIDTH, text_width + 2 * self._PADDING)
height = metrics.height() * len(lines) + 2 * self._PADDING
self.resize(width, height)
def paintEvent(self, _event) -> None:
painter = QPainter(self)
painter.setRenderHint(QPainter.Antialiasing)
path = QPainterPath()
path.addRoundedRect(0, 0, self.width(), self.height(), 10, 10)
painter.fillPath(path, QColor(30, 30, 35, 230))
painter.setFont(self._font)
metrics = QFontMetrics(self._font)
if self._flash:
painter.setPen(QColor(150, 230, 170))
painter.drawText(self.rect(), Qt.AlignCenter, self._flash)
return
painter.setPen(QColor(240, 240, 245))
y = self._PADDING + metrics.ascent()
for line in self._wrapped_lines():
painter.drawText(self._PADDING, y, line)
y += metrics.height()
class PetWindow(QWidget):
talk_requested = Signal()
copied = Signal(str) # bubble text the user just put on the clipboard
# The screen layout, published *to* the controller (queued, cross-thread).
# The window is the only thing allowed to ask Qt about screens, so the
# controller and the window can never disagree about what "monitor 2"
# means — see monitors.py.
monitors_changed = Signal(list) # list[monitors.Monitor]
pet_monitor_changed = Signal(int) # 0-based index the pet is standing on
def __init__(self, sprite_dir: Optional[Path] = None, size: Optional[int] = None):
super().__init__()
flags = Qt.FramelessWindowHint | Qt.Tool
if config.PET_ALWAYS_ON_TOP:
flags |= Qt.WindowStaysOnTopHint
self.setWindowFlags(flags)
self.setAttribute(Qt.WA_TranslucentBackground)
self.sprites = SpriteSet(sprite_dir or (Path(__file__).resolve().parent.parent / "assets" / "sprites"),
size or config.PET_SIZE)
self.resize(self.sprites.size, self.sprites.size)
self._current_state = PetState.IDLE
self._drag_offset: Optional[QPoint] = None
self._press_pos: Optional[QPoint] = None
self._dragged = False
self._bubble = SpeechBubble()
self._bubble.copied.connect(self.copied)
self._napping = False
self._emote: Optional[str] = None
self._emote_tick = 0
self._mask_key = None
self._anim_timer = QTimer(self)
self._anim_timer.timeout.connect(self._advance_frame)
fps = max(1.0, config.IDLE_ANIMATION_FPS)
self._anim_timer.start(int(1000 / fps))
self._wander_enabled = config.PET_WANDER
self._wander_target: Optional[QPoint] = None
self._commanded_move = False # a petctl move — happens even mid-conversation
self._next_wander_at = 0.0
self._bob_offset = 0
# Lip-sync: loudness of what is playing right now, and when it arrived.
self._mouth_level: Optional[float] = None
self._mouth_at = 0.0
self._bob_phase = 0.0
self._walking = False
self._facing = 1 # +1 right, -1 left; the walk art is drawn facing right
self._walk_distance = 0.0
self._mirror_cache: dict[int, QPixmap] = {}
self._schedule_next_wander()
self._wander_timer = QTimer(self)
self._wander_timer.timeout.connect(self._movement_tick)
self._wander_timer.start(_WANDER_TICK_MS)
self._click_through = False
self.set_click_through(config.PET_CLICK_THROUGH)
self._place_start_position()
self._monitors: list[Monitor] = []
self._pet_monitor: Optional[int] = None
self._last_published_pos: Optional[QPoint] = None
app = QApplication.instance()
if app is not None:
# Screens come and go — a laptop docking, a TV waking up. Republish
# rather than letting Bolt jump to a monitor that's been unplugged.
app.screenAdded.connect(lambda _s: self.publish_monitors())
app.screenRemoved.connect(lambda _s: self.publish_monitors())
app.primaryScreenChanged.connect(lambda _s: self.publish_monitors())
self.publish_monitors()
# ── placement ────────────────────────────────────────────────────────
def _place_start_position(self) -> None:
screen = QApplication.primaryScreen()
geo = screen.availableGeometry() if screen else None
try:
x = int(config.PET_START_X) if config.PET_START_X else None
y = int(config.PET_START_Y) if config.PET_START_Y else None
except ValueError:
x = y = None
if x is None and y is None and config.PET_REMEMBER_POSITION:
remembered = window_state.load()
# Only if it still lands on a screen that exists — the usual reason
# a saved position is stale is that the monitor it was on has been
# unplugged, and restoring it faithfully would hide the pet.
if remembered is not None:
rectangles = [
(s.availableGeometry().left(), s.availableGeometry().top(),
s.availableGeometry().right(), s.availableGeometry().bottom())
for s in QApplication.screens()
]
if window_state.is_visible_on(*remembered, self.width(), rectangles):
x, y = remembered
if geo is not None:
x = geo.right() - self.width() - 40 if x is None else x
y = geo.bottom() - self.height() - 60 if y is None else y
self.move(x or 0, y or 0)
self._reposition_bubble()
def _reposition_bubble(self) -> None:
top_left = self.geometry().topLeft()
self._bubble.move(
top_left.x() + self.width() // 2 - self._bubble.width() // 2,
top_left.y() - self._bubble.height() - 8,
)
# ── server-driven actions (petctl) ───────────────────────────────────
def apply_action(self, action: dict) -> None:
"""Perform one parsed petctl action (see pet_actions.py). Called on
the UI thread via a queued signal from the controller."""
kind = action.get("action")
if kind == "move":
target = self._resolve_move_target(action)
if target is not None:
self._wander_target = target
self._commanded_move = True # overrides the idle-only rule
elif kind == "jump":
self.jump_to_monitor(int(action["monitor"]))
elif kind == "emote":
self.start_emote(action["emote"])
elif kind == "say":
self.say(action["text"])
elif kind == "wander":
self.set_wander_enabled(bool(action["enabled"]))
elif kind == "nap":
self.set_napping(bool(action["enabled"]))
def _resolve_move_target(self, action: dict) -> Optional[QPoint]:
geo = self._screen_geometry()
if "x" in action and "y" in action:
point = QPoint(int(action["x"]), int(action["y"]))
return self._clamp_to_screen(point, geo)
anchor = action.get("anchor")
if anchor == "cursor":
cursor = QCursor.pos()
return self._clamp_to_screen(
QPoint(cursor.x() - self.width() // 2, cursor.y() - self.height() // 2), geo
)
if geo is None:
return None
if anchor == "random":
return self._pick_wander_target()
margin = config.PET_WANDER_MARGIN
left, right = geo.left() + margin, geo.right() - self.width() - margin
top, bottom = geo.top() + margin, geo.bottom() - self.height() - margin
middle_x = geo.left() + (geo.width() - self.width()) // 2
middle_y = geo.top() + (geo.height() - self.height()) // 2
positions = {
"top-left": (left, top), "top": (middle_x, top), "top-right": (right, top),
"left": (left, middle_y), "center": (middle_x, middle_y), "right": (right, middle_y),
"bottom-left": (left, bottom), "bottom": (middle_x, bottom), "bottom-right": (right, bottom),
}
if anchor not in positions:
return None
return QPoint(*positions[anchor])
def _clamp_to_screen(self, point: QPoint, geo) -> QPoint:
if geo is None:
return point
x = min(max(point.x(), geo.left()), max(geo.left(), geo.right() - self.width()))
y = min(max(point.y(), geo.top()), max(geo.top(), geo.bottom() - self.height()))
return QPoint(x, y)
# ── emotes ───────────────────────────────────────────────────────────
def start_emote(self, emote: str) -> None:
self._emote = emote
self._emote_tick = 0
self.update()
def _advance_emote(self) -> None:
if self._emote is None:
return
self._emote_tick += 1
if self._emote_tick > _EMOTE_TICKS:
self._emote = None
self._emote_tick = 0
self.update()
def _emote_transform(self) -> tuple[float, float, float, float]:
if self._emote is None:
return 0.0, 0.0, 0.0, 1.0
return emote_transform(self._emote, self._emote_tick / _EMOTE_TICKS)
# ── napping (quiet hours / do-not-disturb) ───────────────────────────
def set_napping(self, napping: bool) -> None:
"""Dim and stand still. Purely cosmetic here — the controller is what
actually suppresses proactive speech."""
if napping == self._napping:
return
self._napping = napping
self.setWindowOpacity(_NAP_OPACITY if napping else 1.0)
if napping:
self._stop_walking()
self.update()
@property
def napping(self) -> bool:
return self._napping
# ── mouse transparency ───────────────────────────────────────────────
def set_click_through(self, enabled: bool) -> None:
"""When on, the pet ignores the mouse entirely (tray-only control) —
for when it's parked over something you need to click a lot."""
self._click_through = enabled
self.setAttribute(Qt.WA_TransparentForMouseEvents, enabled)
if enabled:
self.clearMask()
self._mask_key = None
else:
self._mask_key = None # force the shaped mask to be rebuilt
@property
def click_through(self) -> bool:
return self._click_through
def _apply_input_mask(self, pixmap: Optional[QPixmap], x: int, y: int) -> None:
"""Restrict the window to the sprite's opaque pixels, so the square
window's transparent corners stop swallowing clicks meant for what's
underneath. Rebuilt only when the frame actually changes — the mask
is derived from the pixmap's alpha, which isn't free."""
if self._click_through or not config.PET_SHAPED_INPUT:
return
if pixmap is None:
if self._mask_key is not None:
self.clearMask()
self._mask_key = None
return
key = (pixmap.cacheKey(), x, y)
if key == self._mask_key:
return
self._mask_key = key
try:
region = QRegion(pixmap.mask())
region.translate(x, y)
self.setMask(region)
except Exception:
self.clearMask() # a sprite without an alpha channel — never mind
# ── wandering ────────────────────────────────────────────────────────
def set_wander_enabled(self, enabled: bool) -> None:
self._wander_enabled = enabled
if not enabled:
self._stop_walking()
def wander_now(self) -> None:
"""Stroll immediately (tray menu / anything that wants a nudge)."""
self._next_wander_at = 0.0
# ── monitors ─────────────────────────────────────────────────────────
def _build_monitors(self) -> list[Monitor]:
"""Snapshot Qt's screen list as plain dataclasses.
Full `geometry()`, not `availableGeometry()`: these coordinates are
what a screen grab gets cropped to, and a grab doesn't stop at the
taskbar. Placement uses availableGeometry separately.
"""
primary = QApplication.primaryScreen()
out = []
for index, screen in enumerate(QApplication.screens()):
geo = screen.geometry()
out.append(
Monitor(
index=index,
name=screen.name() or f"screen-{index + 1}",
x=geo.x(),
y=geo.y(),
width=geo.width(),
height=geo.height(),
primary=screen is primary,
)
)
return out
def publish_monitors(self, force: bool = True) -> None:
"""Push the current layout to whoever's listening (the controller).
*force* re-emits even when nothing changed, which is what the initial
wiring in ui/app.py needs: this window is built before the controller
exists, so the constructor's first publish goes to nobody.
"""
monitors = self._build_monitors()
changed = monitors != self._monitors
self._monitors = monitors
if changed or force:
self.monitors_changed.emit(monitors)
self._publish_pet_monitor(force=True)
def monitors(self) -> list[Monitor]:
return list(self._monitors)
def current_monitor_index(self) -> Optional[int]:
center = self.frameGeometry().center()
screens = QApplication.screens()
if not screens:
return None
screen = QApplication.screenAt(center)
if screen is not None:
try:
return screens.index(screen)
except ValueError:
pass
# Straddling a gap or dragged off the desktop entirely — fall back to
# whichever screen centre is nearest rather than reporting nothing.
best = min(
range(len(screens)),
key=lambda i: (screens[i].geometry().center() - center).manhattanLength(),
)
return best
def _publish_pet_monitor(self, force: bool = False) -> None:
index = self.current_monitor_index()
if index is None:
return
if force or index != self._pet_monitor:
self._pet_monitor = index
self.pet_monitor_changed.emit(index)
def jump_to_monitor(self, index: int) -> None:
"""Teleport to *index* (0-based, resolved by the controller) and land
with a hop. Instant rather than a stroll — Bolt asked to *jump*, and
walking between screens would take the long way across the desktop."""
screens = QApplication.screens()
if not (0 <= index < len(screens)):
return
geo = screens[index].availableGeometry()
point = self._clamp_to_screen(
QPoint(
geo.left() + (geo.width() - self.width()) // 2,
geo.top() + (geo.height() - self.height()) // 2,
),
geo,
)
self._stop_walking() # drop any stroll in flight, or it walks straight back
self.move(point)
self._schedule_next_wander()
self._publish_pet_monitor(force=True)
self.start_emote("hop")
self.update()
def _screen_geometry(self):
# screenAt() so a multi-monitor setup keeps the pet on the screen
# it's currently standing on rather than yanking it to the primary.
screen = QApplication.screenAt(self.frameGeometry().center()) or QApplication.primaryScreen()
return screen.availableGeometry() if screen else None
def _schedule_next_wander(self) -> None:
base = max(1.0, config.PET_WANDER_INTERVAL_SECONDS)
self._next_wander_at = time.monotonic() + random.uniform(0.5 * base, 1.5 * base)
def _stop_walking(self) -> None:
if self._wander_target is None and not self._bob_offset and not self._walking:
return
self._wander_target = None
self._commanded_move = False
self._bob_phase = 0.0
self._bob_offset = 0
self._walking = False
self._walk_distance = 0.0
# Back to a standing frame, so the next stroll starts from a contact
# pose instead of mid-stride.
self.sprites.get(WALK).reset()
self.update()
def snap_to_edge(self) -> bool:
"""If the pet has come to rest near a screen edge, tuck it flush
against it — a desktop pet parked 11px off the taskbar looks like a
bug. Returns True if it moved."""
geo = self._screen_geometry()
if geo is None or not config.PET_EDGE_SNAP:
return False
margin = config.PET_SNAP_MARGIN
here = self.pos()
x, y = here.x(), here.y()
if abs(x - geo.left()) <= margin:
x = geo.left()
elif abs(geo.right() - (x + self.width())) <= margin:
x = geo.right() - self.width() + 1
if abs(y - geo.top()) <= margin:
y = geo.top()
elif abs(geo.bottom() - (y + self.height())) <= margin:
y = geo.bottom() - self.height() + 1
if (x, y) == (here.x(), here.y()):
return False
self.move(x, y)
self._reposition_bubble()
return True
def _pick_wander_target(self) -> Optional[QPoint]:
geo = self._screen_geometry()
if geo is None:
return None
margin = config.PET_WANDER_MARGIN
min_x, max_x = geo.left() + margin, geo.right() - self.width() - margin
min_y, max_y = geo.top() + margin, geo.bottom() - self.height() - margin
if max_x <= min_x or max_y <= min_y: # pet bigger than the screen
return None
here = self.pos()
target = QPoint(random.randint(min_x, max_x), random.randint(min_y, max_y))
dx, dy = target.x() - here.x(), target.y() - here.y()
distance = math.hypot(dx, dy)
limit = max(1.0, config.PET_WANDER_MAX_DISTANCE)
if distance > limit: # shorten the trip rather than sprinting the diagonal
scale = limit / distance
target = QPoint(round(here.x() + dx * scale), round(here.y() + dy * scale))
elif distance < 8: # already there — not worth a stroll
return None
return target
def _movement_tick(self) -> None:
"""One timer, two jobs — emotes play whatever the pet is doing, while
wandering only happens when it's otherwise unoccupied."""
self._advance_emote()
self._wander_tick()
# Report crossing a screen boundary — by strolling, by being dragged,
# by anything. Guarded on the position actually changing so the common
# case (a stationary pet, 30x a second) costs one comparison.
position = self.pos()
if position != self._last_published_pos:
self._last_published_pos = position
self._publish_pet_monitor()
def _wander_tick(self) -> None:
# Only stroll while genuinely idle: not mid-drag, not napping, not
# talking/listening, and not while a speech bubble is up (it would walk
# out from under it). A commanded `petctl move` ignores all of that
# except the drag — if Bolt says go, it goes.
busy = (
not self._wander_enabled
or self._napping
or self._current_state != PetState.IDLE
or self._bubble.isVisible()
)
if self._drag_offset is not None or (busy and not self._commanded_move):
self._stop_walking()
self._schedule_next_wander() # settle first, then wander
return
if self._wander_target is None:
if time.monotonic() < self._next_wander_at:
return
self._wander_target = self._pick_wander_target()
if self._wander_target is None:
self._schedule_next_wander()
return
here = self.pos()
dx = self._wander_target.x() - here.x()
dy = self._wander_target.y() - here.y()
distance = math.hypot(dx, dy)
step = max(1.0, config.PET_WANDER_SPEED * _WANDER_TICK_MS / 1000.0)
if distance <= step:
self.move(self._wander_target)
self._stop_walking()
self._schedule_next_wander()
self.snap_to_edge()
else:
self.move(round(here.x() + dx / distance * step), round(here.y() + dy / distance * step))
self._advance_walk(dx, dy, step)
self.update()
self._reposition_bubble()
def _advance_walk(self, dx: float, dy: float, step: float) -> None:
"""Drive the walk cycle from distance travelled (see the constant).
Falls back to the old bob-in-code if there's no walk art, so a sprite
folder without a walk/ directory still looks like it's moving rather
than sliding perfectly flat.
"""
self._walking = True
# Only turn on meaningful horizontal travel: a near-vertical stroll
# would otherwise flip him back and forth on rounding noise.
if abs(dx) > 1.0:
self._facing = 1 if dx > 0 else -1
if not self.sprites.has(WALK):
self._bob_phase += 0.45
self._bob_offset = int(round(-2.5 * abs(math.sin(self._bob_phase))))
return
self._bob_offset = 0 # the walk frames carry their own weight shift
self._walk_distance += step
while self._walk_distance >= _WALK_PIXELS_PER_FRAME:
self._walk_distance -= _WALK_PIXELS_PER_FRAME
self.sprites.get(WALK).advance()
def _animation_key(self):
"""Walking overrides the state animation — but only while genuinely
idle-and-moving, so he doesn't trot on the spot mid-sentence."""
if self._walking and self.sprites.has(WALK):
return WALK
return self._current_state
def _oriented(self, pixmap: Optional[QPixmap]) -> Optional[QPixmap]:
"""Mirror the (right-facing) walk art when he's heading left. Cached
per source frame — flipping on every paint would be wasteful at 30fps."""
if pixmap is None or self._facing >= 0:
return pixmap
key = pixmap.cacheKey()
mirrored = self._mirror_cache.get(key)
if mirrored is None:
mirrored = pixmap.transformed(QTransform().scale(-1, 1), Qt.SmoothTransformation)
self._mirror_cache[key] = mirrored
return mirrored
# ── state / speech ──────────────────────────────────────────────────
def set_state(self, state: PetState) -> None:
if state == self._current_state:
return
self._current_state = state
self.sprites.get(state).reset()
if state != PetState.IDLE and not self._commanded_move:
# Stand still while listening/thinking/talking — but not if Bolt
# just told it to walk somewhere: that command arrives mid-turn,
# and the reply (-> TALKING) lands a moment later.
self._stop_walking()
self.update()
def say(self, text: str, duration_ms: int = 6000) -> None:
self._reposition_bubble()
self._bubble.show_text(text, duration_ms)
def closeEvent(self, event) -> None:
self._bubble.close()
super().closeEvent(event)
# ── animation ────────────────────────────────────────────────────────
def set_mouth(self, level: float) -> None:
"""How loud the pet is *right now* (0..1), straight off the PCM going
to the speakers (audio/tts.level_of).
The talking frames are ordered by mouth openness, so this indexes them
directly: the mouth moves with the actual waveform instead of flapping
on a timer, which is the difference between a talking sprite and a
dubbed one."""
self._mouth_level = max(0.0, min(1.0, float(level)))
self._mouth_at = time.monotonic()
if self._current_state == PetState.TALKING:
self.update()
def _mouth_frame(self, animation) -> Optional[QPixmap]:
"""The frame matching the current loudness, or None to use the timer.
Falls back the moment the levels go stale — offline TTS has no
envelope, and a mouth frozen mid-syllable is worse than a timed loop."""
if self._mouth_level is None or animation is None:
return None
if time.monotonic() - self._mouth_at > _MOUTH_STALE_SECONDS:
return None
frames = animation.frames
if len(frames) < 2:
return None
index = int(round(self._mouth_level * (len(frames) - 1)))
return frames[max(0, min(len(frames) - 1, index))]
def _advance_frame(self) -> None:
# While walking the cycle is stepped by _advance_walk from distance
# travelled; letting this timer also advance it would double-step it
# and put the feet out of sync with the movement.
if self._walking and self.sprites.has(WALK):
return
self.sprites.get(self._current_state).advance()
self.update()
def paintEvent(self, _event) -> None:
painter = QPainter(self)
painter.setRenderHint(QPainter.Antialiasing)
painter.setRenderHint(QPainter.SmoothPixmapTransform)
key = self._animation_key()
animation = self.sprites.get(key)
pixmap: Optional[QPixmap] = None
if key == PetState.TALKING:
pixmap = self._mouth_frame(animation)
if pixmap is None:
pixmap = animation.current()
if key == WALK:
pixmap = self._oriented(pixmap)
if pixmap is None:
self._apply_input_mask(None, 0, 0)
return
# Non-square source art (e.g. the Kenney robot sprites) keeps its
# aspect ratio when scaled in SpriteSet, so it may be narrower or
# shorter than the (square) window — center it either way.
x = (self.width() - pixmap.width()) // 2
y = (self.height() - pixmap.height()) // 2 + self._bob_offset
# The input mask tracks the resting position, not the emote/bob
# offset: rebuilding it every frame of a spin would be both expensive
# and visibly janky, and the offsets are only a few pixels.
self._apply_input_mask(pixmap, x, (self.height() - pixmap.height()) // 2)
dx, dy, angle, scale = self._emote_transform()
if (dx, dy, angle, scale) == (0.0, 0.0, 0.0, 1.0):
painter.drawPixmap(x, y, pixmap)
return
# Rotate/scale about the sprite's own center so a spin doesn't orbit
# the window's corner.
center_x = x + pixmap.width() / 2
center_y = y + pixmap.height() / 2
transform = QTransform()
transform.translate(center_x + dx, center_y + dy)
transform.rotate(angle)
transform.scale(scale, scale)
transform.translate(-pixmap.width() / 2, -pixmap.height() / 2)
painter.setTransform(transform)
painter.drawPixmap(0, 0, pixmap)
# ── drag / click-to-talk ─────────────────────────────────────────────
def mousePressEvent(self, event) -> None:
if event.button() == Qt.LeftButton:
global_pos = event.globalPosition().toPoint()
self._drag_offset = global_pos - self.frameGeometry().topLeft()
self._press_pos = global_pos
self._dragged = False
self._stop_walking() # grabbing it interrupts a stroll at once
def mouseMoveEvent(self, event) -> None:
if self._drag_offset is None:
return
global_pos = event.globalPosition().toPoint()
self.move(global_pos - self._drag_offset)
self._reposition_bubble()
if (global_pos - self._press_pos).manhattanLength() > _DRAG_THRESHOLD_PX:
self._dragged = True
def mouseReleaseEvent(self, event) -> None:
if event.button() != Qt.LeftButton:
return
was_click = not self._dragged
self._drag_offset = None
self._press_pos = None
if not was_click and config.PET_REMEMBER_POSITION:
position = self.geometry().topLeft()
window_state.save(position.x(), position.y())
if was_click:
self.talk_requested.emit()
else:
self.snap_to_edge() # dropped near an edge -> tuck it flush
bolt_pet/ui/sprite.py
-140
View File
@@ -1,140 +0,0 @@
"""Sprite loading + frame animation.
Convention: assets/sprites/<state>/*.png, frames played in filename-sorted
order (e.g. frame_00.png, frame_01.png, ...), looping. <state> matches
bolt_pet.state.PetState values: idle, listening, thinking, talking.
If a state's directory has no frames (real art not dropped in yet), falls
back to a small procedurally-drawn placeholder blob so the app still runs
end-to-end. Swap in real sprite sheets by pointing SPRITE_DIR at your own
folder (see assets/sprites/README.md) — no code changes needed as long as
the same per-state-subfolder-of-PNGs convention is followed. If your sheets
use a different layout (single grid image, etc.), tell me the format and
this loader can be adapted.
"""
from __future__ import annotations
from pathlib import Path
from typing import Optional
from PySide6.QtCore import QSize, Qt
from PySide6.QtGui import QColor, QPainter, QPixmap
from ..state import PetState
DEFAULT_SPRITE_DIR = Path(__file__).resolve().parent.parent / "assets" / "sprites"
# Placeholder palette per state, used only when no frames are found.
_PLACEHOLDER_COLORS = {
PetState.IDLE: QColor(120, 170, 240),
PetState.LISTENING: QColor(120, 220, 160),
PetState.THINKING: QColor(230, 190, 90),
PetState.TALKING: QColor(240, 130, 150),
PetState.ERROR: QColor(220, 90, 90),
}
def _placeholder_frames(state: PetState, size: int) -> list[QPixmap]:
"""A tiny 2-frame "breathing" blob so idle/listening/etc. are visually
distinguishable even before real art exists."""
color = _PLACEHOLDER_COLORS.get(state, QColor(150, 150, 150))
frames = []
for scale in (1.0, 0.92):
pixmap = QPixmap(size, size)
pixmap.fill(Qt.transparent)
painter = QPainter(pixmap)
painter.setRenderHint(QPainter.Antialiasing)
painter.setBrush(color)
painter.setPen(Qt.NoPen)
margin = size * (1 - scale) / 2
painter.drawEllipse(int(margin), int(margin), int(size * scale), int(size * scale))
# simple eyes so it reads as a face, not just a circle
eye_r = max(2, size // 16)
eye_y = int(size * 0.42)
painter.setBrush(QColor(30, 30, 40))
painter.drawEllipse(int(size * 0.36) - eye_r, eye_y - eye_r, eye_r * 2, eye_r * 2)
painter.drawEllipse(int(size * 0.64) - eye_r, eye_y - eye_r, eye_r * 2, eye_r * 2)
painter.end()
frames.append(pixmap)
return frames
class SpriteAnimation:
"""One state's frame sequence + current playback position."""
def __init__(self, frames: list[QPixmap]):
self.frames = frames or []
self._index = 0
def advance(self) -> None:
if self.frames:
self._index = (self._index + 1) % len(self.frames)
def current(self) -> Optional[QPixmap]:
if not self.frames:
return None
return self.frames[self._index]
def reset(self) -> None:
self._index = 0
def _load_frames_from_dir(directory: Path, size: int) -> list[QPixmap]:
if not directory.is_dir():
return []
paths = sorted(directory.glob("*.png")) + sorted(directory.glob("*.PNG"))
frames = []
for path in paths:
pixmap = QPixmap(str(path))
if pixmap.isNull():
continue
if pixmap.size() != QSize(size, size):
pixmap = pixmap.scaled(size, size, Qt.KeepAspectRatio, Qt.SmoothTransformation)
frames.append(pixmap)
return frames
WALK = "walk"
# Animations that aren't pipeline states. Walking is a property of *movement*,
# orthogonal to whether the pet is idle/listening/talking, so it deliberately
# isn't a PetState — state.py stays a description of the conversation, not of
# the body. Loaded the same way, keyed by name.
EXTRA_ANIMATIONS = (WALK,)
class SpriteSet:
"""All animations for every PetState, plus the extras, from *sprite_dir*."""
def __init__(self, sprite_dir: Path = DEFAULT_SPRITE_DIR, size: int = 160):
self.size = size
self._animations: dict[str, SpriteAnimation] = {}
self._loaded: set[str] = set() # keys backed by real art, not placeholders
for state in PetState:
frames = _load_frames_from_dir(sprite_dir / state.value, size)
if frames:
self._loaded.add(state.value)
else:
frames = _placeholder_frames(state, size)
self._animations[state.value] = SpriteAnimation(frames)
for name in EXTRA_ANIMATIONS:
frames = _load_frames_from_dir(sprite_dir / name, size)
if frames:
self._loaded.add(name)
self._animations[name] = SpriteAnimation(frames)
@staticmethod
def _key(key) -> str:
return key.value if isinstance(key, PetState) else str(key)
def get(self, key) -> SpriteAnimation:
"""Animation for a PetState or an extra name. Unknown/absent extras
fall back to idle, so a sprite folder with no walk/ still runs."""
return self._animations.get(self._key(key)) or self._animations[PetState.IDLE.value]
def has(self, key) -> bool:
"""True only when real frames were found — the caller uses this to
decide whether to use an extra animation at all, rather than being
handed a placeholder blob that looks nothing like walking."""
return self._key(key) in self._loaded
bolt_pet/ui/tray.py
-161
View File
@@ -1,161 +0,0 @@
"""System tray icon — the pet window is frameless with no taskbar entry, so
this menu is the only always-available way to control or exit it: talk now,
mute, wander, click-through, nap, history, wake-word tuning, voice reset,
quit.
Every entry is a plain callback passed in by ui/app.py; this file knows
nothing about the controller or the pet window.
"""
from __future__ import annotations
from typing import Callable, Optional
from PySide6.QtCore import Qt
from PySide6.QtGui import QAction, QColor, QIcon, QPainter, QPixmap
from PySide6.QtWidgets import QMenu, QSystemTrayIcon
def _make_icon(muted: bool, napping: bool = False) -> QIcon:
pixmap = QPixmap(32, 32)
pixmap.fill(Qt.transparent)
painter = QPainter(pixmap)
painter.setRenderHint(QPainter.Antialiasing)
if muted:
color = QColor(200, 60, 60)
elif napping:
color = QColor(120, 120, 140)
else:
color = QColor(120, 170, 240)
painter.setBrush(color)
painter.setPen(Qt.NoPen)
painter.drawEllipse(2, 2, 28, 28)
painter.end()
return QIcon(pixmap)
class PetTray(QSystemTrayIcon):
def __init__(
self,
on_talk_now: Callable[[], None],
on_toggle_mute: Callable[[], bool],
on_quit: Callable[[], None],
on_set_wander: Optional[Callable[[bool], None]] = None,
wander_enabled: bool = True,
on_set_click_through: Optional[Callable[[bool], None]] = None,
click_through_enabled: bool = False,
on_set_nap: Optional[Callable[[bool], None]] = None,
on_show_history: Optional[Callable[[], None]] = None,
on_show_wake_tuner: Optional[Callable[[], None]] = None,
on_show_mic_picker: Optional[Callable[[], None]] = None,
on_reset_voice: Optional[Callable[[], None]] = None,
parent=None,
):
super().__init__(_make_icon(muted=False), parent)
self._on_toggle_mute = on_toggle_mute
self._muted = False
self._napping = False
self.setToolTip("Bolt")
menu = QMenu()
self._talk_action = QAction("Talk now", menu)
self._talk_action.triggered.connect(on_talk_now)
menu.addAction(self._talk_action)
self._mute_action = QAction("Mute mic", menu)
self._mute_action.setCheckable(True)
self._mute_action.triggered.connect(self._handle_toggle_mute)
menu.addAction(self._mute_action)
self._nap_action = None
if on_set_nap is not None:
self._nap_action = QAction("Nap (no proactive noise)", menu)
self._nap_action.setCheckable(True)
self._nap_action.triggered.connect(lambda checked: on_set_nap(checked))
menu.addAction(self._nap_action)
menu.addSeparator()
if on_set_wander is not None:
self._wander_action = QAction("Wander around", menu)
self._wander_action.setCheckable(True)
self._wander_action.setChecked(wander_enabled)
self._wander_action.triggered.connect(lambda checked: on_set_wander(checked))
menu.addAction(self._wander_action)
if on_set_click_through is not None:
self._click_through_action = QAction("Click through the pet", menu)
self._click_through_action.setCheckable(True)
self._click_through_action.setChecked(click_through_enabled)
self._click_through_action.setToolTip(
"Ignore the mouse entirely — control it from this menu instead."
)
self._click_through_action.triggered.connect(lambda checked: on_set_click_through(checked))
menu.addAction(self._click_through_action)
menu.addSeparator()
if on_show_history is not None:
history_action = QAction("History…", menu)
history_action.triggered.connect(on_show_history)
menu.addAction(history_action)
if on_show_wake_tuner is not None:
tuner_action = QAction("Wake word tuning…", menu)
tuner_action.triggered.connect(on_show_wake_tuner)
menu.addAction(tuner_action)
if on_show_mic_picker is not None:
mic_action = QAction("Select microphone…", menu)
mic_action.triggered.connect(on_show_mic_picker)
menu.addAction(mic_action)
# Only ever enabled while a server-picked voice (speak_as) is in use —
# it's the way back from "talk like a pirate", which nothing else
# undoes short of a restart.
self._voice_action = None
if on_reset_voice is not None:
self._voice_action = QAction("Use default voice", menu)
self._voice_action.setEnabled(False)
self._voice_action.triggered.connect(on_reset_voice)
menu.addAction(self._voice_action)
menu.addSeparator()
quit_action = QAction("Quit", menu)
quit_action.triggered.connect(on_quit)
menu.addAction(quit_action)
self.setContextMenu(menu)
self.show()
def _handle_toggle_mute(self) -> None:
self._muted = self._on_toggle_mute()
self._mute_action.setChecked(self._muted)
self._refresh_icon()
def set_voice(self, voice: str) -> None:
"""Reflect the voice the controller is speaking in — a name (or id)
when the server picked one, "" for Bolt's own."""
if self._voice_action is None:
return
self._voice_action.setEnabled(bool(voice))
self._voice_action.setText(
f"Use default voice (now: {voice})" if voice else "Use default voice"
)
def set_napping(self, napping: bool) -> None:
"""Reflect a nap the *controller* decided on (quiet hours, fullscreen,
or a petctl command) — not just ones clicked here."""
self._napping = napping
if self._nap_action is not None:
self._nap_action.setChecked(napping)
self._refresh_icon()
def _refresh_icon(self) -> None:
self.setIcon(_make_icon(self._muted, self._napping))
if self._muted:
self.setToolTip("Bolt (muted)")
elif self._napping:
self.setToolTip("Bolt (napping)")
else:
self.setToolTip("Bolt")
bolt_pet/ui/wake_tuner.py
-118
View File
@@ -1,118 +0,0 @@
"""Wake-word sensitivity tuner.
WAKE_WORD_THRESHOLD is otherwise a number you guess at in .env, restart, and
then test by saying "thunderbolt" at your computer repeatedly. This window
makes it evidence-based: a live peak-score readout while you talk, a rolling
list of near misses (frames that scored just under the threshold — i.e. the
times it *nearly* heard you), and a slider that takes effect immediately,
mid-listen, without a restart.
The controller owns the threshold; this window is a view over it.
"""
from __future__ import annotations
import time
from typing import Callable
from PySide6.QtCore import Qt, QTimer
from PySide6.QtWidgets import (
QDialog, QHBoxLayout, QLabel, QListWidget, QPushButton, QSlider, QVBoxLayout,
)
_SLIDER_SCALE = 100 # QSlider is integer-only; threshold is 0.00-1.00
class WakeTunerWindow(QDialog):
def __init__(
self,
get_threshold: Callable[[], float],
set_threshold: Callable[[float], None],
get_stats: Callable[[], dict],
on_reset: Callable[[], None],
):
super().__init__()
self._get_threshold = get_threshold
self._set_threshold = set_threshold
self._get_stats = get_stats
self._on_reset = on_reset
self.setWindowTitle("Bolt — wake word tuning")
self.resize(460, 380)
self._threshold_label = QLabel()
self._slider = QSlider(Qt.Horizontal)
self._slider.setRange(5, 99)
self._slider.setValue(int(round(get_threshold() * _SLIDER_SCALE)))
self._slider.valueChanged.connect(self._threshold_changed)
self._peak_label = QLabel("Peak score since reset: —")
self._peak_label.setToolTip(
'Say "thunderbolt" a few times and watch this. Set the threshold '
"just below the peak you can hit reliably."
)
self._misses = QListWidget()
reset_button = QPushButton("Reset stats")
reset_button.clicked.connect(self._reset)
close_button = QPushButton("Close")
close_button.clicked.connect(self.close)
close_button.setDefault(True)
buttons = QHBoxLayout()
buttons.addWidget(reset_button)
buttons.addStretch(1)
buttons.addWidget(close_button)
layout = QVBoxLayout(self)
layout.addWidget(self._threshold_label)
layout.addWidget(self._slider)
layout.addWidget(self._peak_label)
layout.addWidget(QLabel("Near misses (heard something, didn't quite fire):"))
layout.addWidget(self._misses)
layout.addLayout(buttons)
# Polled rather than signal-driven: scores arrive ~12x/second on the
# audio thread, and a queued signal per frame to repaint a label is
# more traffic than this is worth.
self._timer = QTimer(self)
self._timer.timeout.connect(self.refresh)
self._update_threshold_label()
def _threshold_changed(self, value: int) -> None:
self._set_threshold(value / _SLIDER_SCALE)
self._update_threshold_label()
def _update_threshold_label(self) -> None:
threshold = self._slider.value() / _SLIDER_SCALE
self._threshold_label.setText(
f"Threshold: {threshold:.2f} (lower = more sensitive, more false triggers)"
)
def _reset(self) -> None:
self._on_reset()
self.refresh()
def refresh(self) -> None:
stats = self._get_stats() or {}
peak = stats.get("peak", 0.0)
self._peak_label.setText(f"Peak score since reset: {peak:.3f}")
self._misses.clear()
for timestamp, score, threshold in reversed(stats.get("near_misses", [])):
when = time.strftime("%H:%M:%S", time.localtime(timestamp))
self._misses.addItem(f"{when} scored {score:.3f} (threshold {threshold:.2f})")
if self._misses.count() == 0:
self._misses.addItem("Nothing yet — say the wake phrase a few times.")
def show_refreshed(self) -> None:
self._slider.setValue(int(round(self._get_threshold() * _SLIDER_SCALE)))
self.refresh()
self.show()
self.raise_()
self.activateWindow()
self._timer.start(500)
def closeEvent(self, event) -> None:
self._timer.stop()
super().closeEvent(event)
bolt_pet/updater.py
-323
View File
@@ -1,323 +0,0 @@
"""Self-update from the Gitea releases page.
Polls `<UPDATE_REPO_API>/releases/latest` for a tag newer than
``bolt_pet.__version__`` and, if there is one, moves the checkout to that tag
and restarts the pet. The install is expected to be a git clone (which is how
it's deployed), so "download the update" is just `git fetch` + `git checkout`
— atomic, and the previous ref is one command away if anything goes wrong.
Safety rules, in the order they're enforced:
1. **A dirty working tree is never touched.** Local edits are skipped over,
not stashed — the pet silently discarding your work-in-progress would be
far worse than running an old version.
2. **Everything after checkout is guarded.** Dependency install and an import
smoke test both run before the restart; if either fails, the checkout is
rolled back to the exact ref that was live before (branch name if we were
on one, otherwise the commit) and the deps reinstalled from it.
3. **The restart only happens once the new code imports.** So a broken
release costs you a rollback and a log line, not a pet that won't start.
The git side goes through an injectable *run* callable — ``(args) ->
(returncode, output)`` — so the whole apply/rollback dance is unit-tested
against a fake git rather than a real repo. Version comparison and release
parsing are pure functions for the same reason.
"""
from __future__ import annotations
import os
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
from typing import Callable, Optional, Tuple
import requests
from . import config
# (returncode, combined stdout+stderr)
GitResult = Tuple[int, str]
GitRunner = Callable[[list], GitResult]
_GIT_TIMEOUT_SECONDS = 300
class UpdateError(Exception):
"""Raised when an update can't be applied. If it's raised *after* the
checkout moved, the rollback has already run."""
@dataclass(frozen=True)
class Release:
tag: str
name: str
body: str
prerelease: bool
# ── pure logic ───────────────────────────────────────────────────────────────
def parse_version(tag: str) -> tuple:
"""``"v1.2.3"`` -> ``(1, 2, 3)``. Leading "v" optional; a trailing
suffix ends the parse (``"1.2.3-beta1"`` -> ``(1, 2, 3)``), so a
prerelease of a version compares equal to it rather than sorting
randomly. Junk parses to ``()``, which is never newer than anything."""
parts: list[int] = []
for chunk in (tag or "").strip().lstrip("vV").split("."):
digits = ""
for char in chunk:
if not char.isdigit():
break
digits += char
if not digits:
break
parts.append(int(digits))
return tuple(parts)
def is_newer(candidate: str, current: str) -> bool:
"""True if *candidate* is a strictly newer version than *current*.
Compares zero-padded, so 1.2 == 1.2.0 and 1.2.1 > 1.2."""
new, old = parse_version(candidate), parse_version(current)
if not new:
return False
width = max(len(new), len(old))
return new + (0,) * (width - len(new)) > old + (0,) * (width - len(old))
def release_from_payload(payload: dict) -> Optional[Release]:
"""Gitea's release JSON -> Release, or None if it's a draft or has no
tag. /releases/latest already excludes drafts and prereleases, but the
same parser is used for the full list."""
if not isinstance(payload, dict) or payload.get("draft"):
return None
tag = str(payload.get("tag_name") or "").strip()
if not tag:
return None
return Release(
tag=tag,
name=str(payload.get("name") or tag),
body=str(payload.get("body") or ""),
prerelease=bool(payload.get("prerelease")),
)
# ── talking to Gitea ─────────────────────────────────────────────────────────
def fetch_latest_release(api_url: str = None, token: str = None, timeout: float = 15.0) -> Optional[Release]:
"""Newest published release, or None if the repo has no releases yet
(a fresh repo 404s here, which is not an error worth logging every hour).
Raises UpdateError if the server is unreachable or answers with junk."""
api_url = (api_url if api_url is not None else config.UPDATE_REPO_API).rstrip("/")
if not api_url:
raise UpdateError("UPDATE_REPO_API is not set")
token = config.UPDATE_TOKEN if token is None else token
headers = {"Authorization": f"token {token}"} if token else {}
try:
response = requests.get(f"{api_url}/releases/latest", headers=headers, timeout=timeout)
except Exception as exc:
raise UpdateError(f"couldn't reach the releases API: {exc}") from exc
if response.status_code == 404:
return None
try:
response.raise_for_status()
payload = response.json()
except Exception as exc:
raise UpdateError(f"bad response from the releases API: {exc}") from exc
return release_from_payload(payload)
def check_for_update(current_version: str = None, **kwargs) -> Optional[Release]:
"""The whole "is there anything new?" question in one call. Returns the
Release to move to, or None if we're already current."""
from . import __version__
current = __version__ if current_version is None else current_version
release = fetch_latest_release(**kwargs)
if release is None or not is_newer(release.tag, current):
return None
return release
# ── git ──────────────────────────────────────────────────────────────────────
def git_runner(repo: Path = None) -> GitRunner:
repo = Path(repo or config.HERE)
def run(args: list) -> GitResult:
try:
completed = subprocess.run(
["git", *args], cwd=str(repo), capture_output=True,
text=True, timeout=_GIT_TIMEOUT_SECONDS,
)
except Exception as exc:
return 1, f"git {' '.join(args)} failed to start: {exc}"
return completed.returncode, ((completed.stdout or "") + (completed.stderr or "")).strip()
return run
def is_git_clone(run: GitRunner) -> bool:
return run(["rev-parse", "--git-dir"])[0] == 0
def working_tree_dirty(run: GitRunner) -> bool:
code, output = run(["status", "--porcelain"])
return code != 0 or bool(output.strip())
def current_ref(run: GitRunner) -> str:
"""The branch name if we're on one, else the commit SHA — i.e. whatever
`git checkout` needs to put things back exactly as they were."""
code, output = run(["symbolic-ref", "--quiet", "--short", "HEAD"])
if code == 0 and output.strip():
return output.strip()
code, output = run(["rev-parse", "HEAD"])
if code != 0 or not output.strip():
raise UpdateError("couldn't work out the current git ref")
return output.strip()
def already_at_tag(run: GitRunner, tag: str) -> bool:
"""True if HEAD is already the commit *tag* points at.
Guards the loop you get when a release is cut without bumping
__version__ in the tagged commit: the checkout succeeds (it's a no-op),
the pet restarts, reads the same old __version__, sees the same "newer"
tag, and does it again — a restart every check, forever."""
code, head = run(["rev-parse", "HEAD"])
if code != 0 or not head.strip():
return False
code, target = run(["rev-parse", f"tags/{tag}^{{commit}}"])
if code != 0 or not target.strip():
return False # tag isn't known locally yet, so we're certainly not on it
return head.strip() == target.strip()
def _requirements_changed(run: GitRunner, before: str, after: str) -> bool:
code, output = run(["diff", "--name-only", before, after, "--", "requirements.txt"])
return code == 0 and bool(output.strip())
def _install_deps(repo: Path) -> None:
completed = subprocess.run(
[sys.executable, "-m", "pip", "install", "-r", "requirements.txt"],
cwd=str(repo), capture_output=True, text=True, timeout=_GIT_TIMEOUT_SECONDS,
)
if completed.returncode != 0:
raise UpdateError(f"pip install failed: {(completed.stderr or '')[-500:]}")
def _smoke_test(repo: Path) -> None:
"""Import the freshly checked-out package in a *subprocess* — this one
still has the old modules loaded, so importing here would prove nothing.
Catches the common broken release (syntax error, missing dependency)
before we hand the session over to it."""
completed = subprocess.run(
[sys.executable, "-c", "import bolt_pet; import bolt_pet.controller"],
cwd=str(repo), capture_output=True, text=True, timeout=120,
)
if completed.returncode != 0:
raise UpdateError(f"the new version failed to import: {(completed.stderr or '')[-500:]}")
def apply_update(
tag: str,
run: GitRunner = None,
repo: Path = None,
on_log: Callable[[str], None] = lambda _msg: None,
install_deps: bool = None,
verify: Callable[[Path], None] = None,
) -> str:
"""Move the checkout to *tag*, rolling back to where it was if anything
downstream of the checkout fails. Returns the ref we came from (handy for
logging / a manual `git checkout` back). Raises UpdateError otherwise."""
repo = Path(repo or config.HERE)
run = run or git_runner(repo)
install_deps = config.UPDATE_INSTALL_DEPS if install_deps is None else install_deps
verify = _smoke_test if verify is None else verify
if not is_git_clone(run):
raise UpdateError("not a git clone — auto-update only works on a git checkout")
if working_tree_dirty(run):
raise UpdateError("working tree has local changes — skipping (nothing was touched)")
previous = current_ref(run)
code, output = run(["fetch", "--tags", "--prune", config.UPDATE_GIT_REMOTE])
if code != 0:
raise UpdateError(f"git fetch failed: {output}")
if already_at_tag(run, tag):
from . import __version__
raise UpdateError(
f"already checked out {tag}, but __version__ still reads {__version__} — "
f"bump it in the tagged commit, or every check re-applies the same release"
)
code, output = run(["checkout", "--force", f"tags/{tag}"])
if code != 0:
raise UpdateError(f"git checkout {tag} failed: {output}")
on_log(f"Checked out {tag} (was {previous}).")
# Past this point every failure has to put the checkout back.
try:
if install_deps and _requirements_changed(run, previous, f"tags/{tag}"):
on_log("requirements.txt changed — installing.")
_install_deps(repo)
verify(repo)
except UpdateError as exc:
_rollback(run, previous, repo, on_log, install_deps)
raise UpdateError(f"{exc} — rolled back to {previous}") from exc
except Exception as exc: # a verify() that blows up is still a failed update
_rollback(run, previous, repo, on_log, install_deps)
raise UpdateError(f"update failed ({exc}) — rolled back to {previous}") from exc
return previous
def _rollback(
run: GitRunner,
previous: str,
repo: Path,
on_log: Callable[[str], None],
install_deps: bool,
) -> None:
"""Best-effort return to *previous*. Never raises — it's already running
inside a failure path, and the caller's UpdateError is the thing worth
surfacing. A rollback that itself fails gets its own loud log line,
because that's the one case needing a human."""
code, output = run(["checkout", "--force", previous])
if code != 0:
on_log(f"ROLLBACK FAILED — the checkout is stranded. Run: git checkout {previous} ({output})")
return
on_log(f"Rolled back to {previous}.")
if install_deps:
try:
_install_deps(repo)
except Exception as exc:
on_log(f"Rolled back, but reinstalling the old requirements failed: {exc}")
# ── restart ──────────────────────────────────────────────────────────────────
def restart() -> None:
"""Replace this process with a fresh `python -m bolt_pet`.
execv rather than spawn-and-exit so there's no window with two pets
holding the same mic, and no orphan if the parent dies first. Never
returns when it works; callers should have shut the Qt app and released
the audio device before calling it.
chdir first because `-m bolt_pet` resolves the package from the working
directory: the pet may well have been launched from somewhere else
(autostart entry, run.sh invoked by path), and the new process has to
land on the checkout the update was just applied to."""
os.chdir(str(config.HERE))
os.execv(sys.executable, [sys.executable, "-m", "bolt_pet"])
bolt_pet/window_state.py
-75
View File
@@ -1,75 +0,0 @@
"""Where the pet was left, so it starts there next time.
Listed in the README as a known limitation: drag it somewhere deliberate, and
the next launch puts it back in the bottom-right corner. For something that
lives on your desktop all day that is a small daily annoyance, and it is a
config write on drag-end.
Lives in the cache dir rather than the repo, next to the restart context: it
is per-machine state about this install, not something that belongs in a git
diff. Every function swallows its own errors — a corrupt or unwritable state
file must never stop the pet from starting, it just means the default corner.
Positions are validated against the *current* screen layout on load, because
the common case for a stale position is exactly the case where it is
dangerous: the pet was last on a monitor that is now unplugged, and restoring
it faithfully would put it somewhere you cannot see or reach.
"""
from __future__ import annotations
import json
import logging
import os
import tempfile
from pathlib import Path
from typing import Optional
logger = logging.getLogger("bolt_pet.window_state")
DEFAULT_PATH = Path.home() / ".cache" / "bolt-pet" / "window.json"
def load(path: Optional[Path] = None) -> Optional[tuple[int, int]]:
"""The saved position, or None if there isn't a usable one."""
try:
data = json.loads(Path(path or DEFAULT_PATH).read_text(encoding="utf-8"))
return int(data["x"]), int(data["y"])
except (FileNotFoundError, json.JSONDecodeError, KeyError, TypeError, ValueError, OSError):
return None
def save(x: int, y: int, path: Optional[Path] = None) -> None:
"""Remember where it is now. Atomic, so a crash mid-write can't leave a
half-file that makes the next start fall back to the corner."""
target = Path(path or DEFAULT_PATH)
try:
target.parent.mkdir(parents=True, exist_ok=True)
descriptor, temp_path = tempfile.mkstemp(dir=target.parent, prefix=".window_", suffix=".tmp")
try:
with os.fdopen(descriptor, "w", encoding="utf-8") as handle:
json.dump({"x": int(x), "y": int(y)}, handle)
os.replace(temp_path, target)
except BaseException:
try:
os.unlink(temp_path)
except OSError:
pass
raise
except Exception:
logger.debug("Could not save the window position", exc_info=True)
def is_visible_on(x: int, y: int, size: int, rectangles) -> bool:
"""Whether that position still lands on a screen that exists.
*rectangles* are (left, top, right, bottom) tuples — the caller's job,
because this module has no business importing Qt. Requires a real overlap
rather than a touching edge, so a pet saved flush against the boundary of a
monitor that has since been unplugged is not counted as reachable."""
for left, top, right, bottom in rectangles:
overlap_x = min(x + size, right) - max(x, left)
overlap_y = min(y + size, bottom) - max(y, top)
if overlap_x > size * 0.25 and overlap_y > size * 0.25:
return True
return False
launch.py
+12
View File
@@ -0,0 +1,12 @@
"""Frozen-app launcher.
PyInstaller compiles this as a top-level script, so it must NOT use relative
imports. It imports the package by name and hands off to bolt_console.__main__.
"""
import sys
import bolt_console.__main__ as entry
if __name__ == "__main__":
sys.exit(entry.main())
requirements.txt
+7 -48
View File
@@ -1,54 +1,13 @@
# Desktop UI
# HTTP client to the Bolt desk API.
requests>=2.31
# The window (python -m bolt_console). Not needed for --cli.
PySide6>=6.6
# Audio I/O (mic capture + speaker playback) — cross-platform via PortAudio.
# On Linux you may also need the system package: sudo apt install libportaudio2
# Dictation (the mic button): mic capture + the server's /desk/stt websocket.
# Optional: without them the mic button explains what's missing.
sounddevice>=0.4.6
numpy>=1.24
# HTTP client to the Bolt desk API
requests>=2.31
# Speech-to-text, via the server's /desk/stt websocket relay (audio/stt.py,
# audio/stt_stream.py) — there is no separate REST fallback, so this is now
# load-bearing for transcription to work at all, not just for the
# transcribe-while-talking optimisation STT_STREAMING controls.
websocket-client>=1.7
# Wake-word detection (local, offline after first run) — runs the
# custom-trained thunderbolt.onnx model shipped in this repo, same runtime
# as the main repo's desk_client/bolt_desk.py (bolt.onnx). First use
# downloads openwakeword's feature-extraction sub-models (~few MB, cached
# under the package's own resources/ dir afterward) — needs internet once.
openwakeword
# Offline TTS fallback if the server call isn't configured or fails.
# Uses SAPI5 on Windows, NSSpeechSynthesizer on macOS, espeak on Linux
# (Linux also needs: sudo apt install espeak-ng).
pyttsx3>=2.90
# Global push-to-talk hotkey (PUSH_TO_TALK_HOTKEY). Optional: the pet
# degrades to wake-word + tray + click if it's missing or if the session
# won't allow a global key hook (most Wayland setups; macOS needs
# Accessibility permission).
pynput>=1.7
# Optional — only needed for scripts/slice_spritesheet.py (converting a
# grid sprite sheet into the per-frame-PNG convention sprite.py expects).
# Not imported by the app itself.
Pillow>=10.0
# Screen reading (`petctl read`) — Bolt OCRs a monitor and uses the text in
# his reply. Both optional: without them `petctl read` reports what's missing
# and the rest of the pet is unaffected.
# mss screen capture. X11/Win32/macOS — NOT Wayland.
# pytesseract a thin wrapper; the actual engine is a system package:
# sudo apt install tesseract-ocr
# No-sudo alternative to those two lines: pip install rapidocr-onnxruntime
# (pure pip, reuses the onnxruntime openwakeword already pulls in, slower to
# start). screen_text.resolve_engine() picks whichever is present.
mss>=9.0
pytesseract>=0.3.10
# Test runner (tests/ — pure logic, no audio hardware or display needed;
# run with QT_QPA_PLATFORM=offscreen).
# Test runner (tests/ — no real server or terminal needed).
pytest>=8.0

Some files were not shown because too many files have changed in this diff Show More