themajesticmagician
3a0959f55d
Latency: replies are spoken sentence-by-sentence off the desk API's NDJSON endpoint, so the wait is time-to-first-sentence rather than the whole model call, and Deepgram's live websocket transcribes while you're still talking instead of uploading the WAV afterwards. Both fall back invisibly — a stream that fails before anything was said drops to converse(), and a socket that never opens just means the old one-shot path. Speaking lived in four near-copies in the controller (a reply, a holding line, a streamed sentence, a dialogue scene) that had already drifted: one didn't arm barge-in, another skipped the follow-up rule. It's now speech.Speaker plus an Utterance describing the policy differences, with collaborators injected so the whole of it tests without Qt or audio. The mouth follows the audio rather than a timer: tts.level_of reduces each PCM frame to a 0..1 loudness on a sqrt curve (speech sits well below peak, and a linear map leaves the mouth barely open during normal talking) and that indexes the talking frames, which the sprite script now draws as an openness ramp. Offline pyttsx3 has no waveform, so stale levels hand control back to the timed loop instead of freezing the mouth mid-syllable. Also: the pet starts where you left it (ignoring positions on monitors that are no longer connected, since restoring those faithfully is how it ends up somewhere unreachable), and `python -m bolt_pet --doctor` is a preflight that says what to do about each problem rather than only what's wrong. tests/test_pipeline_smoke.py breaks the pure-logic rule on purpose. Every unit test passed all week while notifications sat unspoken for minutes, the pet said things twice and [laughing] got read aloud — each an interaction between two individually-correct units. It drives whole turns against a real HTTP server on a loopback port, faking only the mic and the speakers. It found a NameError in the paint path that would have fired on every repaint while talking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
53 lines
2.2 KiB
Plaintext
53 lines
2.2 KiB
Plaintext
# Desktop UI
|
|
PySide6>=6.6
|
|
|
|
# Audio I/O (mic capture + speaker playback) — cross-platform via PortAudio.
|
|
# On Linux you may also need the system package: sudo apt install libportaudio2
|
|
sounddevice>=0.4.6
|
|
numpy>=1.24
|
|
|
|
# HTTP client to the Bolt desk API
|
|
requests>=2.31
|
|
# Streaming speech-to-text (audio/stt_stream.py). Optional in practice: without
|
|
# it the pet falls back to uploading the finished clip, exactly as before.
|
|
websocket-client>=1.7
|
|
|
|
# Wake-word detection (local, offline after first run) — runs the
|
|
# custom-trained thunderbolt.onnx model shipped in this repo, same runtime
|
|
# as the main repo's desk_client/bolt_desk.py (bolt.onnx). First use
|
|
# downloads openwakeword's feature-extraction sub-models (~few MB, cached
|
|
# under the package's own resources/ dir afterward) — needs internet once.
|
|
openwakeword
|
|
|
|
# Offline TTS fallback if ElevenLabs isn't configured or a request fails.
|
|
# Uses SAPI5 on Windows, NSSpeechSynthesizer on macOS, espeak on Linux
|
|
# (Linux also needs: sudo apt install espeak-ng).
|
|
pyttsx3>=2.90
|
|
|
|
# Global push-to-talk hotkey (PUSH_TO_TALK_HOTKEY). Optional: the pet
|
|
# degrades to wake-word + tray + click if it's missing or if the session
|
|
# won't allow a global key hook (most Wayland setups; macOS needs
|
|
# Accessibility permission).
|
|
pynput>=1.7
|
|
|
|
# Optional — only needed for scripts/slice_spritesheet.py (converting a
|
|
# grid sprite sheet into the per-frame-PNG convention sprite.py expects).
|
|
# Not imported by the app itself.
|
|
Pillow>=10.0
|
|
|
|
# Screen reading (`petctl read`) — Bolt OCRs a monitor and uses the text in
|
|
# his reply. Both optional: without them `petctl read` reports what's missing
|
|
# and the rest of the pet is unaffected.
|
|
# mss screen capture. X11/Win32/macOS — NOT Wayland.
|
|
# pytesseract a thin wrapper; the actual engine is a system package:
|
|
# sudo apt install tesseract-ocr
|
|
# No-sudo alternative to those two lines: pip install rapidocr-onnxruntime
|
|
# (pure pip, reuses the onnxruntime openwakeword already pulls in, slower to
|
|
# start). screen_text.resolve_engine() picks whichever is present.
|
|
mss>=9.0
|
|
pytesseract>=0.3.10
|
|
|
|
# Test runner (tests/ — pure logic, no audio hardware or display needed;
|
|
# run with QT_QPA_PLATFORM=offscreen).
|
|
pytest>=8.0
|