Streaming replies and STT, amplitude lip-sync, one place for speaking

Latency: replies are spoken sentence-by-sentence off the desk API's NDJSON
endpoint, so the wait is time-to-first-sentence rather than the whole model
call, and Deepgram's live websocket transcribes while you're still talking
instead of uploading the WAV afterwards. Both fall back invisibly — a stream
that fails before anything was said drops to converse(), and a socket that
never opens just means the old one-shot path.

Speaking lived in four near-copies in the controller (a reply, a holding line,
a streamed sentence, a dialogue scene) that had already drifted: one didn't arm
barge-in, another skipped the follow-up rule. It's now speech.Speaker plus an
Utterance describing the policy differences, with collaborators injected so the
whole of it tests without Qt or audio.

The mouth follows the audio rather than a timer: tts.level_of reduces each PCM
frame to a 0..1 loudness on a sqrt curve (speech sits well below peak, and a
linear map leaves the mouth barely open during normal talking) and that indexes
the talking frames, which the sprite script now draws as an openness ramp.
Offline pyttsx3 has no waveform, so stale levels hand control back to the timed
loop instead of freezing the mouth mid-syllable.

Also: the pet starts where you left it (ignoring positions on monitors that are
no longer connected, since restoring those faithfully is how it ends up
somewhere unreachable), and `python -m bolt_pet --doctor` is a preflight that
says what to do about each problem rather than only what's wrong.

tests/test_pipeline_smoke.py breaks the pure-logic rule on purpose. Every unit
test passed all week while notifications sat unspoken for minutes, the pet said
things twice and [laughing] got read aloud — each an interaction between two
individually-correct units. It drives whole turns against a real HTTP server on
a loopback port, faking only the mic and the speakers. It found a NameError in
the paint path that would have fired on every repaint while talking.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

bolt_pet/ui/pet_window.py

@@ -19,7 +19,7 @@ from PySide6.QtGui import (
 )
 from PySide6.QtWidgets import QApplication, QWidget
 
-from .. import config
+from .. import config, window_state
 from ..monitors import Monitor
 from ..state import PetState
 from .sprite import WALK, SpriteSet
@@ -36,6 +36,11 @@ _NAP_OPACITY = 0.35
 # and the feet skate whenever PET_WANDER_SPEED doesn't happen to match the fps.
 # Eight frames at 13px is a ~104px stride cycle, a bit under the pet's width.
 _WALK_PIXELS_PER_FRAME = 13.0
+# How long a loudness level stays believable. The audio thread sends one per
+# ~30ms while a clip plays; if they stop arriving (offline TTS has no envelope,
+# or playback died) the mouth must not stay frozen mid-syllable, so after this
+# long the ordinary looping animation takes back over.
+_MOUTH_STALE_SECONDS = 0.35
 
 
 def emote_transform(emote: str, progress: float) -> tuple[float, float, float, float]:
@@ -213,6 +218,9 @@ class PetWindow(QWidget):
         self._commanded_move = False  # a petctl move — happens even mid-conversation
         self._next_wander_at = 0.0
         self._bob_offset = 0
+        # Lip-sync: loudness of what is playing right now, and when it arrived.
+        self._mouth_level: Optional[float] = None
+        self._mouth_at = 0.0
         self._bob_phase = 0.0
         self._walking = False
         self._facing = 1  # +1 right, -1 left; the walk art is drawn facing right
@@ -249,6 +257,19 @@ class PetWindow(QWidget):
             y = int(config.PET_START_Y) if config.PET_START_Y else None
         except ValueError:
             x = y = None
+        if x is None and y is None and config.PET_REMEMBER_POSITION:
+            remembered = window_state.load()
+            # Only if it still lands on a screen that exists — the usual reason
+            # a saved position is stale is that the monitor it was on has been
+            # unplugged, and restoring it faithfully would hide the pet.
+            if remembered is not None:
+                rectangles = [
+                    (s.availableGeometry().left(), s.availableGeometry().top(),
+                     s.availableGeometry().right(), s.availableGeometry().bottom())
+                    for s in QApplication.screens()
+                ]
+                if window_state.is_visible_on(*remembered, self.width(), rectangles):
+                    x, y = remembered
         if geo is not None:
             x = geo.right() - self.width() - 40 if x is None else x
             y = geo.bottom() - self.height() - 60 if y is None else y
@@ -688,6 +709,34 @@ class PetWindow(QWidget):
 
     # ── animation ────────────────────────────────────────────────────────
 
+    def set_mouth(self, level: float) -> None:
+        """How loud the pet is *right now* (0..1), straight off the PCM going
+        to the speakers (audio/tts.level_of).
+
+        The talking frames are ordered by mouth openness, so this indexes them
+        directly: the mouth moves with the actual waveform instead of flapping
+        on a timer, which is the difference between a talking sprite and a
+        dubbed one."""
+        self._mouth_level = max(0.0, min(1.0, float(level)))
+        self._mouth_at = time.monotonic()
+        if self._current_state == PetState.TALKING:
+            self.update()
+
+    def _mouth_frame(self, animation) -> Optional[QPixmap]:
+        """The frame matching the current loudness, or None to use the timer.
+
+        Falls back the moment the levels go stale — offline TTS has no
+        envelope, and a mouth frozen mid-syllable is worse than a timed loop."""
+        if self._mouth_level is None or animation is None:
+            return None
+        if time.monotonic() - self._mouth_at > _MOUTH_STALE_SECONDS:
+            return None
+        frames = animation.frames
+        if len(frames) < 2:
+            return None
+        index = int(round(self._mouth_level * (len(frames) - 1)))
+        return frames[max(0, min(len(frames) - 1, index))]
+
     def _advance_frame(self) -> None:
         # While walking the cycle is stepped by _advance_walk from distance
         # travelled; letting this timer also advance it would double-step it
@@ -702,7 +751,12 @@ class PetWindow(QWidget):
         painter.setRenderHint(QPainter.Antialiasing)
         painter.setRenderHint(QPainter.SmoothPixmapTransform)
         key = self._animation_key()
-        pixmap: Optional[QPixmap] = self.sprites.get(key).current()
+        animation = self.sprites.get(key)
+        pixmap: Optional[QPixmap] = None
+        if key == PetState.TALKING:
+            pixmap = self._mouth_frame(animation)
+        if pixmap is None:
+            pixmap = animation.current()
         if key == WALK:
             pixmap = self._oriented(pixmap)
         if pixmap is None:
@@ -759,6 +813,9 @@ class PetWindow(QWidget):
         was_click = not self._dragged
         self._drag_offset = None
         self._press_pos = None
+        if not was_click and config.PET_REMEMBER_POSITION:
+            position = self.geometry().topLeft()
+            window_state.save(position.x(), position.y())
         if was_click:
             self.talk_requested.emit()
         else: