Multi-monitor jumps, screen OCR, and generated sprite art

petctl gains screen verbs: `jump` (1-based number, name, next/prev/
primary/other, or a direction resolved from real geometry), `monitors`,
and `read` for OCR of a monitor's contents.

- monitors.py: pure layout model + jump-target resolution. The monitor
  list is published by PetWindow from QGuiApplication.screens() over a
  queued signal, so the controller and window agree on what "monitor 2"
  means; xrandr and Qt order screens differently on the same machine.
- screen_text.py: pull-only OCR (mss capture + Tesseract/RapidOCR).
  Nothing captures unless the server asks, and the text rides back up
  the tool-result relay so Bolt can read a screen mid-turn. Both deps
  optional, soft-failing with a reason. SCREEN_TEXT=false removes it.
- Query verbs are answered in controller._handle_command rather than
  pet_actions.describe(), because their output is the point.
- scripts/generate_bolt_sprites.py draws every frame; walk/ is a
  side-view cycle stepped by distance travelled, not by the animation
  timer, so the planted paw tracks the window exactly. sprite.py loads
  it via EXTRA_ANIMATIONS keyed by name, with has() so callers can
  decline a placeholder blob.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-28 16:17:11 -06:00
parent ccb3aeb7ae
commit b121bbba17
50 changed files with 2460 additions and 49 deletions
+24
View File
@@ -144,6 +144,30 @@ TTS_STREAMING = os.environ.get("TTS_STREAMING", "true").lower() in ("1", "true",
SCREEN_CONTEXT = os.environ.get("SCREEN_CONTEXT", "true").lower() in ("1", "true", "yes", "on")
# ── monitors ────────────────────────────────────────────────────────────────
# A one-line note about the screen layout (how many, their sizes, which one
# the pet is standing on) rides along with each utterance, so Bolt can decide
# to `petctl jump` somewhere without asking you what you've got plugged in.
# Cheap — the list comes from the UI, nothing is probed per turn.
MONITOR_CONTEXT = os.environ.get("MONITOR_CONTEXT", "true").lower() in (
"1", "true", "yes", "on"
)
# ── screen text (OCR) ───────────────────────────────────────────────────────
# Lets Bolt actually read a monitor, via `petctl read`. Pull-only: nothing is
# captured unless the server asks for it, and every read is logged. Needs the
# optional capture/OCR extras — see the comments in requirements.txt.
#
# This widens what can leave the machine more than any other switch here: the
# recognised text of a whole screen goes to the server. It is *not* a new
# capability (the shell relay could already run a screenshot tool and OCR it),
# but it is a much easier one to use by accident. Set SCREEN_TEXT=false to
# take it away entirely.
SCREEN_TEXT = os.environ.get("SCREEN_TEXT", "true").lower() in ("1", "true", "yes", "on")
SCREEN_TEXT_MAX_CHARS = int(os.environ.get("SCREEN_TEXT_MAX_CHARS", "4000"))
# ── quiet hours / do-not-disturb ────────────────────────────────────────────
# Comma-separated HH:MM-HH:MM ranges (wrapping midnight is fine). While
# napping the pet dims, stops wandering, and makes no proactive noise —