Multi-monitor jumps, screen OCR, and generated sprite art

petctl gains screen verbs: `jump` (1-based number, name, next/prev/
primary/other, or a direction resolved from real geometry), `monitors`,
and `read` for OCR of a monitor's contents.

- monitors.py: pure layout model + jump-target resolution. The monitor
  list is published by PetWindow from QGuiApplication.screens() over a
  queued signal, so the controller and window agree on what "monitor 2"
  means; xrandr and Qt order screens differently on the same machine.
- screen_text.py: pull-only OCR (mss capture + Tesseract/RapidOCR).
  Nothing captures unless the server asks, and the text rides back up
  the tool-result relay so Bolt can read a screen mid-turn. Both deps
  optional, soft-failing with a reason. SCREEN_TEXT=false removes it.
- Query verbs are answered in controller._handle_command rather than
  pet_actions.describe(), because their output is the point.
- scripts/generate_bolt_sprites.py draws every frame; walk/ is a
  side-view cycle stepped by distance travelled, not by the animation
  timer, so the planted paw tracks the window exactly. sprite.py loads
  it via EXTRA_ANIMATIONS keyed by name, with has() so callers can
  decline a placeholder blob.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-07-28 16:17:11 -06:00
parent ccb3aeb7ae
commit b121bbba17
50 changed files with 2460 additions and 49 deletions
+30
View File
@@ -29,8 +29,18 @@ ANCHORS = (
EMOTES = ("wave", "hop", "spin", "nod", "shake", "bounce", "wiggle")
# Where `petctl jump` can be aimed. A bare number (1-based) works too, as does
# any unique part of a monitor's name — resolution lives in monitors.resolve().
MONITOR_SPECS = (
"next", "prev", "primary", "other", "random",
"left", "right", "up", "down",
)
HELP = (
"petctl move <x> <y> | <" + "|".join(ANCHORS) + ">\n"
"petctl jump <monitor number|" + "|".join(MONITOR_SPECS) + "|name>\n"
"petctl monitors\n"
"petctl read [monitor number|here|all]\n"
"petctl emote <" + "|".join(EMOTES) + ">\n"
"petctl say <text>\n"
"petctl wander on|off\n"
@@ -82,6 +92,24 @@ def parse(command: str) -> Optional[dict]:
raise ActionError(f"unknown position {args[0]!r}; try one of: " + ", ".join(ANCHORS))
return {"action": "move", "anchor": anchor}
if verb in ("jump", "monitor", "screen"):
if not args:
raise ActionError(
"jump needs a monitor: a number, a name, or one of "
+ ", ".join(MONITOR_SPECS)
)
# The spec isn't validated here on purpose: which monitors exist is a
# runtime fact this pure module doesn't have. monitors.resolve() does
# it once the published screen list is in hand.
return {"action": "jump", "target": " ".join(args).strip()}
if verb in ("monitors", "screens", "displays"):
return {"action": "monitors"}
if verb in ("read", "look", "ocr", "see"):
target = (" ".join(args).strip() or "here").lower()
return {"action": "read", "target": target}
if verb in ("emote", "do"):
if not args:
raise ActionError("emote needs a name: " + ", ".join(EMOTES))
@@ -124,6 +152,8 @@ def describe(action: dict) -> str:
if kind == "move":
where = action.get("anchor") or f"({action.get('x')}, {action.get('y')})"
return f"[pet] walking to {where}"
if kind == "jump":
return f"[pet] jumping to monitor {action['target']}"
if kind == "emote":
return f"[pet] {action['emote']}"
if kind == "say":