Multi-monitor jumps, screen OCR, and generated sprite art
petctl gains screen verbs: `jump` (1-based number, name, next/prev/ primary/other, or a direction resolved from real geometry), `monitors`, and `read` for OCR of a monitor's contents. - monitors.py: pure layout model + jump-target resolution. The monitor list is published by PetWindow from QGuiApplication.screens() over a queued signal, so the controller and window agree on what "monitor 2" means; xrandr and Qt order screens differently on the same machine. - screen_text.py: pull-only OCR (mss capture + Tesseract/RapidOCR). Nothing captures unless the server asks, and the text rides back up the tool-result relay so Bolt can read a screen mid-turn. Both deps optional, soft-failing with a reason. SCREEN_TEXT=false removes it. - Query verbs are answered in controller._handle_command rather than pet_actions.describe(), because their output is the point. - scripts/generate_bolt_sprites.py draws every frame; walk/ is a side-view cycle stepped by distance travelled, not by the animation timer, so the planted paw tracks the window exactly. sprite.py loads it via EXTRA_ANIMATIONS keyed by name, with has() so callers can decline a placeholder blob. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -32,6 +32,18 @@ pynput>=1.7
|
||||
# Not imported by the app itself.
|
||||
Pillow>=10.0
|
||||
|
||||
# Screen reading (`petctl read`) — Bolt OCRs a monitor and uses the text in
|
||||
# his reply. Both optional: without them `petctl read` reports what's missing
|
||||
# and the rest of the pet is unaffected.
|
||||
# mss screen capture. X11/Win32/macOS — NOT Wayland.
|
||||
# pytesseract a thin wrapper; the actual engine is a system package:
|
||||
# sudo apt install tesseract-ocr
|
||||
# No-sudo alternative to those two lines: pip install rapidocr-onnxruntime
|
||||
# (pure pip, reuses the onnxruntime openwakeword already pulls in, slower to
|
||||
# start). screen_text.resolve_engine() picks whichever is present.
|
||||
mss>=9.0
|
||||
pytesseract>=0.3.10
|
||||
|
||||
# Test runner (tests/ — pure logic, no audio hardware or display needed;
|
||||
# run with QT_QPA_PLATFORM=offscreen).
|
||||
pytest>=8.0
|
||||
|
||||
Reference in New Issue
Block a user