Merge remote-tracking branch 'origin/main' into screens

# Conflicts:
#	.claude/settings.local.json
#	CLAUDE.md
#	bolt_pet/controller.py
This commit is contained in:
2026-07-28 16:19:26 -06:00
12 changed files with 1029 additions and 37 deletions
+58 -5
View File
@@ -68,10 +68,10 @@ logs a missing-config message and exits its thread instead of starting.
a `QWidget` directly. Also drives the periodic heartbeat
(`_maybe_heartbeat`, gated by `HEARTBEAT_INTERVAL_SECONDS`) which lets the
server push proactive spoken announcements between user turns, and on the
same tick re-evaluates nap state and drains queued desktop notifications.
It owns the live wake-word threshold (`wake_threshold()` is passed to
`listen_for_wake_word` as a *callable* so the tray slider takes effect
mid-listen) and the conversation `history`.
same tick re-evaluates nap state, checks for delivered files, and drains
queued desktop notifications. It owns the live wake-word threshold
(`wake_threshold()` is passed to `listen_for_wake_word` as a *callable* so
the tray slider takes effect mid-listen) and the conversation `history`.
- **`server_client.py`** — HTTP client for the desk API, dependency-free
beyond `requests` so it's easy to mock in tests. `converse()` loops relaying
server-issued shell commands (`run_local_command`, executed via
@@ -79,7 +79,24 @@ logs a missing-config message and exits its thread instead of starting.
`/desk/tool_result` until the server sends a final `reply` (capped at
`_MAX_RELAY_HOPS`). This is the same "full desktop control" trust model as
the server repo's other desk clients — commands only ever originate from
the user's own voice/click requests in their own session.
the user's own voice/click requests in their own session. `list_outbox_files`
/ `download_outbox_file` hit the same `/desk/files` and `/desk/files/<id>`
endpoints the server's `deliver_files` tool queues onto — see `file_delivery.py`.
- **`file_delivery.py`** — the filesystem half of receiving files the server
queues via its `deliver_files` tool (`ai/desk_api.py` in the main tmn-api
repo — "send me that report" during a conversation spools the matched
workspace files, zipping multiple into one, onto the session's outbox).
`controller._check_deliveries` lists `/desk/files` and downloads anything
queued — right after a conversation/notification turn (the common case)
and once per heartbeat tick for anything queued out-of-band — saving each
under `DELIVERED_FILES_DIR` (default `~/Downloads/Bolt`). Downloading a
file dequeues it server-side, so it's only ever handed out once; `save()`
never overwrites an existing download, suffixing `" (1)"`, `" (2)"`, ... on
a name collision. `sanitize_filename()` reduces a server-supplied name to
its bare filename (`Path(...).name`), which is defense-in-depth against a
delivered name that's secretly a path, since a per-user desk API key means
the name isn't always coming from someone as trusted as the owner. Toggle
off entirely with `RECEIVE_FILES=false`.
- **`audio/`** — `mic.py` (energy-based VAD utterance capture, ported from the
server repo's `bolt_desk.py`), `wake_word.py` (openWakeWord `thunderbolt.onnx`
detection + `NearMissLog` for threshold tuning — see below), `stt.py`
@@ -109,6 +126,35 @@ logs a missing-config message and exits its thread instead of starting.
Query verbs (`monitors`, `read`) are answered in `_handle_command` rather
than by `pet_actions.describe()`, because their output *is* the point: it
goes back up the tool-result relay for Bolt to use in his reply.
- **`file_ops.py`** — `filectl` pseudo-commands, checked in `_handle_command`
right after petctl and before falling through to a real shell command.
Executing arbitrary commands already worked via the shell relay
(`run_local_command` — see `server_client.py` below); what filectl adds is
a *reliable* way to do the read/write/edit/list slice of that, since
getting the model to hand-roll a shell heredoc for multi-line content full
of quotes/`$`/backticks is failure-prone — and `list` exists as its own op
(rather than relying on the model shelling out to `ls`/`dir`) because this
project is cross-platform and the model shouldn't have to guess which
listing command applies on Windows vs. Linux vs. macOS; one glob-based op
(`pattern`, default `*`; `recursive` for `rglob` instead of `glob`) covers
all three. Wire format is `filectl <json>` where `<json>` is a
**single-line** compact JSON object —
`{"op": "list"|"read"|"write"|"edit", "path": ..., ...}` — not a multi-line
marker block (an earlier design): the server relays this as the argument
to the ordinary `command` tool marker, and that marker's extractor
(`ai/agents/default.py` in the main repo) only captures up to the next
newline, so anything genuinely multi-line silently got truncated no matter
how the prompt worded it. JSON sidesteps that for free — `json.dumps`
already encodes embedded newlines as the two characters `\n`, not a real
line break, so multi-line file content still fits on the one physical line
the extractor sees. `edit` requires the old text to match exactly once —
same discipline as this project's own code-editing tool — and raises
rather than guessing if it's missing or ambiguous. This doesn't expand
what the server can do to this machine (a relayed shell command could
already overwrite anything the desktop user can write — see the security
notes below); it's a safer path to the same capability. Pure parsing
(`parse`) is separated from the filesystem I/O (`execute`), matching
pet_actions.py's parse/describe split.
- **`screen_context.py`** — active-window title (xprop/xdotool, Win32,
osascript) appended to each utterance via `context_for()`, plus
`is_fullscreen_active()` for do-not-disturb. Text only — the desk API takes
@@ -296,6 +342,13 @@ matches the trust model of the server repo's other desk clients. Keep
`DESK_API_KEY` private and don't expose the desk API port to the open
internet.
`file_ops.py`'s `filectl` read/write/edit pseudo-commands ride that same
relay and are bound by the same trust model — no path is off-limits beyond
normal filesystem permissions for the desktop user, exactly like a relayed
`cat`/`sed`/`rm` already isn't. They don't grant the server anything a shell
command couldn't already do; they just make the read/write/edit path
reliable instead of relying on the model getting shell quoting right.
`sudo_askpass.py` widens that further, by design: with `SUDO_ASKPASS_PROMPT`
on (the default), a relayed bare `sudo` is rewritten to `sudo -A` and the
password is collected in a desktop dialog, so commands can escalate to root