Update
This commit is contained in:
@@ -60,10 +60,10 @@ logs a missing-config message and exits its thread instead of starting.
|
||||
a `QWidget` directly. Also drives the periodic heartbeat
|
||||
(`_maybe_heartbeat`, gated by `HEARTBEAT_INTERVAL_SECONDS`) which lets the
|
||||
server push proactive spoken announcements between user turns, and on the
|
||||
same tick re-evaluates nap state and drains queued desktop notifications.
|
||||
It owns the live wake-word threshold (`wake_threshold()` is passed to
|
||||
`listen_for_wake_word` as a *callable* so the tray slider takes effect
|
||||
mid-listen) and the conversation `history`.
|
||||
same tick re-evaluates nap state, checks for delivered files, and drains
|
||||
queued desktop notifications. It owns the live wake-word threshold
|
||||
(`wake_threshold()` is passed to `listen_for_wake_word` as a *callable* so
|
||||
the tray slider takes effect mid-listen) and the conversation `history`.
|
||||
- **`server_client.py`** — HTTP client for the desk API, dependency-free
|
||||
beyond `requests` so it's easy to mock in tests. `converse()` loops relaying
|
||||
server-issued shell commands (`run_local_command`, executed via
|
||||
@@ -71,7 +71,24 @@ logs a missing-config message and exits its thread instead of starting.
|
||||
`/desk/tool_result` until the server sends a final `reply` (capped at
|
||||
`_MAX_RELAY_HOPS`). This is the same "full desktop control" trust model as
|
||||
the server repo's other desk clients — commands only ever originate from
|
||||
the user's own voice/click requests in their own session.
|
||||
the user's own voice/click requests in their own session. `list_outbox_files`
|
||||
/ `download_outbox_file` hit the same `/desk/files` and `/desk/files/<id>`
|
||||
endpoints the server's `deliver_files` tool queues onto — see `file_delivery.py`.
|
||||
- **`file_delivery.py`** — the filesystem half of receiving files the server
|
||||
queues via its `deliver_files` tool (`ai/desk_api.py` in the main tmn-api
|
||||
repo — "send me that report" during a conversation spools the matched
|
||||
workspace files, zipping multiple into one, onto the session's outbox).
|
||||
`controller._check_deliveries` lists `/desk/files` and downloads anything
|
||||
queued — right after a conversation/notification turn (the common case)
|
||||
and once per heartbeat tick for anything queued out-of-band — saving each
|
||||
under `DELIVERED_FILES_DIR` (default `~/Downloads/Bolt`). Downloading a
|
||||
file dequeues it server-side, so it's only ever handed out once; `save()`
|
||||
never overwrites an existing download, suffixing `" (1)"`, `" (2)"`, ... on
|
||||
a name collision. `sanitize_filename()` reduces a server-supplied name to
|
||||
its bare filename (`Path(...).name`), which is defense-in-depth against a
|
||||
delivered name that's secretly a path, since a per-user desk API key means
|
||||
the name isn't always coming from someone as trusted as the owner. Toggle
|
||||
off entirely with `RECEIVE_FILES=false`.
|
||||
- **`audio/`** — `mic.py` (energy-based VAD utterance capture, ported from the
|
||||
server repo's `bolt_desk.py`), `wake_word.py` (openWakeWord `thunderbolt.onnx`
|
||||
detection + `NearMissLog` for threshold tuning — see below), `stt.py`
|
||||
@@ -95,6 +112,35 @@ logs a missing-config message and exits its thread instead of starting.
|
||||
existing shell-command relay: `controller._handle_command` parses them and
|
||||
they never reach `subprocess`; anything else is a real shell command exactly
|
||||
as before. Pure parsing; the UI half is `PetWindow.apply_action`.
|
||||
- **`file_ops.py`** — `filectl` pseudo-commands, checked in `_handle_command`
|
||||
right after petctl and before falling through to a real shell command.
|
||||
Executing arbitrary commands already worked via the shell relay
|
||||
(`run_local_command` — see `server_client.py` below); what filectl adds is
|
||||
a *reliable* way to do the read/write/edit/list slice of that, since
|
||||
getting the model to hand-roll a shell heredoc for multi-line content full
|
||||
of quotes/`$`/backticks is failure-prone — and `list` exists as its own op
|
||||
(rather than relying on the model shelling out to `ls`/`dir`) because this
|
||||
project is cross-platform and the model shouldn't have to guess which
|
||||
listing command applies on Windows vs. Linux vs. macOS; one glob-based op
|
||||
(`pattern`, default `*`; `recursive` for `rglob` instead of `glob`) covers
|
||||
all three. Wire format is `filectl <json>` where `<json>` is a
|
||||
**single-line** compact JSON object —
|
||||
`{"op": "list"|"read"|"write"|"edit", "path": ..., ...}` — not a multi-line
|
||||
marker block (an earlier design): the server relays this as the argument
|
||||
to the ordinary `command` tool marker, and that marker's extractor
|
||||
(`ai/agents/default.py` in the main repo) only captures up to the next
|
||||
newline, so anything genuinely multi-line silently got truncated no matter
|
||||
how the prompt worded it. JSON sidesteps that for free — `json.dumps`
|
||||
already encodes embedded newlines as the two characters `\n`, not a real
|
||||
line break, so multi-line file content still fits on the one physical line
|
||||
the extractor sees. `edit` requires the old text to match exactly once —
|
||||
same discipline as this project's own code-editing tool — and raises
|
||||
rather than guessing if it's missing or ambiguous. This doesn't expand
|
||||
what the server can do to this machine (a relayed shell command could
|
||||
already overwrite anything the desktop user can write — see the security
|
||||
notes below); it's a safer path to the same capability. Pure parsing
|
||||
(`parse`) is separated from the filesystem I/O (`execute`), matching
|
||||
pet_actions.py's parse/describe split.
|
||||
- **`screen_context.py`** — active-window title (xprop/xdotool, Win32,
|
||||
osascript) appended to each utterance via `context_for()`, plus
|
||||
`is_fullscreen_active()` for do-not-disturb. Text only — the desk API takes
|
||||
@@ -245,6 +291,13 @@ matches the trust model of the server repo's other desk clients. Keep
|
||||
`DESK_API_KEY` private and don't expose the desk API port to the open
|
||||
internet.
|
||||
|
||||
`file_ops.py`'s `filectl` read/write/edit pseudo-commands ride that same
|
||||
relay and are bound by the same trust model — no path is off-limits beyond
|
||||
normal filesystem permissions for the desktop user, exactly like a relayed
|
||||
`cat`/`sed`/`rm` already isn't. They don't grant the server anything a shell
|
||||
command couldn't already do; they just make the read/write/edit path
|
||||
reliable instead of relying on the model getting shell quoting right.
|
||||
|
||||
`sudo_askpass.py` widens that further, by design: with `SUDO_ASKPASS_PROMPT`
|
||||
on (the default), a relayed bare `sudo` is rewritten to `sudo -A` and the
|
||||
password is collected in a desktop dialog, so commands can escalate to root
|
||||
|
||||
Reference in New Issue
Block a user