Repository navigation
feat(box): let the phone app read conversations, follow events and get notified #207
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -19,7 +19,8 @@ before any of it runs. | |||||
| | A **kit's** author | Nothing, until you apply the kit you reviewed | [Kit review](#kits), pinned to a hash | | ||||||
| | A **plugin's** author | Everything the app can, once you allow the plugin | [Plugin consent](#plugins), pinned to a hash | | ||||||
| | Whoever writes **event data** (a PR title, a review comment, a branch name, an agent's output) | Text in your flows, never commands | [How flows fill in values](#flows-and-event-data) | | ||||||
| | Someone with a box's **phone token** | Read agents' screens and type into sessions on that box, from your tailnet | [The phone](#the-phone) | | ||||||
| | Someone with a box's **phone token** | Read agents' screens and recent conversations and type into sessions on that box, from your tailnet; have it notify their phone | [The phone](#the-phone) | | ||||||
| | The **push relay** | The notices it sends a phone: a box, worktree and session name, never what an agent wrote | [The phone](/guides/phone#notifications-through-the-push-relay) | | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: This row under-describes what the push relay receives. The notice also carries the project (location) name and the run name, as the phone guide states. Name them so the security table matches the payload. Prompt for AI agents
Suggested change
|
||||||
| | The **network** | Nothing: it sees encrypted traffic between pinned keys | [Identity and transport](#identity-and-transport) | | ||||||
|
|
||||||
| Pair only laptops you own, and only trust repositories, kits and plugins | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -47,11 +47,45 @@ The phone can: | |||||||||
| "needs you" notification too, with ntfy when that is on, and the push | ||||||||||
| opens the run. | ||||||||||
|
|
||||||||||
| For the Shipyard phone app, the box also offers (the | ||||||||||
| [phone API](/reference/phone-api)'s version 2): | ||||||||||
|
|
||||||||||
| - an agent's recent conversation, as its chat shows it: prompts, replies, | ||||||||||
| one-line tool calls, the files it edited with their line counts, its | ||||||||||
| questions and their answers. Never tool output, thinking, file contents, | ||||||||||
| diffs or older pages; | ||||||||||
|
Comment on lines
+55
to
+56
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: This promises no file contents or diffs reach the phone, but Prompt for AI agents
Suggested change
|
||||||||||
| - what happens as it happens: which session or run changed, by name, so the | ||||||||||
| app updates without asking every few seconds; | ||||||||||
| - notifications without ntfy, sent through Shipyard's push relay (below); | ||||||||||
| - stopping an agent, the way **Stop** does in the desktop app. | ||||||||||
|
|
||||||||||
| It cannot reach anything else on the box: not worktrees, locations, | ||||||||||
| sessions' commands, hooks, flows, kits, shares, upgrades, or the box's own | ||||||||||
| API. Those are different routes on a different port, and the phone listener | ||||||||||
| does not serve them. | ||||||||||
|
|
||||||||||
| ## Notifications through the push relay | ||||||||||
|
|
||||||||||
| The Shipyard phone app asks each box it pairs with to notify it. Apple and | ||||||||||
| Google deliver a notification only from a server holding the app's own key, | ||||||||||
| which a box can't have, so the box asks Shipyard's push relay | ||||||||||
| (`push.berthd.app`) to send it: | ||||||||||
|
|
||||||||||
| 1. The app gets a device token from the phone and gives it to the relay, | ||||||||||
| which seals it in a key only the relay can open. | ||||||||||
| 2. The app gives that key to each box (`POST /phone/v1/push`). A box never | ||||||||||
| sees the device token. | ||||||||||
| 3. When an agent needs you, the box sends the relay the key and the notice: | ||||||||||
| "Claude Code needs you", "shop/checkout on devl", and which session or run | ||||||||||
| it is. The relay makes the notification from those few fields itself and | ||||||||||
| keeps nothing. | ||||||||||
|
|
||||||||||
| The relay sees your box's name, the project and worktree, and the session or | ||||||||||
| run's name, never what an agent wrote. Settings → Phone lists the phones each | ||||||||||
| box notifies, and removes one. A new code, or turning phone access off, | ||||||||||
| forgets them all. If the relay says a phone no longer takes notifications | ||||||||||
| (the app was deleted), the box forgets it too. | ||||||||||
|
|
||||||||||
| ## Threat model | ||||||||||
|
|
||||||||||
| - **Off by default.** A box serves nothing for phones until a paired laptop | ||||||||||
|
|
@@ -76,11 +110,24 @@ does not serve them. | |||||||||
| the token in a header, never a cookie. The app's page denies framing and | ||||||||||
| only loads its own scripts. | ||||||||||
| - **What a stolen token allows.** Anyone on your tailnet with a box's token | ||||||||||
| can read its agents' screens and type into its sessions, which is enough to | ||||||||||
| instruct an agent. Treat the code like a password: make a new one if a | ||||||||||
| can read its agents' screens and recent conversations, follow which | ||||||||||
| sessions change, and type into its sessions, which is enough to instruct | ||||||||||
| an agent. They can also have the box notify a phone of theirs. Treat the code like a password: make a new one if a | ||||||||||
| phone is lost, and turn phone access off on boxes you don't need it on. | ||||||||||
| `before:session.send` hooks see phone prompts and key presses | ||||||||||
| (`from: "phone"`, with `key` for a key) and can refuse them. | ||||||||||
| `before:session.send` hooks see phone prompts, key presses and stops | ||||||||||
| (`from: "phone"`, with `key` for a key) and can refuse them, and | ||||||||||
| `before:phone.push` hooks see a phone asking for notifications. | ||||||||||
| - **The conversation is a copy, field by field.** The phone reads a | ||||||||||
| conversation through its own route, which copies each field it may show | ||||||||||
| from the box's reading, so a field added to the box's chat reaches a phone | ||||||||||
| only when it is added there on purpose. Tool call ids, the agent's mode, | ||||||||||
| model and tasks, published pages and the record's own name stay on the | ||||||||||
| box. A command's output is cut to 2 KB. | ||||||||||
| - **Events are rebuilt, not passed on.** The phone hears agents', sessions' | ||||||||||
| and runs' changes, by name and state, and a conversation that grew. Never | ||||||||||
| paths, logins, secrets, hooks or any other event, and at most the last 500 | ||||||||||
| when it catches up. At most 16 streams are open on a box; a new code | ||||||||||
| closes every one. | ||||||||||
| - **Notifications** carry which worktree and box need you, never what the | ||||||||||
| agent wrote. An ntfy topic is readable by anyone who knows its name, so | ||||||||||
| the topic Shipyard makes is long and random; use your own ntfy server for | ||||||||||
|
|
@@ -90,6 +137,9 @@ does not serve them. | |||||||||
| `~/.berth/network.json` (see | ||||||||||
| [outbound requests](/guides/automations#outbound-requests)); the check is | ||||||||||
| made on the address actually dialled, redirects included. | ||||||||||
| - **The push relay** is chosen by a paired laptop, never by a phone, and the | ||||||||||
| box reaches it on public addresses only. It gets the same notice ntfy | ||||||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P2: Prompt for AI agents
Suggested change
|
||||||||||
| does, plus the key that names the phone, and keeps neither. | ||||||||||
|
|
||||||||||
| ## What works where | ||||||||||
|
|
||||||||||
|
|
@@ -99,6 +149,7 @@ does not serve them. | |||||||||
| | Add to home screen | Yes, opens full screen | Yes | | ||||||||||
| | Notifications through ntfy | Yes, with the ntfy app | Yes, with the ntfy app | | ||||||||||
| | Web Push straight from a box | No | No | | ||||||||||
| | Notifications through the push relay | The Shipyard app | The Shipyard app, later | | ||||||||||
|
|
||||||||||
| Web Push needs a secure context (HTTPS) for the service worker it relies on, | ||||||||||
| and a tailnet address over HTTP is not one. Serving the app over HTTPS would | ||||||||||
|
|
||||||||||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -144,6 +144,8 @@ events. | |||||
| | `helper.installed` | `plugin`, `name`, `version`: a [plugin's helper](/guides/plugins#give-a-plugin-a-helper) was installed on the box and turned on | | ||||||
| | `helper.removed` | `plugin`, `purged`, `attached`: one was stopped and removed, with its data when `purged`; `attached` counts the running services that still point at it. Not announced when nothing of the helper was on the box | | ||||||
| | `phone.changed` | `enabled` | | ||||||
| | `phone.push.added` | `name`, `platform`: a phone app asked to be notified (`origin: "phone"`) | | ||||||
| | `phone.push.removed` | `name`, `platform`: a phone app is no longer notified: removed from the phone or a laptop, forgotten with a new code, or `reason: "gone"` when the push relay said it no longer takes notifications | | ||||||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: The Prompt for AI agents
Suggested change
|
||||||
| | `turncheck.changed` | `enabled`, `provider`, `url`, `model`, `key` (a reference, or empty), `key_pasted`, `key_removed`; never a key's value | | ||||||
| | `box.upgraded` | `build` | | ||||||
| | `pairing.invited` | `for`, `by`, `expires`: a paired laptop (`by`, as `laptop:<name>`) asked for a pairing code for another computer (`berth invite`); never the code | | ||||||
|
|
@@ -219,6 +221,7 @@ exits non-zero refuses it with what it printed. Gates in a laptop's | |||||
| | `before:integrations.install` | `tool` | | ||||||
| | `before:config.change` | `location`, `flows`, `env` or `guard`: what is about to change | | ||||||
| | `before:phone.change` | | | ||||||
| | `before:phone.push` | `id`, `name`, `platform`, `from: "phone"`: a phone app asking to be notified | | ||||||
| | `before:turncheck.change` | the same as `turncheck.changed` | | ||||||
| | `before:location.remove` | `location` | | ||||||
| | `before:hooks.change` | | | ||||||
|
|
||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| { "title": "Reference", "pages": ["cli", "berthd", "config", "events", "app-api", "plugin-sdk", "files"] } | ||
| { "title": "Reference", "pages": ["cli", "berthd", "config", "events", "app-api", "phone-api", "plugin-sdk", "files"] } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| --- | ||
| title: The phone API | ||
| description: What a box serves phones on its tailnet address, port 1379, for the web phone app and the Shipyard phone app. | ||
| --- | ||
|
|
||
| Each box with [phone access](/guides/phone) on serves a small API for phones | ||
| on its tailnet address, port 1379, beside the web phone app. It is a | ||
| different listener from the box's own API, and offers only what is below. | ||
|
|
||
| ## Transport | ||
|
|
||
| - **Address.** `http://<tailnet address>:1379`, plain HTTP inside | ||
| WireGuard. The `Host` must be the box's tailnet address, a `*.ts.net` | ||
| name, or its own short name, with the port. | ||
| - **Token.** Every request carries `Authorization: Bearer <token>`, the box's | ||
| phone token from the pairing link. Ten wrong tokens in a minute from one | ||
| address answer 429 until the minute is up. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. P3: The tenth bad token still returns 401; only subsequent requests get 429 while ten failures remain in the window. Say the lockout starts after ten failures. Prompt for AI agents |
||
| - **CORS.** Pages served from tailnet addresses and `*.ts.net` may read it | ||
| (another box's web phone app); the token is always a header, never a | ||
| cookie. | ||
| - **Errors.** A failed request answers `{ "error": "…" }`, for people. | ||
|
|
||
| ## Version 1 | ||
|
|
||
| | Route | Answers | | ||
| | --- | --- | | ||
| | `GET /phone/v1/info` | `{ box, version, capabilities }`. Version 1 boxes answer `version: 1` and no capabilities | | ||
| | `GET /phone/v1/sessions` | the box's sessions, as `GET /v1/sessions` lists them, each with its `worktree_title` | | ||
| | `GET /phone/v1/sessions/{name}/screen?history=N` | `{ screen }`, with up to 2000 lines above it | | ||
| | `POST /phone/v1/sessions/{name}/send` | `{ text, enter? }` types a prompt (Enter after it unless `enter: false`) | | ||
| | `POST /phone/v1/sessions/{name}/keys` | `{ key }` presses one of `enter`, `escape`, `interrupt` (Ctrl-C), `up`, `down`, `tab`, `1`–`9`, `y`, `n` | | ||
| | `GET /phone/v1/runs` | up to 30 active and 10 finished [runs](/guides/runs) | | ||
| | `GET /phone/v1/runs/{id}` | a run in outline: its steps with the first line of each output, without its variables, parameters, trigger or flow | | ||
| | `POST /phone/v1/runs/{id}/gates/{step}/decide` | `{ approve, pick? }` decides the gate a run waits at | | ||
| | `POST /phone/v1/runs/{id}/cancel` | cancels a run | | ||
|
|
||
| Sends and key presses go through `before:session.send` with `from: "phone"`. | ||
|
|
||
| ## Version 2 | ||
|
|
||
| A box whose `info` answers `version: 2` adds these, and lists them in | ||
| `capabilities`: `transcript`, `events`, `push` and `interrupt`. The phone app | ||
| reads every box at version 1 the way the web phone app does. | ||
|
|
||
| ### `GET /phone/v1/sessions/{name}/transcript?since=N&gen=G` | ||
|
|
||
| The agent's recent conversation, read from its own record like the desktop | ||
| app's chat, as `{ source, items, next, gen?, reset?, start?, truncated?, last?, reason?, record? }`. | ||
| Ask again from `next`, with `gen`; an answer with `reset` is the whole window | ||
| again, from `start` (see the [app's API](/reference/app-api)). | ||
|
|
||
| Items are the chat's kinds `user`, `text`, `tools`, `edit`, `crew`, | ||
| `command`, `notice`, `question`, `report`, `agent-message` and `ping`, each | ||
| with only these fields: | ||
|
|
||
| | Kind | Fields | | ||
| | --- | --- | | ||
| | every item | `kind`, `id`, `off`, `midTurn` | | ||
| | `user`, `text` | `text` | | ||
| | `tools` | `verb`, `done`, `items: [{ verb, target, file }]` | | ||
| | `edit` | `file` (in the worktree), `added`, `removed` | | ||
| | `crew` | `names` | | ||
| | `command` | `command`, `args`, `markdown`, `error`, and `text` cut to 2 KB | | ||
| | `notice` | `notice`, `level`, `resets`, `text` | | ||
| | `question` | `questions: [{ question, header, multi, options: [{ label, description }] }]`, `answers`, `done`, `error` | | ||
| | `report` | `report: { kind, session, run, title, worktree, branch, status, duration, files, added, removed, summary, answer, needs }` | | ||
| | `agent-message`, `ping` | `msg: { from: { name, kind }, intent, status, title, summary, body, repeat, at, answered }` | | ||
|
|
||
| `record` is a short hash of the record read: when it changes, the agent began | ||
| another conversation. With nothing to read (`source: "none"`), `reason` says | ||
| so without naming folders. Older pages (`?before=`) and a tool call's details | ||
| are not offered. | ||
|
|
||
| ### `GET /phone/v1/events?since=SEQ` | ||
|
|
||
| The box's events a phone hears, as NDJSON, one event a line, with a blank | ||
| line every 25 seconds to keep the connection open. With `since`, it first | ||
| replays those after `SEQ`, at most the last 500. At most 16 streams are open | ||
| on a box (429 past that), and a new code or phone access turned off ends | ||
| them all. | ||
|
|
||
| Each event is `{ seq, type, time, box, data }`, with only these types and | ||
| fields: | ||
|
|
||
| | Type | `data` | | ||
| | --- | --- | | ||
| | `agent.ready`, `agent.started`, `agent.waiting`, `agent.finished`, `agent.exited` | `session`, `agent`, `reason`, `status` | | ||
| | `session.started`, `session.stopped`, `session.renamed`, `session.sent`, `session.queued` | `name` | | ||
| | `transcript.changed` | `session` | | ||
| | `run.created`, `run.queued`, `run.started`, `run.resumed`, `run.step`, `run.gate`, `run.gate_decided`, `run.finished` | `run`, `status`, `title`, `step`, `step_status`, `gate_title`, `approve` | | ||
|
|
||
| ### `POST /phone/v1/push` | ||
|
|
||
| `{ id, name?, platform, key, on? }` asks the box to notify a phone: | ||
|
|
||
| - `id`: the app's install id, 32 hex characters. The same id again | ||
| replaces that phone's entry. | ||
| - `name`: up to 40 characters, shown in Settings → Phone. | ||
| - `platform`: `ios` or `android`. | ||
| - `key`: the key the [push relay](/guides/phone#notifications-through-the-push-relay) | ||
| gave the app. | ||
| - `on`: `waiting` (the default) and `finished`. | ||
|
|
||
| It answers `{ registered: true }`, after `before:phone.push`. A box notifies | ||
| at most 10 phones; another takes the place of the one registered longest ago. | ||
| The relay is chosen by a paired laptop (`PUT /v1/phone` with `relay`), never | ||
| here. | ||
|
|
||
| ### `DELETE /phone/v1/push/{id}` | ||
|
|
||
| Stops notifying the phone with that install id: `{ removed }`. | ||
|
|
||
| ### `POST /phone/v1/sessions/{name}/interrupt` | ||
|
|
||
| Stops the agent the way its session can: Esc in its terminal, or its own | ||
| interrupt for an agent run without one. It goes through | ||
| `before:session.send` with `from: "phone"` and `key: "escape"`, and answers | ||
| `{ sent, stopped }`. | ||
|
|
||
| ## For a laptop | ||
|
|
||
| A paired laptop reads and changes phone access over the box's own API: | ||
| `GET /v1/phone` and `PUT /v1/phone { enabled?, rotate?, notify?, clear_notify?, remove_push?, relay? }`. | ||
| Its answer lists the phones in `push` (`id`, `name`, `platform`, `on`, | ||
| `added`, `last_ok`, `last_error`), never their keys. `rotate` and turning | ||
| access off forget every phone; `relay` must be `https`, and `""` goes back | ||
| to `https://push.berthd.app`. | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: This says push is used 'instead of ntfy', but the code sends both when each is configured, and the guide says push works without ntfy. Change the wording to 'alongside ntfy'.
Prompt for AI agents