diff --git a/.github/workflows/frontend.yml b/.github/workflows/frontend.yml index e64dfac65..dc39c6771 100644 --- a/.github/workflows/frontend.yml +++ b/.github/workflows/frontend.yml @@ -121,7 +121,7 @@ jobs: run: npm ci - name: Packaged dependency isolation regressions - run: node --test scripts/verify-packaged-dependencies.test.cjs scripts/npm-command.test.cjs scripts/prepare-native-dependencies.test.cjs scripts/vite-optional-ws.test.mjs + run: node --test scripts/verify-packaged-dependencies.test.cjs scripts/npm-command.test.cjs scripts/prepare-native-dependencies.test.cjs scripts/vite-optional-ws.test.mjs scripts/fetch-esbuild.test.cjs # Windows allocates a NEW, VISIBLE console for every console-subsystem # child of a process that owns none — which the Electron main process is. diff --git a/.github/workflows/rust.yml b/.github/workflows/rust.yml index 2e457f3a6..9da2c42e5 100644 --- a/.github/workflows/rust.yml +++ b/.github/workflows/rust.yml @@ -257,6 +257,39 @@ jobs: sudo apt-get update sudo apt-get install --yes libdbus-1-dev libxcb1-dev pkg-config + - uses: actions/setup-node@v4 + with: + node-version: 24 + + - name: Windows launcher runtime digest (actual PowerShell template) + if: matrix.os == 'windows-latest' + timeout-minutes: 2 + run: node scripts/agent-drafter/check-windows-runtime-digest.mjs + + - name: Full export payload (generated launchers and authenticated daemon fixture) + timeout-minutes: 2 + run: node scripts/agent-drafter/check-export-payload.mjs + + - name: Stage pinned native app compiler + timeout-minutes: 2 + shell: bash + run: | + node <<'NODE' + const fs = require('node:fs'); + const path = require('node:path'); + const { execFileSync } = require('node:child_process'); + const { fetchEsbuild, ESBUILD_VERSION } = require('./ui/desktop/scripts/fetch-esbuild.js'); + const compiler = path.resolve(fetchEsbuild(process.platform, process.arch, { + binDir: path.join(process.env.RUNNER_TEMP, 'biorouter-test-tools'), + cacheDir: path.join(process.env.RUNNER_TEMP, 'biorouter-esbuild-cache'), + })); + const version = execFileSync(compiler, ['--version'], { + encoding: 'utf8', timeout: 5000, maxBuffer: 64, + }).trim(); + if (version !== ESBUILD_VERSION) throw new Error('Native esbuild does not match the pinned release'); + fs.appendFileSync(process.env.GITHUB_ENV, `BIOROUTER_ESBUILD_BIN=${compiler}\n`); + NODE + # Start narrow (lib + bins on every OS) so the matrix is green on arrival; # widen as it proves stable. This selector leaves out `tests/`; those # binaries run in their own step below, ubuntu only, with no network. diff --git a/CLAUDE.md b/CLAUDE.md index 6c0782c61..8d4832831 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1060,8 +1060,16 @@ per-app agent over `GET /apps//agent`. Full design in need no region — the SDK always provides a `.br-dock` drawer. - **Apps vendor their own `src/sdk.ts`**, so `manifest.sdk_hash` fingerprints the SDK a bundle was built from. `build_app` refreshes the vendored copy and stamps - it; `serve_index` rebuilds on drift. Otherwise an app built before a protocol - addition silently ignores the new frames forever. + it; `serve_index` rebuilds on drift. The bundle also records the SDK's source + version, so an older daemon preserves a newer SDK across rebuilds. The desktop + ships pinned, integrity-checked esbuild beside the daemon; CLI packages stage + the same binary. Builds publish JavaScript and imported CSS together. +- **Browser launch uses the daemon origin.** The desktop's shared-daemon proxy + requires its secret, so an external browser cannot use that proxy's address. + `POST /apps/{id}/launch` returns the daemon's own loopback URL and a one-time + access link; desktop launch and managed preview use that origin. Browser-open + failures are shown to the person. Build-chat buttons check session metadata + before opening a tab, and show a disabled reason when the chat is gone. - **Export is directly runnable.** `export_app` writes `manifest.json` (without it the daemon 404s the app), leaves the endpoint unset so the SDK derives it from the page origin (the desktop app starts `biorouterd` on an *ephemeral* @@ -1071,6 +1079,29 @@ per-app agent over `GET /apps//agent`. Full design in binds loopback only. `.vault/` is excluded. The daemon's port env var is **`BIOROUTER_PORT`**, not `BIOROUTER_SERVER__PORT` (only the secret key uses the `__` form — `Settings` is a flat struct). +- **Exports have one implementation:** `agent_drafter/export.rs`. The desktop + downloads the `POST /apps/{id}/export` zip, preserves executable modes, and + safely extracts it into the person's chosen folder. Launcher, Full and bundled + daemon options reach the exporter; Full filters shipped skills and privacy + gates knowledge payloads. Exported HTML stays raw for daemon assembly, binary + assets are preserved, and working files, secrets and build-chat ids are omitted. + Launchers honor `BIOROUTER_PATH_ROOT` for an isolated recipient store and retain + backups before replacing installed contents. Full launchers start or reuse the + authenticated daemon before importing knowledge archives through its existing + multipart API. Import receipts prevent duplicate imports on retries; only + matching app declarations are rebound to allocated knowledge ids. Payload + consent and archive privacy/owner floors remain enforced. +- **App pages require a document navigation** on browser credentials in a secure + browser context (loopback or HTTPS). Plain HTTP on a network address refuses + app surfaces and the secret-bearing serve shell, because Fetch Metadata and + opener protections do not apply there. Forwarded HTTPS on a network host also + requires browser-controlled Fetch Metadata, so a spoofed proxy header alone + cannot bypass that refusal. Authored inline scripts and handlers run under a response-specific CSP; + runtime-injected handlers remain blocked. Figure frames have a separate policy + and are admitted by the desktop preview only as the same app's subframe. + Deleting an app retires its sockets and tokens; a replacement has a new + `app_uid` and a separate durable conversation. Per-app origins and private-app + classification remain the separately documented platform followups. - **Tests:** `cargo test -p biorouter-mcp --lib agent_drafter::`, `cargo test -p biorouter-mcp --test ui_example_apps`, `cargo test -p biorouter-server --lib routes::apps`. Browser-level: diff --git a/crates/biorouter-cli/src/commands/apps.rs b/crates/biorouter-cli/src/commands/apps.rs index 74a86a370..fd5a56815 100644 --- a/crates/biorouter-cli/src/commands/apps.rs +++ b/crates/biorouter-cli/src/commands/apps.rs @@ -34,13 +34,21 @@ //! `apps open` must keep passing [`Supervision::Detached`]: tying its daemon to //! this process would kill the daemon as `open` returned. //! -//! **Daemon management is deliberately minimal.** The CLI has no pre-existing -//! biorouterd-supervision helper, so `open`/`serve` first health-check the -//! configured port (`BIOROUTER_PORT`, default 3000) via the auth-exempt -//! `GET /status`, reuse a daemon if one answers, and otherwise best-effort spawn -//! the sibling `biorouterd agent`. If no `biorouterd` binary can be located the -//! command fails with the exact command to run — an honest v1 over a fragile -//! spawn. +//! **The profile's shared daemon comes first.** When the desktop app or +//! `biorouter crew` runs this profile's shared daemon, `open` and `serve` mint the +//! link there, over its private socket, and send the browser to that daemon's own +//! listener ([`shared_daemon_launch`]); nothing is started. They used to start a +//! second full daemon beside it that never exited. A port the person names in +//! `BIOROUTER_PORT` keeps its meaning: the shared daemon is used then only when +//! it listens on that port ([`named_port`]). +//! +//! **Otherwise daemon management is deliberately minimal.** From the configured +//! port (`BIOROUTER_PORT`, default 3000) up, `open`/`serve` health-check each port +//! via the auth-exempt `GET /status`, taking as a Biorouter daemon only one that +//! answers `ok` ([`port_state`]) and passing over a port anything else holds, +//! reuse a daemon whose key they know, and otherwise best-effort spawn the sibling +//! `biorouterd agent`. If no `biorouterd` binary can be located the command fails +//! with the exact command to run — an honest v1 over a fragile spawn. //! //! **Opening an app takes the daemon's secret (W2-HRD-1).** A daemon serves an //! app's page only to a browser holding that app's access cookie, set by a @@ -276,6 +284,60 @@ pub(crate) async fn daemon_ok(host: &str, port: u16) -> bool { } } +/// What answers on a loopback port, as far as `apps open` and `apps serve` care. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum PortState { + /// Nothing accepts a connection: a daemon can be started here. + Free, + /// A Biorouter daemon: `GET /status` answers 200 with the body `ok`. + Biorouter, + /// Something else holds the port: a dev server's catch-all that answers 200 + /// to any path, a listener that never answers, another service. + Foreign, +} + +/// What answers on `port` (see [`PortState`]). +/// +/// Stricter than [`daemon_ok`], which takes any 200 and which other commands +/// share: a Vite or CRA dev server on :3000 answers `/status` with its page, and +/// this command then said a Biorouter daemon whose key it did not know held the +/// port. HTTP/1.0, like [`with_secret`], so the body comes unchunked. +async fn port_state(port: u16) -> PortState { + let addr = format!("{DAEMON_HOST}:{port}"); + let connect = biorouter::net::connect_non_inheritable(&addr); + let mut stream = match tokio::time::timeout(Duration::from_millis(600), connect).await { + Ok(Ok(stream)) => stream, + _ => return PortState::Free, + }; + let request = format!("GET /status HTTP/1.0\r\nHost: {addr}\r\nConnection: close\r\n\r\n"); + if stream.write_all(request.as_bytes()).await.is_err() { + return PortState::Foreign; + } + let mut response = Vec::new(); + let read = tokio::time::timeout( + Duration::from_secs(2), + (&mut stream).take(4096).read_to_end(&mut response), + ) + .await; + if !matches!(read, Ok(Ok(_))) { + return PortState::Foreign; + } + let text = String::from_utf8_lossy(&response); + match text.split_once("\r\n\r\n") { + Some((head, body)) + if head.split_whitespace().nth(1) == Some("200") && body.trim() == "ok" => + { + PortState::Biorouter + } + _ => PortState::Foreign, + } +} + +/// How many ports `apps open` and `apps serve` try, from the configured one up, +/// before giving up on finding one that is free or holds a daemon they can use. +/// The exported launchers try the same six (3000–3005 by default). +const PORT_ATTEMPTS: u16 = 6; + /// Locate the `biorouterd` binary: prefer the sibling of the running `biorouter` /// executable (dev tree and installed app both colocate them), then the sibling /// of the installation this copy came from (a Windows install has nothing else @@ -620,6 +682,11 @@ pub(crate) fn open_page_in_browser(page: &Path) -> std::io::Result<()> { webbrowser::open(page_url.as_str()) } +/// `CREATE_NEW_PROCESS_GROUP | CREATE_NO_WINDOW`: a detached daemon's own +/// process group, and no console to receive this one's Ctrl-C or close. +#[cfg(windows)] +const DETACHED_DAEMON_FLAGS: u32 = 0x0000_0200 | 0x0800_0000; + /// Whether a daemon this command starts is tied to this process's lifetime. #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub(crate) enum Supervision { @@ -632,27 +699,67 @@ pub(crate) enum Supervision { TiedToThisProcess, } -/// Ensure a daemon whose secret this command knows is reachable on the configured -/// port, spawning one if needed. Answers the daemon and its secret. -async fn ensure_daemon(port: u16, supervision: Supervision) -> Result<(Daemon, String)> { - if daemon_ok(DAEMON_HOST, port).await { - let known = std::env::var("BIOROUTER_SERVER__SECRET_KEY") - .ok() - .filter(|secret| !secret.is_empty()) - .into_iter() - .chain(remembered_secret(port)); - for secret in known { - if secret_works(port, &secret).await { - return Ok((Daemon::Reused, secret)); +/// The secret of the daemon on `port`, when this command knows one that works: +/// `BIOROUTER_SERVER__SECRET_KEY`, or the record of a daemon `apps open` or +/// `apps serve` started there. +async fn known_secret(port: u16) -> Option { + let known = std::env::var("BIOROUTER_SERVER__SECRET_KEY") + .ok() + .filter(|secret| !secret.is_empty()) + .into_iter() + .chain(remembered_secret(port)); + for secret in known { + if secret_works(port, &secret).await { + return Some(secret); + } + } + None +} + +/// Ensure a daemon whose secret this command knows is reachable, from the +/// configured port up, spawning one if needed. Answers the daemon, its secret and +/// the port it is on. +/// +/// A port held by something that is not Biorouter is passed over for the next, +/// as the exported launchers do; it used to stop the command with a claim that a +/// Biorouter daemon whose key it lacked was running there. A Biorouter daemon on +/// the configured port whose key this command does not know still stops it: that +/// is a daemon the person pointed it at. +async fn ensure_daemon(first: u16, supervision: Supervision) -> Result<(Daemon, String, u16)> { + let mut port = first; + loop { + match port_state(port).await { + PortState::Free => break, + PortState::Biorouter => { + if let Some(secret) = known_secret(port).await { + return Ok((Daemon::Reused, secret, port)); + } + if port == first { + bail!( + "a Biorouter daemon is already running on port {port}, and opening an app on it takes \ + that daemon's secret key, which this command does not know. It knows the key of a \ + daemon `biorouter apps open` or `biorouter apps serve` started; for any other, set \ + BIOROUTER_SERVER__SECRET_KEY to its key, or set BIOROUTER_PORT to a free port so \ + this command starts a daemon of its own." + ); + } + eprintln!( + " {} port {port} has a Biorouter daemon this command has no key for; trying the next", + style("·").dim() + ); } + PortState::Foreign => eprintln!( + " {} port {port} is in use by something that is not Biorouter; trying the next", + style("·").dim() + ), } - bail!( - "a Biorouter daemon is already running on port {port}, and opening an app on it takes \ - that daemon's secret key, which this command does not know. It knows the key of a \ - daemon `biorouter apps open` or `biorouter apps serve` started; for any other, set \ - BIOROUTER_SERVER__SECRET_KEY to its key, or set BIOROUTER_PORT to a free port so \ - this command starts a daemon of its own." - ); + port = match port.checked_add(1) { + Some(next) if next - first < PORT_ATTEMPTS => next, + _ => bail!( + "no port from {first} to {port} is free for a Biorouter daemon. Set BIOROUTER_PORT \ + to a free port." + ), + }; } let secret = new_daemon_secret(); @@ -667,6 +774,15 @@ async fn ensure_daemon(port: u16, supervision: Supervision) -> Result<(Daemon, S .arg("--exit-with-parent") .arg(std::process::id().to_string()); } + // A daemon `apps open` leaves running must not share this console. Windows + // sends Ctrl-C, and the close of the console window, to every process + // attached to it, and the daemon stops on either: the next Ctrl-C at that + // prompt, or closing the window, took the open app down with it. Its own + // process group and no console, as `run.ps1` gives the daemon it starts. + #[cfg(windows)] + if supervision == Supervision::Detached { + command.creation_flags(DETACHED_DAEMON_FLAGS); + } let mut child = command .env("BIOROUTER_PORT", port.to_string()) // The secret rides the environment, which only this account can read, @@ -714,7 +830,7 @@ async fn ensure_daemon(port: u16, supervision: Supervision) -> Result<(Daemon, S // a later `apps open` or `apps serve`, which can mint it only with // this secret. `serve` removes the record when it stops the daemon. let remembered = remember_daemon(port, &secret); - return Ok((Daemon::Started { child, remembered }, secret)); + return Ok((Daemon::Started { child, remembered }, secret, port)); } if tokio::time::Instant::now() >= deadline { let _ = child.start_kill(); @@ -767,30 +883,204 @@ fn serve_link_advice(port: u16, id: &str, started: Option) -> String { } // ────────────────────────────────────────────────────────────────────────────── -// open +// the profile's shared daemon // ────────────────────────────────────────────────────────────────────────────── -pub async fn handle_apps_open(id: String) -> Result<()> { - require_app(&id)?; - let port = configured_port(); - let (daemon, secret) = ensure_daemon(port, Supervision::Detached).await?; - let url = app_url(port, &id); - let launch = launch_url(port, &id, &secret).await?; +/// `base` as an origin a browser on this machine is sent to: plain http to a +/// loopback host on an explicit port. A daemon listening on every interface names +/// `0.0.0.0` or `[::]`, an address to listen on and not one to visit, so it +/// becomes the loopback address of its family. `None` for anything else. +// Reached only through the shared daemon, which is unix-only for now. +#[cfg_attr(not(unix), allow(dead_code))] +fn loopback_origin(base: &str) -> Option { + let url = url::Url::parse(base).ok()?; + if url.scheme() != "http" + || !url.username().is_empty() + || url.password().is_some() + || url.path() != "/" + || url.query().is_some() + || url.fragment().is_some() + { + return None; + } + let port = url.port()?; + let host = match url.host_str()? { + "127.0.0.1" | "0.0.0.0" => "127.0.0.1", + "localhost" => "localhost", + "[::1]" | "[::]" => "[::1]", + _ => return None, + }; + Some(format!("http://{host}:{port}")) +} - match daemon { - Daemon::Reused => { - println!( - " {} using the daemon already running on port {port}", - style("·").dim() - ); +/// The address a browser on this machine opens app `id` at, from the shared +/// daemon's launch answer: the link on the daemon's own listener. That is the +/// answer's `url` when it names one, else the link's path on `listener` (the +/// daemon's `/headless/health` `api_base_url`, for a daemon from before the +/// answer named it). `None` unless the answer is exactly `id`'s page with a +/// token, on a loopback listener. +#[cfg_attr(not(unix), allow(dead_code))] +fn shared_launch_link( + answer: &serde_json::Value, + id: &str, + listener: Option<&str>, +) -> Option { + let path = answer.get("path")?.as_str()?; + if !is_launch_path(path, id) { + return None; + } + match answer.get("url") { + Some(serde_json::Value::String(url)) => { + let origin = url.strip_suffix(path)?; + (loopback_origin(origin)? == origin).then(|| url.clone()) } - Daemon::Started { .. } => { - // Leave the child running (detached) so the app stays served after - // this command returns — the tokio child is not killed on drop. - println!(" {} started biorouterd on port {port}", style("✓").green()); + None | Some(serde_json::Value::Null) => { + Some(format!("{}{path}", loopback_origin(listener?)?)) } + Some(_) => None, } +} + +/// Mint app `id`'s launch link on the shared daemon `client` reaches. +#[cfg(unix)] +async fn launch_on_shared_daemon( + client: &crate::daemon_client::CrewClient, + id: &str, +) -> Result { + // The id goes into a request line, so it must be an app name and nothing else. + biorouter_mcp::agent_drafter::store::validate_artifact_id(id) + .map_err(|_| anyhow!("'{id}' is not an app name"))?; + // `request_text`, not `request`: the daemon refuses a missing app in plain + // text, which `request` reports only as "not JSON". + let answer = client + .request_text("POST", &format!("/apps/{id}/launch"), None) + .await + .map_err( + |error| match error.downcast_ref::() { + Some(refusal) if refusal.status == 404 => { + anyhow!( + "this profile's Biorouter daemon would not open '{id}': it has no such app" + ) + } + _ => anyhow!("this profile's Biorouter daemon would not open '{id}': {error}"), + }, + )?; + let answer: serde_json::Value = serde_json::from_str(&answer).unwrap_or_default(); + let listener = if answer.get("url").is_some_and(|url| !url.is_null()) { + None + } else { + client + .request("GET", "/headless/health", None) + .await + .ok() + .and_then(|health| Some(health.get("api_base_url")?.as_str()?.to_string())) + }; + shared_launch_link(&answer, id, listener.as_deref()).ok_or_else(|| { + anyhow!("this profile's Biorouter daemon answered with an unexpected address for '{id}'") + }) +} + +/// App `id`'s launch link on this profile's shared daemon, the one the desktop +/// app and `biorouter crew` run, when one is running and this command can use +/// it: `Ok(None)` sends the caller on to a daemon of its own. +/// +/// These commands used to look only at `BIOROUTER_PORT` (default 3000), so with +/// the desktop app open they started a second full daemon, with its own MCP +/// servers and scheduler, that never exited. The shared daemon is reached over +/// its private socket with the key its starter saved, as every `crew` command +/// reaches it; nothing is started here (`connect(true)`), and a daemon this +/// command cannot use (started by an older Biorouter, or its key refused) is left +/// running and passed over, with a line saying why. +/// +/// A port named in `BIOROUTER_PORT` is honoured: see [`named_port`]. +async fn shared_daemon_launch(id: &str) -> Result> { + let named = named_port(std::env::var("BIOROUTER_PORT").ok().as_deref()); + shared_daemon_launch_on(id, named).await +} + +/// The port the person named in `BIOROUTER_PORT` (`value`), if they named one. +/// +/// Someone who runs `BIOROUTER_PORT=8080 biorouter apps serve ` to match an +/// SSH tunnel or a firewall rule means that port, and the shared daemon listens +/// on one of its own choosing. So a named port sends these commands past the +/// shared daemon unless it is the port that daemon listens on +/// ([`on_named_port`]). ⚠ Not "skip the shared daemon whenever the variable is +/// set": a shell the agent runs inside the shared daemon inherits the daemon's +/// own `BIOROUTER_PORT`, and that shell is where the agent itself runs +/// `apps open`. The desktop starts the daemon on a port it names, which then +/// matches; `biorouter crew` starts it with `0`, which asks for any free port and +/// names none. +fn named_port(value: Option<&str>) -> Option { + value?.trim().parse::().ok().filter(|port| *port != 0) +} + +/// Whether `link`, minted on the shared daemon, is on the port the person named. +#[cfg_attr(not(unix), allow(dead_code))] +fn on_named_port(link: &str, named: Option) -> bool { + match named { + None => true, + Some(port) => url::Url::parse(link).ok().and_then(|url| url.port()) == Some(port), + } +} + +/// [`shared_daemon_launch`], for the port the person named (`named`). +async fn shared_daemon_launch_on(id: &str, named: Option) -> Result> { + #[cfg(not(unix))] + { + let _ = (id, named); + Ok(None) + } + #[cfg(unix)] + { + use crate::daemon_client::{CrewClient, Restated, DAEMON_NOT_RUNNING_CODE}; + let client = match CrewClient::connect(true).await { + Ok(client) => client, + Err(error) => { + let absent = error + .downcast_ref::() + .is_some_and(|restated| restated.code == Some(DAEMON_NOT_RUNNING_CODE)); + if !absent { + let reason = error.to_string(); + eprintln!( + " {} not using this profile's Biorouter daemon: {}", + style("·").dim(), + reason.lines().next().unwrap_or_default() + ); + } + return Ok(None); + } + }; + // A named port is checked before anything is minted, so a shared daemon + // on another port is passed over without a link being made for nobody, + // and without its refusals deciding a command meant for another daemon. + if let Some(port) = named { + let listener = client + .request("GET", "/headless/health", None) + .await + .ok() + .and_then(|health| Some(health.get("api_base_url")?.as_str()?.to_string())); + if !listener + .as_deref() + .is_some_and(|listener| on_named_port(listener, named)) + { + eprintln!( + " {} not using this profile's Biorouter daemon, which listens on {}: \ + BIOROUTER_PORT names port {port}. Unset BIOROUTER_PORT to use it.", + style("·").dim(), + listener.as_deref().unwrap_or("another port"), + ); + return Ok(None); + } + } + let link = launch_on_shared_daemon(&client, id).await?; + // And the link itself, which is what the browser is sent to. + Ok(on_named_port(&link, named).then_some(link)) + } +} +/// Open `launch` in the default browser through a private launch page, and print +/// it for when no browser opened or one could not read the page. +fn open_or_print(launch: &str) { // The link itself never reaches an opener's command line: see // `write_launch_page`. The terminal is this account's alone, so the link is // printed there when no browser could be opened. @@ -799,7 +1089,7 @@ pub async fn handle_apps_open(id: String) -> Result<()> { // page (a snap-packaged browser cannot read hidden folders such as the state // dir) shows an error, and `open` still reports success. let once = format!("it works once, within {LAUNCH_LINK_LIFETIME}"); - match open_launch_link(&launch, &launch_pages_dir(), open_page_in_browser) { + match open_launch_link(launch, &launch_pages_dir(), open_page_in_browser) { Ok(_) => println!( " {} if the app does not appear, open this address instead ({once}): {launch}", style("·").dim() @@ -812,6 +1102,41 @@ pub async fn handle_apps_open(id: String) -> Result<()> { ); } } +} + +// ────────────────────────────────────────────────────────────────────────────── +// open +// ────────────────────────────────────────────────────────────────────────────── + +pub async fn handle_apps_open(id: String) -> Result<()> { + require_app(&id)?; + if let Some(launch) = shared_daemon_launch(&id).await? { + println!( + " {} using this profile's Biorouter daemon", + style("·").dim() + ); + open_or_print(&launch); + return Ok(()); + } + let (daemon, secret, port) = ensure_daemon(configured_port(), Supervision::Detached).await?; + let url = app_url(port, &id); + let launch = launch_url(port, &id, &secret).await?; + + match daemon { + Daemon::Reused => { + println!( + " {} using the daemon already running on port {port}", + style("·").dim() + ); + } + Daemon::Started { .. } => { + // Leave the child running (detached) so the app stays served after + // this command returns — the tokio child is not killed on drop. + println!(" {} started biorouterd on port {port}", style("✓").green()); + } + } + + open_or_print(&launch); println!(" {} {}", style("→").fg(ACCENT), style(&url).bold()); Ok(()) } @@ -822,14 +1147,30 @@ pub async fn handle_apps_open(id: String) -> Result<()> { pub async fn handle_apps_serve(id: String) -> Result<()> { require_app(&id)?; - let port = configured_port(); + // The profile's shared daemon is already served by whoever runs it, so there + // is nothing to hold in the foreground: print a link and return, as for any + // daemon this command did not start. + if let Some(launch) = shared_daemon_launch(&id).await? { + println!( + " {} the address below opens the app once, within {LAUNCH_LINK_LIFETIME}. \ + Run this again for another.", + style("·").dim() + ); + println!(" {} {}", style("→").fg(ACCENT), style(&launch).bold()); + println!( + " {} served by this profile's Biorouter daemon; it keeps running after this command.", + style("·").dim() + ); + return Ok(()); + } // ⚠ BEFORE the spawn, and before anything that can park. A handler replaces // the default action — which for SIGTERM is to end this process on the spot // and leave the daemon behind — so from here on a signal waits to be read, // including one that lands during the readiness wait inside `ensure_daemon`. let mut stop = StopSignals::install()?; - let (daemon, secret) = ensure_daemon(port, Supervision::TiedToThisProcess).await?; + let (daemon, secret, port) = + ensure_daemon(configured_port(), Supervision::TiedToThisProcess).await?; let url = match launch_url(port, &id, &secret).await { Ok(url) => url, Err(error) => { @@ -1308,11 +1649,12 @@ mod tests { assert!(unknown.contains("does not know"), "{unknown}"); assert!(remember_daemon(port, SECRET)); - let (daemon, secret) = ensure_daemon(port, Supervision::TiedToThisProcess) + let (daemon, secret, reused) = ensure_daemon(port, Supervision::TiedToThisProcess) .await .expect("a recorded daemon is reused"); assert!(matches!(daemon, Daemon::Reused)); assert_eq!(secret, SECRET); + assert_eq!(reused, port); let first = launch_url(port, "beta", &secret).await.unwrap(); let second = launch_url(port, "beta", &secret).await.unwrap(); @@ -1357,6 +1699,372 @@ mod tests { } } + /// A server that answers every request with `status` and `body`, as a dev + /// server's catch-all does, on `listener`. + fn answer_everything( + listener: tokio::net::TcpListener, + status: &'static str, + body: &'static str, + ) { + tokio::spawn(async move { + while let Ok((mut stream, _)) = listener.accept().await { + tokio::spawn(async move { + let mut chunk = [0u8; 1024]; + let _ = stream.read(&mut chunk).await; + let answer = format!( + "HTTP/1.1 {status}\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}", + body.len() + ); + let _ = stream.write_all(answer.as_bytes()).await; + let _ = stream.shutdown().await; + }); + } + }); + } + + /// Two listeners on adjacent loopback ports, the lower one first. + async fn adjacent_listeners() -> (tokio::net::TcpListener, tokio::net::TcpListener) { + for _ in 0..50 { + let lower = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let port = lower.local_addr().unwrap().port(); + let Some(next) = port.checked_add(1) else { + continue; + }; + if let Ok(upper) = tokio::net::TcpListener::bind(("127.0.0.1", next)).await { + return (lower, upper); + } + } + panic!("no two adjacent loopback ports were free"); + } + + /// Only a daemon that says `ok` on `/status` is a Biorouter daemon. Any 200 + /// used to be, so a dev server's catch-all on :3000 stopped `apps open` with + /// a claim that a Biorouter daemon whose key it lacked was running there. + #[tokio::test] + async fn only_a_status_that_says_ok_is_a_biorouter_daemon() { + let daemon = a_daemon_with("k", "beta").await; + assert_eq!(port_state(daemon).await, PortState::Biorouter); + + for (status, body) in [ + ("200 OK", "
"), + ("200 OK", ""), + ("404 Not Found", "ok"), + ] { + let listener = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let port = listener.local_addr().unwrap().port(); + answer_everything(listener, status, body); + assert_eq!( + port_state(port).await, + PortState::Foreign, + "{status} {body:?}" + ); + } + + let closed = tokio::net::TcpListener::bind("127.0.0.1:0").await.unwrap(); + let free = closed.local_addr().unwrap().port(); + drop(closed); + assert_eq!(port_state(free).await, PortState::Free); + } + + /// A port held by something that is not Biorouter is passed over for the + /// next, as the exported launchers do, and a daemon there that this command + /// started is reused. + #[tokio::test] + async fn a_port_held_by_something_else_is_passed_over() { + const SECRET: &str = "a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a11a"; + let (foreign, relay) = adjacent_listeners().await; + let first = foreign.local_addr().unwrap().port(); + answer_everything(foreign, "200 OK", ""); + let next = relay.local_addr().unwrap().port(); + // `a_daemon_with` binds a port of its own, so stand the daemon up on `next` + // through the listener already holding it, which hands its connections to + // the same stand-in. ⚠ Not dropped and bound again: macOS hands out + // ephemeral ports in order, so the stand-in's own `:0` took `next` in the + // gap and the second bind failed with `AddrInUse`. + let daemon = a_daemon_with(SECRET, "beta").await; + tokio::spawn(async move { + while let Ok((mut inbound, _)) = relay.accept().await { + tokio::spawn(async move { + if let Ok(mut outbound) = + tokio::net::TcpStream::connect(("127.0.0.1", daemon)).await + { + let _ = tokio::io::copy_bidirectional(&mut inbound, &mut outbound).await; + } + }); + } + }); + assert!(remember_daemon(next, SECRET)); + + let (daemon_state, secret, port) = ensure_daemon(first, Supervision::Detached) + .await + .expect("the daemon on the next port is reused"); + assert!(matches!(daemon_state, Daemon::Reused)); + assert_eq!((secret.as_str(), port), (SECRET, next)); + forget_daemon(next, SECRET); + } + + #[test] + fn a_shared_daemons_listener_is_a_loopback_origin() { + for (base, origin) in [ + ("http://127.0.0.1:51851", Some("http://127.0.0.1:51851")), + ("http://0.0.0.0:51851", Some("http://127.0.0.1:51851")), + ("http://[::]:51851", Some("http://[::1]:51851")), + ("http://localhost:51851/", Some("http://localhost:51851")), + ("https://127.0.0.1:51851", None), + ("http://10.0.0.5:51851", None), + ("http://127.0.0.1", None), + ("http://127.0.0.1:51851/below", None), + ("http://u@127.0.0.1:51851", None), + ("not a url", None), + ] { + assert_eq!(loopback_origin(base).as_deref(), origin, "{base}"); + } + } + + /// The link `apps open` hands the browser for a shared daemon is the one on + /// the daemon's own listener: the daemon's `url`, or else its path on the + /// listener the daemon names in `/headless/health`. + #[test] + fn a_shared_daemons_link_is_on_its_own_listener() { + let token = "ab".repeat(32); + let path = format!("/apps/beta/?t={token}"); + let url = format!("http://127.0.0.1:51851{path}"); + let answer = serde_json::json!({ "path": path, "url": url }); + assert_eq!(shared_launch_link(&answer, "beta", None), Some(url.clone())); + + let older = serde_json::json!({ "path": path }); + assert_eq!( + shared_launch_link(&older, "beta", Some("http://0.0.0.0:51851")), + Some(url.clone()) + ); + assert_eq!(shared_launch_link(&older, "beta", None), None); + assert_eq!( + shared_launch_link(&older, "beta", Some("http://10.0.0.5:1")), + None + ); + + for wrong in [ + serde_json::json!({ "path": path, "url": format!("http://127.0.0.1:51851/apps/other/?t={token}") }), + serde_json::json!({ "path": path, "url": format!("{url}&next=/sessions") }), + serde_json::json!({ "path": path, "url": format!("http://evil.test:1{path}") }), + serde_json::json!({ "path": path, "url": 42 }), + serde_json::json!({ "path": format!("/apps/other/?t={token}"), "url": url }), + ] { + assert_eq!( + shared_launch_link(&wrong, "beta", Some("http://127.0.0.1:1")), + None, + "{wrong}" + ); + } + } + + /// A stand-in for the profile's shared daemon on its private socket: its + /// identity, the route a client checks its key on, an app's launch route + /// (answering `url` only when `names_listener`) and `/headless/health`. + #[cfg(unix)] + async fn a_shared_daemon( + names_listener: bool, + ) -> ( + biorouter::daemon_runtime::Descriptor, + std::sync::Arc>>, + ) { + use biorouter::daemon_runtime::{self, Descriptor, Endpoint}; + use bytes::Bytes; + use http_body_util::Full; + use hyper::{server::conn::http1, service::service_fn, Request, Response}; + use hyper_util::rt::TokioIo; + use std::os::unix::fs::PermissionsExt; + + let directory = daemon_runtime::runtime_directory(); + daemon_runtime::private_directory(&directory).unwrap(); + let socket = directory.join("daemon.sock"); + let _ = fs::remove_file(&socket); + let listener = tokio::net::UnixListener::bind(&socket).unwrap(); + fs::set_permissions(&socket, fs::Permissions::from_mode(0o600)).unwrap(); + let descriptor = Descriptor { + version: daemon_runtime::VERSION, + profile_id: daemon_runtime::profile_identity().unwrap(), + instance_id: "33333333-3333-4333-8333-333333333333".to_owned(), + pid: std::process::id(), + endpoint: Endpoint::Unix { path: socket }, + api_secret: "shared-daemon-secret-for-apps-tests".to_owned(), + user_action_installed: true, + }; + daemon_runtime::write_private(&daemon_runtime::descriptor_path(), &descriptor).unwrap(); + daemon_runtime::write_user_action_key(&descriptor, &"cd".repeat(32)).unwrap(); + let identity = descriptor.identity(); + let seen = std::sync::Arc::new(std::sync::Mutex::new(Vec::new())); + let requests = std::sync::Arc::clone(&seen); + tokio::spawn(async move { + while let Ok((stream, _)) = listener.accept().await { + let identity = identity.clone(); + let requests = std::sync::Arc::clone(&requests); + tokio::spawn(async move { + let service = service_fn(move |request: Request| { + let identity = identity.clone(); + let requests = std::sync::Arc::clone(&requests); + async move { + let line = format!("{} {}", request.method(), request.uri().path()); + requests.lock().unwrap().push(line.clone()); + let path = format!("/apps/beta/?t={}", "ef".repeat(32)); + let body = match line.as_str() { + "GET /daemon/identity" => serde_json::to_value(&identity).unwrap(), + "GET /crew/connections" => serde_json::json!([]), + "POST /apps/beta/launch" if names_listener => serde_json::json!({ + "path": path, + "url": format!("http://127.0.0.1:51851{path}"), + }), + "POST /apps/beta/launch" => serde_json::json!({ "path": path }), + "GET /headless/health" => serde_json::json!({ + "status": "ok", + "api_base_url": "http://127.0.0.1:51900", + }), + _ => { + return Ok::<_, std::convert::Infallible>( + Response::builder() + .status(404) + .header("content-type", "text/plain") + .body(Full::new(Bytes::from_static(b"no such app"))) + .unwrap(), + ) + } + }; + Ok(Response::builder() + .header("content-type", "application/json") + .body(Full::new(Bytes::from(body.to_string()))) + .unwrap()) + } + }); + let _ = http1::Builder::new() + .serve_connection(TokioIo::new(stream), service) + .await; + }); + } + }); + (descriptor, seen) + } + + #[cfg(unix)] + fn forget_shared_daemon() { + let _ = fs::remove_file(biorouter::daemon_runtime::descriptor_path()); + let _ = fs::remove_file(biorouter::daemon_runtime::user_action_key_path()); + } + + /// With the desktop app open, `apps open` and `apps serve` started a second + /// full daemon, keyless and never exiting, beside the profile's shared one. + /// They now mint the link on the shared daemon and send the browser to its + /// own listener. + #[cfg(unix)] + #[tokio::test] + #[serial_test::serial] + async fn apps_open_prefers_the_profiles_shared_daemon() { + // `_on(…, None)`, not `shared_daemon_launch`: a test run from a shell + // inside a daemon inherits that daemon's BIOROUTER_PORT. + forget_shared_daemon(); + assert_eq!( + shared_daemon_launch_on("beta", None).await.unwrap(), + None, + "with no shared daemon the command starts one of its own" + ); + + let (_descriptor, seen) = a_shared_daemon(true).await; + let link = shared_daemon_launch_on("beta", None).await.unwrap(); + assert_eq!( + link, + Some(format!( + "http://127.0.0.1:51851/apps/beta/?t={}", + "ef".repeat(32) + )) + ); + assert!(seen + .lock() + .unwrap() + .iter() + .any(|line| line == "POST /apps/beta/launch")); + let missing = shared_daemon_launch_on("gone", None) + .await + .unwrap_err() + .to_string(); + assert!(missing.contains("it has no such app"), "{missing}"); + forget_shared_daemon(); + + // A daemon from before the answer named its listener: the listener its + // health route names. + let (_descriptor, _seen) = a_shared_daemon(false).await; + assert_eq!( + shared_daemon_launch_on("beta", None).await.unwrap(), + Some(format!( + "http://127.0.0.1:51900/apps/beta/?t={}", + "ef".repeat(32) + )) + ); + forget_shared_daemon(); + } + + /// `BIOROUTER_PORT=8080 biorouter apps serve `, run to match an SSH + /// tunnel, printed a link on the shared daemon's own port and returned, with + /// nothing listening on 8080. A named port is honoured; `0`, which a shell + /// inside a `crew`-started daemon inherits, names none. + #[test] + fn a_port_the_person_names_is_one_they_mean() { + for (value, named) in [ + (None, None), + (Some("0"), None), + (Some(""), None), + (Some("eighty"), None), + (Some("70000"), None), + (Some("8080"), Some(8080)), + (Some(" 8080\n"), Some(8080)), + ] { + assert_eq!(named_port(value), named, "{value:?}"); + } + let link = format!("http://127.0.0.1:51851/apps/beta/?t={}", "ef".repeat(32)); + assert!(on_named_port(&link, None)); + assert!(on_named_port(&link, Some(51851))); + assert!(on_named_port("http://0.0.0.0:51851", Some(51851))); + assert!(!on_named_port(&link, Some(8080))); + assert!(!on_named_port("not a url", Some(8080))); + } + + #[cfg(unix)] + #[tokio::test] + #[serial_test::serial] + async fn a_named_port_passes_over_a_shared_daemon_on_another() { + forget_shared_daemon(); + // The stand-in's health route names 51900, and so does the link it + // mints when its answer names no listener. + let (_descriptor, seen) = a_shared_daemon(false).await; + assert_eq!( + shared_daemon_launch_on("beta", Some(8080)).await.unwrap(), + None + ); + assert!( + !seen + .lock() + .unwrap() + .iter() + .any(|line| line.ends_with("/launch")), + "nothing is minted on a daemon the command will not use" + ); + // The shell inside the daemon: its own port is the one named. + assert_eq!( + shared_daemon_launch_on("beta", Some(51900)).await.unwrap(), + Some(format!( + "http://127.0.0.1:51900/apps/beta/?t={}", + "ef".repeat(32) + )) + ); + forget_shared_daemon(); + + // A link off the named port is never handed out, whatever health said. + let (_descriptor, _seen) = a_shared_daemon(true).await; + assert_eq!( + shared_daemon_launch_on("beta", Some(51900)).await.unwrap(), + None + ); + forget_shared_daemon(); + } + /// What the person is told about a link's lifetime is the daemon's figure, /// which lives in a crate this one cannot import. #[test] diff --git a/crates/biorouter-mcp/src/agent_drafter/bundle.rs b/crates/biorouter-mcp/src/agent_drafter/bundle.rs index e1022c1f1..0cb6bc100 100644 --- a/crates/biorouter-mcp/src/agent_drafter/bundle.rs +++ b/crates/biorouter-mcp/src/agent_drafter/bundle.rs @@ -2,19 +2,28 @@ //! //! Apps are authored in TypeScript (`src/main.ts` importing `src/sdk.ts`) and //! bundled into a single browser-runnable `dist/app.js` with **esbuild**. esbuild -//! is located via (in order): `$BIOROUTER_ESBUILD_BIN`, the desktop app's bundled -//! `node_modules/.bin/esbuild` (dev tree), `esbuild` on `PATH`, then `npx esbuild`. +//! is located via (in order, see [`find_esbuild`]): `$BIOROUTER_ESBUILD_BIN`, the +//! copy the desktop app ships beside `biorouterd`, the checkout's +//! `ui/desktop/node_modules/.bin/esbuild` (dev tree), `esbuild` on the process +//! or login-shell `PATH`, then `npx --yes esbuild@`. //! //! When no esbuild is available (e.g. a headless CLI install with no Node), a -//! best-effort vendored type-stripper concatenates the sources into a runnable -//! bundle so simple apps still work. The stripper is intentionally conservative; -//! complex TypeScript should be built where esbuild is present. +//! vendored type-stripper builds the simple shapes it can prove it handles and +//! REFUSES the rest, saying so. ⚠ It is a last resort, not a second compiler: it +//! used to accept everything, which shipped broken bundles that looked built. +//! See [`fallback_bundle`] for what it refuses and why. +//! +//! Every bundle this module writes starts with a one-line banner +//! ([`BundleBanner`]) naming the builder, the App SDK it carries and the +//! Biorouter that built it. That is how a daemon tells a fallback bundle from a +//! real one, and never rebuilds an app down to an older SDK. +use std::ffi::{OsStr, OsString}; use std::io::{self, Read}; use std::path::{Path, PathBuf}; use std::process::{Child, Command, Stdio}; -use std::sync::atomic::{AtomicBool, Ordering as AtomicOrdering}; -use std::sync::{Mutex, MutexGuard}; +use std::sync::atomic::{AtomicBool, AtomicU64, Ordering as AtomicOrdering}; +use std::sync::{Mutex, MutexGuard, OnceLock}; use std::time::{Duration, Instant}; use crate::agent_drafter::store::{ArtifactKind, Manifest}; @@ -101,6 +110,17 @@ pub fn lint_app(project_dir: &Path) -> Vec { break; } } + // Inline `"#, + "", + Some(manifest), + ); + assert!(!wired.contains("won't do anything"), "{wired}"); + assert!(!wired.contains("javascript:"), "{wired}"); + + let dir = TempDir::new().unwrap(); + std::fs::write( + dir.path().join("index.html"), + r#"
Go
"#, + ) + .unwrap(); + std::fs::write(dir.path().join("manifest.json"), manifest).unwrap(); + let findings = lint_app(dir.path()); + let url = findings + .iter() + .find(|f| f.msg.contains("javascript:")) + .expect("a javascript: URL is reported"); + assert_eq!(url.level, LintLevel::Warn); + } + /// H6: a hardcoded text color won't adapt to the theme and can go invisible. #[test] fn lint_warns_on_hardcoded_text_color_but_not_theme_tokens() { @@ -2870,12 +4644,11 @@ document.getElementById("edit")!.addEventListener("click", () => { #[test] #[ignore] fn leak_probe_prints_esbuild_child_env() { - let (program, lead) = find_esbuild().expect("BIOROUTER_ESBUILD_BIN shim"); + let cmd = find_esbuild().expect("BIOROUTER_ESBUILD_BIN shim"); let work = TempDir::new().unwrap(); let entry = work.path().join("main.ts"); std::fs::write(&entry, "const x: number = 1;\n").unwrap(); - let report = - run_esbuild(&program, &lead, &entry, &work.path().join("app.js")).expect("spawn"); + let report = run_esbuild(&cmd, &entry, &work.path().join("app.js")).expect("spawn"); let canary = std::env::var("BR_TEST_CANARY").unwrap_or_default(); // Report BioRouter's namespace, the variables the parent injected, // and any line carrying the canary under *any* key. Everything else diff --git a/crates/biorouter-mcp/src/agent_drafter/control.rs b/crates/biorouter-mcp/src/agent_drafter/control.rs index 65163722e..b40a7f711 100644 --- a/crates/biorouter-mcp/src/agent_drafter/control.rs +++ b/crates/biorouter-mcp/src/agent_drafter/control.rs @@ -588,11 +588,10 @@ pub struct ConsultRequest { struct BridgeInner { /// Outbound half of the *current* connection. `None` between connections. tx: Mutex>>, - /// Monotonic id of the current connection. Bumped by every `attach`; a - /// `detach` only tears down when it still owns this generation, so a stale - /// connection closing *after* a reload can't kill the fresh connection's - /// channel or cancel its parked `ui_ask`. - generation: AtomicU64, + /// Serializes the complete attach/detach lifecycle. A generation check + /// outside this lock could pass before a reload and then clear its new + /// sender or cancel its freshly parked tools. + generation: Mutex, /// `requestId` → responder for a parked `ui_ask`. pending: Mutex>>, /// `callId` → responder for a parked `app_call`. Separate from [`Self::pending`] @@ -678,7 +677,7 @@ impl UiBridge { Self { inner: Arc::new(BridgeInner { tx: Mutex::new(None), - generation: AtomicU64::new(0), + generation: Mutex::new(0), pending: Mutex::new(HashMap::new()), pending_calls: Mutex::new(HashMap::new()), consult_tx: Mutex::new(None), @@ -707,9 +706,13 @@ impl UiBridge { /// the reloaded page has none of it. The semantic `state` bag survives and is /// re-emitted so the fresh page rehydrates. pub fn attach(&self) -> (mpsc::UnboundedReceiver, ConnToken) { - // Claim the next generation first: any concurrent stale detach now sees a - // newer generation and becomes a no-op. - let gen = self.inner.generation.fetch_add(1, Ordering::AcqRel) + 1; + let mut generation = self + .inner + .generation + .lock() + .unwrap_or_else(|p| p.into_inner()); + *generation += 1; + let gen = *generation; self.cancel_all(); let (tx, rx) = mpsc::unbounded_channel(); if let Ok(mut slot) = self.inner.tx.lock() { @@ -744,7 +747,12 @@ impl UiBridge { /// a stale connection closing *after* a reload cannot tear down the fresh /// one. Parked `ui_ask` calls are unblocked so no tool outlives its socket. pub fn detach(&self, token: ConnToken) { - if self.inner.generation.load(Ordering::Acquire) != token.0 { + let generation = self + .inner + .generation + .lock() + .unwrap_or_else(|p| p.into_inner()); + if *generation != token.0 { return; // a newer connection owns the bridge now } if let Ok(mut slot) = self.inner.tx.lock() { @@ -5194,6 +5202,47 @@ mod tests { ); } + #[test] + fn a_detach_paused_at_its_sender_cannot_clear_a_concurrent_reattach() { + let bridge = UiBridge::new(); + let (_old_frames, old_token) = bridge.attach(); + + // Force detach to pause after claiming the connection lifecycle, while + // a reload tries to install a new sender and park a new UI request. + let sender = bridge.inner.tx.lock().unwrap(); + let closing = bridge.clone(); + let detached = std::thread::spawn(move || closing.detach(old_token)); + let deadline = std::time::Instant::now() + Duration::from_secs(2); + loop { + if matches!( + bridge.inner.generation.try_lock(), + Err(std::sync::TryLockError::WouldBlock) + ) { + break; + } + assert!( + std::time::Instant::now() < deadline, + "detach did not claim the lifecycle lock" + ); + std::thread::yield_now(); + } + + let reloading = bridge.clone(); + let attached = std::thread::spawn(move || { + let (frames, token) = reloading.attach(); + let answer = reloading.register_ask("concurrent-reconnect-ask".to_string()); + assert!(reloading.emit_frame(json!({"message":"fresh connection"}))); + (frames, token, answer) + }); + drop(sender); + detached.join().unwrap(); + let (mut frames, token, mut answer) = attached.join().unwrap(); + assert_eq!(frames.try_recv().unwrap()["message"], "fresh connection"); + assert!(bridge.resolve("concurrent-reconnect-ask", json!({"answer":"kept"}))); + assert_eq!(answer.try_recv().unwrap(), json!({"answer":"kept"})); + bridge.detach(token); + } + /// Panels are per-connection (the reloaded DOM has none), but the semantic /// state bag survives and is replayed so the fresh page rehydrates. #[tokio::test] diff --git a/crates/biorouter-mcp/src/agent_drafter/export.rs b/crates/biorouter-mcp/src/agent_drafter/export.rs new file mode 100644 index 000000000..b72090472 --- /dev/null +++ b/crates/biorouter-mcp/src/agent_drafter/export.rs @@ -0,0 +1,3259 @@ +//! Exporting a Biorouter app as a folder that runs on another machine. +//! +//! **One implementation behind every door.** The `export_app` tool writes the +//! folder where it was told to ([`export_to_dir`]); the desktop's Export button +//! and a `biorouter serve` tab receive it as a zip that `POST /apps/{id}/export` +//! builds with [`export_to_staging`]; and the legacy `GET /apps/{id}/export` +//! answers older desktops with [`legacy_export_map`]. There used to be two +//! writers, and the desktop's silently ignored every option its dialog offered: +//! a "Full" export came out as a launcher export, and nothing said so. +//! +//! What a folder holds, and the failure each rule exists for: +//! - The **raw authored entry**, never an assembled page. Every way an export +//! runs is a daemon serving the page, and the daemon assembles it with this +//! run's socket token. A page assembled at export time carried a token-less +//! config island that the SDK read first, so no exported agentic app could +//! reach its agent, and running an export on the author's machine wrote that +//! page over the live app. +//! - Every other authored file **byte for byte** (`assets/logo.png`, fonts, +//! wasm). Reading files as text dropped every binary one without a word. +//! - Never `workspace/` (the app's runtime data), never a `.vault`, `.git` or +//! `node_modules` component anywhere, never a file named like a credential, +//! and never a symlink. ⚠ The vault exclusion used to match `/.vault/` +//! only, while the vault lives at `workspace/.vault/`: sealed secrets stayed +//! out solely because ciphertext is not valid UTF-8, so the binary-safe copy +//! above could not land without this. +//! - `manifest.json` without `session_id`. A chat id means nothing on another +//! machine, and there it can name an unrelated chat. +//! - The generated launchers, executable. An authored file with one of their +//! names moves under `app-files/`, so it can never replace a launcher. +//! - In full mode, `payload/` (knowledge bases and skills) and `export.json`. +//! An absent selection key means "everything the app references", an empty +//! list means "none", and each knowledge base is asked of the caller's reach +//! through [`PayloadAccess`], so a private base never leaves for a caller that +//! could not open it. + +use std::collections::BTreeSet; +use std::io; +use std::path::{Component, Path, PathBuf}; + +use serde::Serialize; + +use super::store::{validate_artifact_id, ArtifactKind, ArtifactStore, Manifest}; +use super::{bundle, render}; +use crate::knowledge::caller::KbCaller; +use crate::knowledge::service::KnowledgeService; + +/// Path components that never leave the machine, wherever they appear: the +/// app's sealed secrets, a repository's history (remote URLs carry tokens), and +/// installed packages. The same rule the app jail applies (`developer::jail`), +/// by component rather than by prefix. +const EXCLUDED_COMPONENTS: &[&str] = &[".vault", ".git", "node_modules"]; + +/// How deep an app or skill tree is followed. A real one is a few levels deep; +/// the bound is what makes a pathological tree end. +const MAX_DEPTH: usize = 24; +/// More files than any authored app has; an app past it is refused rather than +/// half-exported. +const MAX_APP_FILES: usize = 20_000; +/// A skill is a few documents and scripts. These bounds keep a skill folder that +/// happens to hold a dataset from turning an export into a copy of it. +const MAX_SKILL_FILES: usize = 5_000; +const MAX_SKILL_BYTES: u64 = 256 * 1024 * 1024; + +// --------------------------------------------------------------------------- +// The request +// --------------------------------------------------------------------------- + +/// What the exported folder carries besides the app. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum ExportMode { + /// The app and its launchers. It runs against whatever knowledge bases, + /// skills and extensions the recipient already has. + #[default] + Launcher, + /// Also the app's server-side payload under `payload/`, and `export.json`. + Full, +} + +impl ExportMode { + pub fn parse(value: &str) -> Option { + match value { + "launcher" => Some(Self::Launcher), + "full" => Some(Self::Full), + _ => None, + } + } + + pub fn as_str(self) -> &'static str { + match self { + Self::Launcher => "launcher", + Self::Full => "full", + } + } +} + +/// A full export's selection. `None` for a key means "everything the app +/// references"; `Some(vec![])` means "none of these". ⚠ The two must stay +/// distinct all the way to the wire: the desktop used to drop an empty list, so +/// unchecking a private knowledge base sent no key at all and the base shipped +/// anyway. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct ExportSelection { + pub knowledge_bases: Option>, + pub skills: Option>, + pub extensions: Option>, +} + +impl ExportSelection { + /// Read `{"knowledge_bases": [...], "skills": [...], "extensions": [...]}`. + /// A missing or `null` key is "everything referenced"; anything that is not + /// a list of strings is refused rather than read as "everything". + pub fn from_json(include: Option<&serde_json::Value>) -> Result { + let Some(include) = include.filter(|value| !value.is_null()) else { + return Ok(Self::default()); + }; + let Some(object) = include.as_object() else { + return Err( + "`include` must be an object with `knowledge_bases`, `skills` and `extensions` \ + lists" + .to_string(), + ); + }; + let list = |key: &str| -> Result>, String> { + match object.get(key) { + None | Some(serde_json::Value::Null) => Ok(None), + Some(serde_json::Value::Array(items)) => items + .iter() + .map(|item| { + item.as_str() + .map(str::to_string) + .ok_or_else(|| format!("`include.{key}` must list names as strings")) + }) + .collect::, _>>() + .map(Some), + Some(_) => Err(format!("`include.{key}` must be a list")), + } + }; + Ok(Self { + knowledge_bases: list("knowledge_bases")?, + skills: list("skills")?, + extensions: list("extensions")?, + }) + } +} + +/// One export, as every door asks for it. +#[derive(Debug, Clone, Default)] +pub struct ExportRequest { + pub mode: ExportMode, + /// Read in full mode only. + pub include: ExportSelection, + /// Copy this platform's `biorouterd` into `payload/bin/` ("fat" export). + pub bundle_daemon: bool, + /// Export even when the harness lint finds ERRORs. Only a person decides + /// this: the desktop asks them, and the model-facing tool never sets it. + pub allow_lint_errors: bool, +} + +// --------------------------------------------------------------------------- +// What the caller may carry out +// --------------------------------------------------------------------------- + +/// Where a skill lives: its folder, and the root that folder must stay inside. +#[derive(Debug, Clone)] +pub struct SkillLocation { + pub dir: PathBuf, + pub root: PathBuf, +} + +/// The two things a full export reaches outside the app, answered by the door +/// that asked. `biorouter-mcp` cannot depend on `biorouter`, so the HTTP route +/// plugs in its own reach decision (`HttpCaller`) and the daemon's skill +/// catalog, while the `export_app` tool uses [`DrafterPayload`]. +pub trait PayloadAccess { + /// `Ok` when this caller may carry knowledge base `kb_id` out of the + /// machine; `Err` is the sentence the export notes in its place. + fn reach_kb(&self, kb_id: &str) -> Result<(), String>; + /// The base as a `.brkb` archive. Asked only after [`Self::reach_kb`]. + fn export_kb(&self, kb_id: &str) -> Result, String>; + /// The knowledge store's root, or `None` when it cannot be opened. An + /// export target inside it is refused. + fn knowledge_root(&self) -> Option; + /// Where skill `name` (already validated by [`is_valid_item_name`]) lives. + fn skill_dir(&self, name: &str) -> Option; + /// Whether `name` is a skill Biorouter ships, which no export carries + /// ([`render::SHIPPED_SKILL_NAMES`]). The HTTP door asks the seeder's own + /// list instead, which a test holds equal to this one. + fn is_shipped_skill(&self, name: &str) -> bool { + render::SHIPPED_SKILL_NAMES + .iter() + .any(|shipped| shipped.eq_ignore_ascii_case(name)) + } +} + +/// The `export_app` tool's [`PayloadAccess`]: the calling chat's capability +/// (issue #56, CP4) against the knowledge store, and the installed-skills +/// folder. +pub struct DrafterPayload<'a> { + caller: &'a KbCaller, + knowledge: Option, +} + +impl<'a> DrafterPayload<'a> { + pub fn new(caller: &'a KbCaller) -> Self { + Self { + caller, + knowledge: knowledge_service_for_export(), + } + } +} + +impl PayloadAccess for DrafterPayload<'_> { + fn reach_kb(&self, kb_id: &str) -> Result<(), String> { + let Some(svc) = &self.knowledge else { + return Err("the knowledge store could not be opened".to_string()); + }; + // Issue #56 (CP4), Task 10C. `export_brkb` writes the WHOLE base into + // the payload, and the id can come from the model; skip-and-note rather + // than fail the export, matching `search_visible_bases`. + self.caller + .assert_reachable(svc.root(), kb_id) + .map_err(|e| e.to_string()) + } + + fn export_kb(&self, kb_id: &str) -> Result, String> { + let Some(svc) = &self.knowledge else { + return Err("the knowledge store could not be opened".to_string()); + }; + svc.export_brkb(kb_id).map_err(|e| e.to_string()) + } + + fn knowledge_root(&self) -> Option { + self.knowledge.as_ref().map(|svc| svc.root().to_path_buf()) + } + + fn skill_dir(&self, name: &str) -> Option { + skill_in_root(&skills_root_for_export(), name) + } +} + +/// Resolve a `KnowledgeService` for the drafter's exports. Honours +/// `BIOROUTER_KNOWLEDGE_DIR` (an override used by tests and power users); +/// otherwise the canonical store the `/knowledge` routes use. `None` when the +/// store cannot be opened, and the export then notes the bases it left out. +pub(crate) fn knowledge_service_for_export() -> Option { + if let Ok(dir) = std::env::var("BIOROUTER_KNOWLEDGE_DIR") { + if !dir.trim().is_empty() { + return Some(KnowledgeService::new(PathBuf::from(dir))); + } + } + KnowledgeService::new_default().ok() +} + +/// The installed-skills directory (`/skills`). Honours +/// `BIOROUTER_SKILLS_DIR` as a test/override hook. +pub fn skills_root_for_export() -> PathBuf { + if let Ok(dir) = std::env::var("BIOROUTER_SKILLS_DIR") { + if !dir.trim().is_empty() { + return PathBuf::from(dir); + } + } + crate::paths::in_config_dir("skills") +} + +/// A skill or extension name as one plain path component: +/// `^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$`. +/// +/// ⚠ This is what stops `include.skills: ["../../.ssh"]` — the join used to take +/// any string, so a name climbed out of the skills root on both the source and +/// the destination side, and an absolute name replaced the root outright (with +/// source and destination then the same file, which `fs::copy` truncates). +pub fn is_valid_item_name(name: &str) -> bool { + let bytes = name.as_bytes(); + (1..=128).contains(&bytes.len()) + && bytes[0].is_ascii_alphanumeric() + && bytes + .iter() + .all(|b| b.is_ascii_alphanumeric() || matches!(b, b'.' | b'_' | b'-')) +} + +/// Find skill `name` under `root`: `/`, else one bundle level down +/// (`//`), which is how skill discovery scans a root. The +/// direct path is returned whatever it is, so a link there is reported as a +/// link rather than as "not installed". +pub fn skill_in_root(root: &Path, name: &str) -> Option { + if !is_valid_item_name(name) { + return None; + } + let direct = root.join(name); + if std::fs::symlink_metadata(&direct).is_ok() { + return Some(SkillLocation { + dir: direct, + root: root.to_path_buf(), + }); + } + let mut bundles: Vec = std::fs::read_dir(root) + .ok()? + .flatten() + .filter(|entry| entry.file_type().is_ok_and(|kind| kind.is_dir())) + .map(|entry| entry.path()) + .collect(); + bundles.sort(); + bundles + .into_iter() + .map(|bundle| bundle.join(name)) + .find(|candidate| { + std::fs::symlink_metadata(candidate).is_ok_and(|meta| meta.is_dir()) + && std::fs::symlink_metadata(candidate.join("SKILL.md")) + .is_ok_and(|meta| meta.is_file()) + }) + .map(|dir| SkillLocation { + dir, + root: root.to_path_buf(), + }) +} + +// --------------------------------------------------------------------------- +// The result +// --------------------------------------------------------------------------- + +/// What an export did, for the tool's summary, the HTTP report header and the +/// desktop's toast. +#[derive(Debug, Clone, Default, Serialize)] +pub struct ExportReport { + pub id: String, + pub mode: &'static str, + /// Where the folder is, for a door that wrote one the caller chose. + pub target: String, + /// Files in the folder, and their bytes. + pub files: usize, + pub bytes: u64, + /// Knowledge bases, skills and external extension references the payload + /// carries. + pub kbs: Vec, + pub skills: Vec, + pub extensions: Vec, + /// The bundled daemon's `export.json` record, for a fat export. + pub daemon: Option, + /// Things the person should know: what was left out and why, a rename. + pub notes: Vec, + /// Harness findings that did not block the export. + pub lint_warnings: Vec, + /// App or skill files deliberately not exported, each with its reason. + pub skipped: Vec, + pub wrote_export_json: bool, + /// Every relative path written, sorted. For tests and the zip writer. + #[serde(skip)] + pub written: Vec, +} + +impl ExportReport { + fn new(id: &str, mode: ExportMode) -> Self { + Self { + id: id.to_string(), + mode: mode.as_str(), + ..Default::default() + } + } + + /// The report as the HTTP route sends it in a response header, without the + /// daemon's temporary path and capped at about 6 KB. + /// + /// ⚠ The cap is not cosmetic: Node's HTTP client refuses a response whose + /// headers exceed 16 KB, so a long list of skipped files would turn a good + /// export into a failed download. The lists are trimmed from the end, with + /// a line saying how many were left out. + pub fn header_json(&self) -> serde_json::Value { + const LIMIT: usize = 6_000; + let mut notes = self.notes.clone(); + let mut lint = self.lint_warnings.clone(); + let mut skipped = self.skipped.clone(); + let render = |notes: &[String], lint: &[String], skipped: &[String]| { + serde_json::json!({ + "id": self.id, + "mode": self.mode, + "files": self.files, + "bytes": self.bytes, + "kbs": self.kbs, + "skills": self.skills, + "extensions": self.extensions, + "daemon": self.daemon, + "notes": notes, + "lint_warnings": lint, + "skipped": skipped, + "wrote_export_json": self.wrote_export_json, + }) + }; + let mut dropped = 0usize; + loop { + let mut value = render(¬es, &lint, &skipped); + if value.to_string().len() <= LIMIT { + if dropped > 0 { + if let Some(list) = value["notes"].as_array_mut() { + list.push(serde_json::Value::String(format!( + "{dropped} more line(s) were left out of this summary" + ))); + } + } + return value; + } + // Halve the longest list (a handful of passes, however long it + // is); once every list is down to one line, halve the longest line. + let longest = [&mut skipped, &mut lint, &mut notes] + .into_iter() + .filter(|list| !list.is_empty()) + .max_by_key(|list| list.len()); + match longest { + Some(list) if list.len() > 1 => { + let keep = list.len() / 2; + dropped += list.len() - keep; + list.truncate(keep); + } + Some(list) => { + let line = &mut list[0]; + let mut cut = line.len() / 2; + while !line.is_char_boundary(cut) { + cut -= 1; + } + if cut == 0 { + list.clear(); + dropped += 1; + } else { + line.truncate(cut); + line.push('…'); + } + } + None => return render(&[], &[], &[]), + } + } + } + + /// The `export_app` tool's answer to the model. + pub fn summary(&self) -> String { + let mut msg = format!( + "Exported '{}' ({} mode) to {} ({} files). To run it: double-click run.command \ + (macOS), `bash run.sh` (Linux), or run.bat (Windows). That installs the app, \ + starts a biorouterd if one isn't already up, and opens it in the browser, with no \ + npm install and no build step.", + self.id, self.mode, self.target, self.files + ); + if self.mode == ExportMode::Full.as_str() { + msg.push_str(&format!( + "\nPayload: {} knowledge base(s), {} skill(s), {} external extension \ + reference(s).", + self.kbs.len(), + self.skills.len(), + self.extensions.len() + )); + } + if self.wrote_export_json { + msg.push_str(" Wrote export.json (audit manifest)."); + } + for note in &self.notes { + msg.push_str(&format!("\n- {note}")); + } + if !self.skipped.is_empty() { + msg.push_str("\nNot exported:"); + for line in self.skipped.iter().take(20) { + msg.push_str(&format!("\n- {line}")); + } + if self.skipped.len() > 20 { + msg.push_str(&format!("\n- and {} more", self.skipped.len() - 20)); + } + } + msg + } +} + +/// Why an export did not happen. Each carries the sentence the person reads. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ExportError { + /// There is no such app (HTTP 404). + NotFound(String), + /// The request or the target is unusable (HTTP 400). + BadRequest(String), + /// The harness lint found ERRORs and nobody chose to export anyway + /// (HTTP 422, with the findings). + Lint { + message: String, + findings: Vec, + }, + /// The app itself cannot be exported as it stands: its entry is missing or + /// unsafe, or its manifest does not load (HTTP 409). + Unexportable(String), + /// Reading the app or writing the folder failed (HTTP 500). + Io(String), +} + +impl std::fmt::Display for ExportError { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::NotFound(m) | Self::BadRequest(m) | Self::Unexportable(m) | Self::Io(m) => { + f.write_str(m) + } + Self::Lint { message, .. } => f.write_str(message), + } + } +} + +impl std::error::Error for ExportError {} + +fn io_error(what: &str, error: impl std::fmt::Display) -> ExportError { + ExportError::Io(format!("{what}: {error}")) +} + +// --------------------------------------------------------------------------- +// Entry points +// --------------------------------------------------------------------------- + +/// Build the export of app `id` into `out`, a folder that must not exist yet +/// and whose parent must. For a door that owns `out` — the HTTP route builds into +/// a temporary directory of its own and zips it. +pub fn export_to_staging( + store_root: &Path, + id: &str, + request: &ExportRequest, + access: &dyn PayloadAccess, + out: &Path, +) -> Result { + build_export(store_root, id, request, access, out) +} + +/// Export app `id` into `target`, a folder a model named. +/// +/// The target is refused unless it is absolute, outside the Biorouter config +/// folder, the app store and the knowledge store, and a folder of the person's +/// own ([`placement_refusal`]: below Desktop, Documents or Downloads, never +/// hidden or a folder code is imported from); and unless it does not exist, +/// or is empty. With `replace`, a +/// non-empty target is accepted only when it is an earlier export of this same +/// app (its `manifest.json` names `id` and it has `biorouter-launch.sh`) that +/// holds nothing the export does not write ([`foreign_entries`]). +/// +/// ⚠ The folder is built beside the target and renamed into place, so a failed +/// export leaves nothing half-written, a replaced export is swapped whole (no +/// stale `payload/` or deleted `src/` files survive), and nothing is ever +/// written through a link that was already in the target. The replaced export +/// is kept beside it, renamed `.replaced-