diff --git a/images/macos/README.md b/images/macos/README.md index 65cfae9..20d92c9 100644 --- a/images/macos/README.md +++ b/images/macos/README.md @@ -32,10 +32,10 @@ current, matches the host's own build, and has a publisher-independent digest. | Piece | Where | | --- | --- | -| Lume 0.5.3 | `/usr/local/bin/lume` plus `/usr/local/bin/lume.app` (the wrapper execs the bundle) | +| Backend Lume | `/Users/agentcompute/bin/lume`; account-local source build and installed SHA-256 in [`pins/lume.yaml`](../../pins/lume.yaml) | | Backend account | `agentcompute`, standard (not admin), hidden, member of `com.apple.access_ssh` | | VM store | `/Users/agentcompute/.lume` | -| API | `lume serve --port 7777`, `/Library/LaunchDaemons/io.gilman.agentcompute.lume-serve.plist`, bound to `127.0.0.1` only | +| API | `/Users/agentcompute/bin/lume serve --port 7777`, `/Library/LaunchDaemons/io.gilman.agentcompute.lume-serve.plist`, bound to `127.0.0.1` only | | Approved server key policy | `from="",no-agent-forwarding,no-X11-forwarding`; normal shell and local TCP forwarding | | Root-owned account policy | `/etc/ssh/sshd_config.d/110-agentcompute.conf` (`Match User agentcompute`) | | Guest key | Per-image key on the server; the seed's provisioned key is `/Users/agentcompute/.ssh/guest_ed25519` on Studio | @@ -204,8 +204,8 @@ Machine-readable evidence is in Qualification used a runtime-only systemd override, then restored the fleet-pinned v0.1.1 binary, configuration, and catalog. Temporary credentials and binaries were removed. This is qualification evidence, not a permanent -Lume rollout. PF and tailnet ACLs were not changed; the VNC restriction below -remains an operator rollout prerequisite. +Lume rollout. PF and tailnet ACLs were not changed. Phase 9b replaces the +proposed high-port firewall gate with disabled VNC at its source, below. Three live-only defects were fixed and regression-tested: SSH negotiated an unpinned host-key algorithm; Lume rejected an equal-size disk PATCH; and zsh @@ -218,70 +218,31 @@ One concurrent-listing finding remains outside this qualification fix: a listed sandbox before its details are read. This occurred during parallel live-lane teardown; the sequential lifecycle and TTL checks passed. -## Restrict VNC before serving workers - -Lume's VNC listener uses an ephemeral TCP port, not port 5900. The observed -host range is 49152–65535 (`sysctl net.inet.ip.portrange.first -net.inet.ip.portrange.last`). `pf.anchor` admits that range only through -loopback and the current Tailscale interface. - -**Impact:** this range also contains non-Lume applications. Review -`sudo lsof -nP -iTCP -sTCP:LISTEN` before applying it. Do not apply it if -another application requires LAN access in this range; resolve that conflict -with the owner first. These rules were syntax-checked, not installed during -the identity experiment. - -1. Stop all worker VMs through the HTTP API so no pre-existing VNC states - survive the policy change. Leave the seed stopped. -2. Find the current Tailscale interface with - `route -n get `; use its `interface` value below. - It was `utun9` during the experiment, but the number can change. -3. Validate and install only the anchor file: - - ```sh - tailnet_if=utun9 # replace with the observed interface - sudo pfctl -n -D "tailnet_if=$tailnet_if" -f images/macos/pf.anchor - sudo install -o root -g wheel -m 0644 images/macos/pf.anchor \ - /etc/pf.anchors/io.gilman.agentcompute - ``` - -4. Back up `/etc/pf.conf`. Add `anchor "io.gilman.agentcompute" quick` - immediately before its existing `anchor "com.apple/*"` filter anchor. - Preserve every existing scrub, NAT, redirect, and Apple anchor. Compare - the live root rules (`sudo pfctl -sr`, `sudo pfctl -sn`) with the file: - reloading it can discard dynamically inserted root rules. If they differ, - stop and reconcile with the owner rather than losing another service's - rules. Check `sudo pfctl -nf /etc/pf.conf` before the maintenance-window - reload with `sudo pfctl -f /etc/pf.conf`. Never use a global flush. -5. Load the child rules: - - ```sh - sudo pfctl -a io.gilman.agentcompute \ - -D "tailnet_if=$tailnet_if" -f /etc/pf.anchors/io.gilman.agentcompute - sudo pfctl -a io.gilman.agentcompute -sr - sudo pfctl -s info - ``` - - If PF is disabled, enable it with `sudo pfctl -E` and retain its reference - token. Never use `pfctl -d`: other host services also depend on PF. -6. Start one disposable worker. Read its actual `vncUrl` port and verify a - fresh TCP connection from loopback and an allowed tailnet peer succeeds, - while fresh connections to both LAN addresses fail. Also test IPv6 if a - wildcard IPv6 listener exists. Stop the worker after checking. - -The child rules must be reloaded after a reboot or Tailscale interface -change, **before** starting workers. The main anchor declaration alone does -not load them. An unattended host boot loader is not installed by this -procedure. If the interface cannot be identified or the anchor is absent, -do not serve workers. - -Rollback: stop workers, then unload only this anchor with -`sudo pfctl -a io.gilman.agentcompute -F rules`. Release only the PF reference -token acquired by this procedure with `sudo pfctl -X `. Restore the -saved main configuration if its declaration must also be removed. - -Upstream bind-address request: -[trycua/cua#3878](https://github.com/trycua/cua/issues/3878). +## Disable VNC before serving workers + +Released Lume 0.5.3 opens a wildcard VNC listener even with `--display none`. +The owner rejected a high-port PF block because it would also affect +Continuity and `rapportd`. The obsolete anchor is removed; do not install it. + +The backend instead requires the account-local source build in +[`pins/lume.yaml`](../../pins/lume.yaml), which includes upstream +[cua#3209](https://github.com/trycua/cua/pull/3209). Every run sends +`{"noDisplay":true,"vnc":"disabled"}`. Startup checks both the CLI and daemon +and fails closed if either lacks the policy. Inventory must report no VNC URL. +The global 0.5.3 install remains the seed qualification/release rollback +reference, not an automatic fallback for backend guests. + +`build-lume.sh` runs only as the standard `agentcompute` account and refuses +to install a binary whose SHA-256 differs from the pin. Its source commit and +dependencies are pinned, but a measured clean rebuild is **not bitwise +reproducible**; a deliberate rebuild needs a reviewed artifact re-pin. +The version string remains 0.5.3, so verify the installed path and hash. + +The canonical [agentcompute runbook](https://github.com/GilmanLab/root/blob/master/docs/docs/runbooks/agentcompute.md) +owns daemon activation, listener verification from request through Running, +credential delivery, and manual console fallback. No PF or Internet Sharing +change is required. Return to a release pin when upstream releases the +disabled-VNC feature; the pin records that revisit condition. ## Build a seed @@ -297,18 +258,20 @@ read_pin() { ( . images/macos/lib.sh; pin "$pins" "$1" "$2" ); } ### 1. Install the pinned Lume +Follow `images/macos/build-lume.sh`'s staging instructions so the service +account can read the script without access to the owner's home. Run the build +as `agentcompute`, never root. The build touches only that account's home. + ```sh -curl -fsSL -o /tmp/lume.tar.gz "$(read_pin lume url)" -shasum -a 256 /tmp/lume.tar.gz # must equal $(read_pin lume sha256) -tar xzf /tmp/lume.tar.gz -C /tmp -sudo ditto /tmp/lume.app /usr/local/bin/lume.app -sudo install -m 0755 /tmp/lume /usr/local/bin/lume -lume --version # 0.5.3 -sudo -u agentcompute -H lume config telemetry disable +sudo -u agentcompute -H /tmp/lume-build/images/macos/build-lume.sh +export PATH=/Users/agentcompute/bin:$PATH +lume run --help ``` -The archive ships a shell wrapper plus `lume.app`; both must land in the same -directory or the wrapper cannot find the binary. +Require the distinct `--vnc ` option, not just `--vnc-port`. Repoint the +account's daemon using the canonical runbook before cloning a worker. The +released archive remains pinned in `images/macos/pins.lock.yaml` as the +historical seed-build input and in `pins/lume.yaml` as a rollback reference. ### 2. Fetch and verify the IPSW @@ -462,7 +425,7 @@ Never run it while the clone is running. Then start through HTTP: ```sh curl -sS -X POST http://127.0.0.1:7777/lume/vms/ac-smoke-1/run -H 'Content-Type: application/json' \ - -d '{"noDisplay":true}' + -d '{"noDisplay":true,"vnc":"disabled"}' ``` Wait until `GET /lume/vms/ac-smoke-1` reports a NAT address and SSH accepts @@ -514,7 +477,7 @@ experiment did not modify the seed or grant new permissions. ## Host-wide guest-count rule Before a create/start request makes any Lume HTTP call, inspect the dedicated -account's inventory with `/usr/local/bin/lume ls --format json`. Count running +account's inventory with `/Users/agentcompute/bin/lume ls --format json`. Count running **macOS** guests regardless of name, including a running seed, and serialize pending starts. Refuse a third Lume-owned running macOS guest: diff --git a/images/macos/build-lume.sh b/images/macos/build-lume.sh new file mode 100755 index 0000000..accb0e9 --- /dev/null +++ b/images/macos/build-lume.sh @@ -0,0 +1,157 @@ +#!/usr/bin/env bash +# Build and install the account-local Lume that can start a VM with no VNC +# listener, from the exact upstream commit pinned in pins/lume.yaml. +# +# Runs on the Apple Silicon host, AS the service account that owns the VMs +# (agentcompute), never as root and never through the global install. It +# touches only paths under that account's home: the global +# /usr/local/bin/lume (released 0.5.3) is left exactly as it is, and no +# service, LaunchDaemon, LaunchAgent, or VM is started here. +# +# Idempotent: re-running refetches the same commit, rebuilds in the same +# SwiftPM cache, and replaces the installed bundle in place. +# +# Usage, as the service account: +# sudo -u agentcompute -H /images/macos/build-lume.sh +# sudo -u agentcompute -H /images/macos/build-lume.sh --pins +# +# The service account cannot read a developer's home directory, so a checkout +# under ~/code is not reachable from it. Stage the three files it needs in a +# world-readable directory, keeping the layout so the default --pins path +# resolves: +# +# mkdir -p /tmp/lume-build/images/macos /tmp/lume-build/pins +# cp images/macos/build-lume.sh images/macos/lib.sh /tmp/lume-build/images/macos/ +# cp pins/lume.yaml /tmp/lume-build/pins/ +# chmod -R a+rX /tmp/lume-build +set -euo pipefail + +here=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) +# shellcheck source=images/macos/lib.sh +. "$here/lib.sh" + +pins="$here/../../pins/lume.yaml" + +while [ $# -gt 0 ]; do + case "$1" in + --pins) + pins=${2:?--pins needs a file} + shift 2 + ;; + *) die "unknown argument: $1" ;; + esac +done + +[ "$(id -u)" -ne 0 ] || die "run as the service account that owns the VMs, not as root" +require_cmd git swift codesign shasum ditto xattr awk sed + +commit=$(pin "$pins" lume source_commit) +source_url=$(pin "$pins" lume source_url) +subpath=$(pin "$pins" lume source_subpath) +entitlements=$(pin "$pins" lume entitlements_file) +want_sha=$(pin "$pins" lume installed_sha256) +want_wrapper=$(pin "$pins" lume wrapper_sha256) +src=$(pin "$pins" lume source_dir) +app_dir=$(pin "$pins" lume app_install_dir) +install_bin=$(pin "$pins" lume install_bin) + +case "$commit" in + [0-9a-f]*) [ ${#commit} -eq 40 ] || die "$pins: lume.source_commit must be a full 40-character SHA" ;; + *) die "$pins: lume.source_commit must be a full 40-character SHA" ;; +esac +for path in "$src" "$app_dir" "$install_bin"; do + case "$path" in + "$HOME"/*) ;; + *) die "$path is outside $HOME; this build never writes outside the service account" ;; + esac +done + +log "fetching $commit from $source_url into $src" +mkdir -p "$src" +cd "$src" +[ -d .git ] || git init -q +git remote get-url origin >/dev/null 2>&1 || git remote add origin "$source_url" +# A blobless, single-commit, sparse fetch: the monorepo is large and only +# libs/lume is built here. +git fetch --depth 1 --filter=blob:none origin "$commit" +git sparse-checkout set --cone "$subpath" +git checkout -q --detach "$commit" +[ "$(git rev-parse HEAD)" = "$commit" ] || die "checkout is not $commit" + +lume_dir="$src/$subpath" +cd "$lume_dir" +[ -r Package.resolved ] || die "$lume_dir/Package.resolved is missing; refusing to resolve dependencies freely" + +log "building lume (release) with the dependency versions in Package.resolved" +swift build -c release --product lume --only-use-versions-from-resolved-file + +# Assemble the .app the way upstream's own release build does: the resource +# bundle must sit in Contents/Resources, or codesign rejects the layout. +build_path="$lume_dir/.build/release" +app="$build_path/lume.app" +rm -rf "$app" +mkdir -p "$app/Contents/MacOS" "$app/Contents/Resources" +cp -f "$build_path/lume" "$app/Contents/MacOS/lume" +[ -d "$build_path/lume_lume.bundle" ] || die "resource bundle lume_lume.bundle was not built" +cp -Rf "$build_path/lume_lume.bundle" "$app/Contents/Resources/" +cp -f "$lume_dir/resources/AppIcon.icns" "$app/Contents/Resources/AppIcon.icns" +sed "s/__VERSION__/$(cat "$lume_dir/VERSION")/g" "$lume_dir/resources/Info.plist" \ + >"$app/Contents/Info.plist" + +# Ad-hoc signature, local entitlements. `com.apple.vm.networking` is +# deliberately absent: it is an Apple-granted entitlement, and macOS kills an +# ad-hoc binary that carries it. NAT networking, which this deployment uses, +# needs only com.apple.security.virtualization. No provisioning profile is +# embedded for the same reason. +# +# The bundle is signed ONCE, with the entitlements. Signing the bundle also +# signs its main executable, so entitling the inner Mach-O first (as +# scripts/build/build-release.sh does) is pointless: the bundle signature +# overwrites it and the installed binary ends up with no entitlements at all. +# Upstream's notarized release path signs the bundle with entitlements, and +# that is the order copied here. +log "signing with $entitlements" +xattr -cr "$app" 2>/dev/null || true +codesign --force --entitlements "$lume_dir/$entitlements" --sign - "$app" +codesign --verify --strict "$app" +codesign -d --entitlements - --xml "$app" 2>/dev/null | + grep -q com.apple.security.virtualization || + die "signed bundle carries no virtualization entitlement" + +# The pin names the artifact this host is supposed to run, so the hash is +# checked BEFORE anything is installed: a build that disagrees with the pin +# never reaches the service account's install paths. Re-pinning is a decision +# for a human, not a warning to scroll past. +got_sha=$(shasum -a 256 "$app/Contents/MacOS/lume" | awk '{print $1}') +[ "$got_sha" = "$want_sha" ] || die "built binary is $got_sha, but $pins pins $want_sha +Nothing was installed. If this build is the one you intend to deploy, re-read +the commit, then set lume.installed_sha256 to $got_sha in $pins and say why in +the commit message." + +log "installing to $app_dir and $install_bin" +mkdir -p "$app_dir" "$(dirname "$install_bin")" +rm -rf "$app_dir/lume.app" +# ditto, not cp: it preserves the signature and its extended attributes. +ditto "$app" "$app_dir/lume.app" +# Replace the inode so taskgated cannot reuse a stale signature cache. +rm -f "$install_bin" +cat >"$install_bin" </dev/null | awk '/--vnc apiBodyLimit { - return fmt.Errorf("lume api %s %s: response too large", method, path) + return 0, nil, fmt.Errorf("lume api %s %s: response too large", method, path) } - - return decodeAPIResponse(method, path, resp.StatusCode, payload, result) + return resp.StatusCode, payload, nil } func decodeAPIResponse(method, path string, status int, payload []byte, result any) error { diff --git a/internal/lume/client_test.go b/internal/lume/client_test.go index 6cbaa91..5ce4550 100644 --- a/internal/lume/client_test.go +++ b/internal/lume/client_test.go @@ -233,7 +233,7 @@ func TestHostCallbackRunsBeforeShell(t *testing.T) { return false, nil }, ) - out, err := client.host(t.Context(), "/usr/local/bin/lume ls --format json", nil) + out, err := client.host(t.Context(), lumeBin+" ls --format json", nil) require.NoError(t, err) assert.Contains(t, string(out), `"name":"fake"`) diff --git a/internal/lume/fixture_test.go b/internal/lume/fixture_test.go index 836504f..45455f6 100644 --- a/internal/lume/fixture_test.go +++ b/internal/lume/fixture_test.go @@ -17,6 +17,8 @@ import ( "os" "os/exec" "path/filepath" + "slices" + "strings" "sync" "syscall" "testing" @@ -55,20 +57,39 @@ type tunnelOpts struct { GuestHostKey ssh.PublicKey Seed string Image string + + // RunHelp replaces the `lume run --help` output the startup capability + // gate reads. Empty means the pinned source build. + RunHelp string + + // LegacyDaemon makes the fake lume serve behave like 0.5.3: it drops the + // unknown vnc key and accepts the run-policy probe. + LegacyDaemon bool } type tunnelState struct { client *Client home string + daemon *vncPolicyDaemon } func startTunnel(t *testing.T, opts tunnelOpts) *tunnelState { t.Helper() + state, err := dialTunnel(t, opts) + require.NoError(t, err) + return state +} + +// dialTunnel builds the fixture and returns New's error so the fail-closed +// startup contract can be exercised directly. +func dialTunnel(t *testing.T, opts tunnelOpts) (*tunnelState, error) { + t.Helper() if opts.API == nil { opts.API = http.NotFoundHandler() } - httpServer := httptest.NewServer(opts.API) + daemon := &vncPolicyDaemon{next: opts.API, legacy: opts.LegacyDaemon} + httpServer := httptest.NewServer(daemon) t.Cleanup(httpServer.Close) home := filepath.Join(t.TempDir(), "home") @@ -113,7 +134,7 @@ func startTunnel(t *testing.T, opts tunnelOpts) *tunnelState { natDial: opts.GuestDial, home: home, user: hostUser, - hostFn: opts.Host, + hostFn: capableHost(opts.Host, opts.RunHelp), } var acceptWG sync.WaitGroup acceptWG.Go(func() { @@ -155,9 +176,122 @@ func startTunnel(t *testing.T, opts tunnelOpts) *tunnelState { GuestKnownHostsFile: guestKnown, GuestKeys: keys, }) - require.NoError(t, err) + if err != nil { + return nil, err + } t.Cleanup(func() { _ = client.Close() }) - return &tunnelState{client: client, home: home} + return &tunnelState{client: client, home: home, daemon: daemon}, nil +} + +// vncCapableRunHelp is the option block the pinned Lume source build prints +// for `lume run --help`. +const vncCapableRunHelp = `OPTIONS: + --no-display Do not open a display window + --vnc VNC server policy: 'enabled' (default) or + 'disabled'. + --vnc-port Port to use for the VNC server. + --vnc-password Password for the VNC server. + -h, --help Show help information. +` + +// vncLegacyRunHelp is Lume 0.5.3: the port and password flags exist, the +// policy flag does not. +const vncLegacyRunHelp = `OPTIONS: + --no-display Do not open a display window + --vnc-port Port to use for the VNC server. + --vnc-password Password for the VNC server. + -h, --help Show help information. +` + +// capableHost supplies the host-side halves of the startup capability gate — +// the pinned CLI's help output, and an empty inventory for tests that bring +// no VMs of their own — after the test's own script handler has had its say. +func capableHost(next hostScriptHandler, runHelp string) hostScriptHandler { + if runHelp == "" { + runHelp = vncCapableRunHelp + } + return func(script string, stdin io.Reader, stdout, stderr io.Writer) (bool, error) { + if next != nil { + if handled, err := next(script, stdin, stdout, stderr); handled || err != nil { + return handled, err + } + } + if strings.Contains(script, lumeBin+" run --help") { + _, _ = io.WriteString(stdout, runHelp) + return true, nil + } + if strings.Contains(script, lumeBin+" ls --format json") { + _, _ = io.WriteString(stdout, "[]") + return true, nil + } + return false, nil + } +} + +type vncProbe struct { + VM string + NoDisplay bool + Policy string +} + +// vncPolicyDaemon answers the VNC run-policy probe the way the pinned Lume +// build does and forwards every other request to the test's own handler. With +// legacy set it answers like 0.5.3, which has no vnc field at all, drops the +// unknown key, and accepts the run. +type vncPolicyDaemon struct { + next http.Handler + legacy bool + + mu sync.Mutex + probes []vncProbe +} + +func (d *vncPolicyDaemon) ServeHTTP(w http.ResponseWriter, r *http.Request) { + name, ok := vncProbeTarget(r) + if !ok { + d.next.ServeHTTP(w, r) + return + } + var body struct { + NoDisplay bool `json:"noDisplay"` + VNC string `json:"vnc"` + } + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + http.Error(w, `{"message":"Invalid run request body"}`, http.StatusBadRequest) + return + } + d.mu.Lock() + d.probes = append(d.probes, vncProbe{VM: name, NoDisplay: body.NoDisplay, Policy: body.VNC}) + d.mu.Unlock() + if d.legacy { + w.WriteHeader(http.StatusAccepted) + _, _ = w.Write([]byte(`{"message":"VM start initiated","status":"pending"}`)) + return + } + if body.NoDisplay || body.VNC != vncPolicyDisabled { + http.Error(w, `{"message":"probe must combine a disabled policy with a display"}`, + http.StatusUnprocessableEntity) + return + } + http.Error(w, `{"message":"VNC is disabled for this run, so '--display vnc' cannot be used."}`, + http.StatusBadRequest) +} + +func (d *vncPolicyDaemon) probeCalls() []vncProbe { + d.mu.Lock() + defer d.mu.Unlock() + return slices.Clone(d.probes) +} + +func vncProbeTarget(r *http.Request) (string, bool) { + if r.Method != http.MethodPost || !strings.HasSuffix(r.URL.Path, "/run") { + return "", false + } + name := strings.TrimSuffix(strings.TrimPrefix(r.URL.Path, "/lume/vms/"), "/run") + if !strings.HasPrefix(name, vncProbeVM) { + return "", false + } + return name, true } type sshTestServer struct { diff --git a/internal/lume/instance.go b/internal/lume/instance.go index 51b1ec7..7e78ee5 100644 --- a/internal/lume/instance.go +++ b/internal/lume/instance.go @@ -634,7 +634,7 @@ func (c *Client) getVM(ctx context.Context, name string) (lumeVM, error) { func (c *Client) lumeList(ctx context.Context) ([]lumeVM, error) { out, err := c.host(ctx, `set -eu -/usr/local/bin/lume ls --format json +`+lumeBin+` ls --format json `, nil) if err != nil { return nil, err @@ -882,17 +882,31 @@ func (c *Client) waitVMStatus(ctx context.Context, name, want string) error { } func (c *Client) runLocked(ctx context.Context, name string, inventory []lumeVM) error { + // An already-running guest is only accepted when its own run left no + // listener behind; one started outside this backend can still be holding + // a wildcard VNC port open. if vm, ok := findVM(inventory, name); ok && vm.running() { - return nil + return c.requireNoVNCListener(ctx, name, vm) } if countRunningMacOS(inventory) >= maxMacOSGuests { return agentError(errLimitOwned) } + // A daemon that was replaced or rolled back under a live process would + // answer the run below with the same 202 either way, so the policy is + // re-proven immediately before the start that depends on it. The probe + // runs before the log cursor so its own rejection never lands inside the + // window stoppedRunError reads. + if err := c.requireVNCPolicyEnforced(ctx, inventory); err != nil { + return err + } cursor, err := c.serveLogCursor(ctx) if err != nil { return err } - body := map[string]any{"noDisplay": true} + // noDisplay stays true: Lume treats a display as an implicit VNC client + // and rejects the disabled policy when it is combined with one. It is the + // vnc field, not noDisplay, that keeps the listener from ever starting. + body := map[string]any{"noDisplay": true, "vnc": vncPolicyDisabled} if err := c.api(ctx, http.MethodPost, "/lume/vms/"+name+"/run", body, nil); err != nil { return err } @@ -906,7 +920,7 @@ func (c *Client) runLocked(ctx context.Context, name string, inventory []lumeVM) return err } if vm.running() { - return nil + return c.requireNoVNCListener(ctx, name, vm) } if time.Now().After(deadline) { return c.stoppedRunError(ctx, name, cursor) diff --git a/internal/lume/lifecycle_test.go b/internal/lume/lifecycle_test.go index 01a9bad..e80f787 100644 --- a/internal/lume/lifecycle_test.go +++ b/internal/lume/lifecycle_test.go @@ -8,6 +8,7 @@ import ( "io" "net/http" "os" + "slices" "strconv" "strings" "sync" @@ -25,6 +26,19 @@ type lifecycleHost struct { mu sync.Mutex vms map[string]lumeVM calls []string + runs []lumeRun + // vncOnRun makes a started guest report a VNC endpoint, the exposure the + // disabled run policy exists to prevent. + vncOnRun string + // onRun fires once a run request has been recorded, letting a test end + // the guest-readiness wait that no fake host can satisfy. + onRun func() +} + +type lumeRun struct { + VM string + NoDisplay bool + Policy string } func (h *lifecycleHost) ServeHTTP(w http.ResponseWriter, r *http.Request) { @@ -60,6 +74,9 @@ func (h *lifecycleHost) ServeHTTP(w http.ResponseWriter, r *http.Request) { delete(h.vms, name) w.WriteHeader(http.StatusOK) return + case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/run"): + h.runVM(w, r) + return case r.Method == http.MethodPost && strings.HasSuffix(r.URL.Path, "/stop"): name := strings.TrimPrefix(strings.TrimSuffix(r.URL.Path, "/stop"), "/lume/vms/") vm := h.vms[name] @@ -109,8 +126,43 @@ func (h *lifecycleHost) patchVM(w http.ResponseWriter, r *http.Request) { _ = json.NewEncoder(w).Encode(map[string]string{"message": "updated"}) } +func (h *lifecycleHost) runVM(w http.ResponseWriter, r *http.Request) { + var body struct { + NoDisplay bool `json:"noDisplay"` + VNC string `json:"vnc"` + } + if err := json.NewDecoder(r.Body).Decode(&body); err != nil { + http.Error(w, err.Error(), http.StatusBadRequest) + return + } + name := strings.TrimPrefix(strings.TrimSuffix(r.URL.Path, "/run"), "/lume/vms/") + vm, ok := h.vms[name] + if !ok { + http.Error(w, `{"message":"not found"}`, http.StatusBadRequest) + return + } + h.runs = append(h.runs, lumeRun{VM: name, NoDisplay: body.NoDisplay, Policy: body.VNC}) + vm.Status = vmStatusRunning + vm.IPAddress = new(testGuestNAT) + if h.vncOnRun != "" { + vm.VNCURL = new(h.vncOnRun) + } + h.vms[name] = vm + if h.onRun != nil { + h.onRun() + } + w.WriteHeader(http.StatusAccepted) + _, _ = w.Write([]byte(`{"message":"VM start initiated","status":"pending"}`)) +} + +func (h *lifecycleHost) runCalls() []lumeRun { + h.mu.Lock() + defer h.mu.Unlock() + return slices.Clone(h.runs) +} + func (h *lifecycleHost) host(script string, _ io.Reader, stdout, _ io.Writer) (bool, error) { - if strings.Contains(script, "/usr/local/bin/lume ls --format json") { + if strings.Contains(script, lumeBin+" ls --format json") { h.mu.Lock() defer h.mu.Unlock() vms := make([]lumeVM, 0, len(h.vms)) diff --git a/internal/lume/snapshot_test.go b/internal/lume/snapshot_test.go index 24dcbe0..f72bffd 100644 --- a/internal/lume/snapshot_test.go +++ b/internal/lume/snapshot_test.go @@ -122,7 +122,7 @@ func (h *snapshotHost) ServeHTTP(w http.ResponseWriter, r *http.Request) { } func (h *snapshotHost) host(script string, _ io.Reader, stdout, _ io.Writer) (bool, error) { - if !strings.Contains(script, "/usr/local/bin/lume ls --format json") { + if !strings.Contains(script, lumeBin+" ls --format json") { return false, nil } h.mu.Lock() diff --git a/internal/lume/vnc.go b/internal/lume/vnc.go new file mode 100644 index 0000000..567a757 --- /dev/null +++ b/internal/lume/vnc.go @@ -0,0 +1,149 @@ +package lume + +import ( + "context" + "errors" + "fmt" + "net/http" + "strings" + + "github.com/GilmanLab/agentcompute/internal/compute" +) + +const ( + // lumeBin is the account-local Lume this backend drives. The global + // /usr/local/bin/lume is 0.5.3, which always starts a per-VM VNC server + // bound to every interface and offers no way to turn it off, so the + // backend never touches it. + lumeBin = "/Users/agentcompute/bin/lume" + + // vncPolicyDisabled is the run-policy value that starts a guest with no + // VNC listener, publishes no credentials, and reports a null vncUrl. + vncPolicyDisabled = "disabled" + + // vncPinFile is the single source of installation and provenance details. + vncPinFile = "pins/lume.yaml" + + // vncCLIOption is how the pinned build renders the run flag in help + // output. 0.5.3 already ships --vnc-port and --vnc-password, so the value + // placeholder is what separates the policy flag from those. + vncCLIOption = "--vnc " + + // vncConflictMarker is the pinned daemon's own rejection of a disabled + // policy combined with a display, lowercased for matching. It comes from + // VMError.vncDisabledConflict, a LocalizedError, so the HTTP body carries + // this sentence rather than an opaque bridged description. + vncConflictMarker = "vnc is disabled for this run" + + // vncProbeVM prefixes the throwaway VM name used by the daemon probe. + vncProbeVM = "ac-vnc-policy-probe-" +) + +// requireVNCDisableSupport fails closed unless both halves of the deployment +// understand the disabled VNC run policy: the account-local CLI this backend +// shells out to for inventory, and the lume serve daemon behind the tunnel +// that actually starts guests. Checking only the binary on disk would pass on +// a host where launchd still serves the old 0.5.3 daemon. +func (c *Client) requireVNCDisableSupport(ctx context.Context) error { + if err := c.requireVNCCLIOption(ctx); err != nil { + return err + } + inventory, err := c.lumeList(ctx) + if err != nil { + return err + } + return c.requireVNCPolicyEnforced(ctx, inventory) +} + +// requireVNCCLIOption checks the installed executable's policy capability. +func (c *Client) requireVNCCLIOption(ctx context.Context) error { + script := `set -eu +if [ ! -x ` + quote(lumeBin) + ` ]; then + echo 'not installed or not executable' >&2 + exit 1 +fi +` + lumeBin + ` run --help +` + out, err := c.host(ctx, script, nil) + if err != nil { + // An unreachable host is an outage, not an unpinned install; saying + // otherwise sends the operator to reinstall a working Lume. + if errors.Is(err, compute.ErrUnavailable) { + return err + } + return vncUnsupported(fmt.Sprintf("%s run --help failed: %v", lumeBin, err)) + } + if !strings.Contains(string(out), vncCLIOption) { + return vncUnsupported(fmt.Sprintf("%s run has no %q option", lumeBin, vncCLIOption)) + } + return nil +} + +// requireVNCPolicyEnforced proves the running daemon parses the run policy +// instead of dropping it. +// +// The probe asks for a disabled policy together with a display, which the +// pinned daemon rejects while decoding the request — before it looks up, +// creates, or starts anything. Lume 0.5.3 has no vnc field at all, silently +// drops the unknown key, and answers 202, which is exactly the silent +// fail-open this guards against. The probe name is random, verified absent +// from inventory, and colon-free so 0.5.3's image-reference auto-pull path +// cannot fire either; on that daemon the accepted request can only resolve to +// "virtual machine not found" in its background task. +func (c *Client) requireVNCPolicyEnforced(ctx context.Context, inventory []lumeVM) error { + name, err := vncProbeName(inventory) + if err != nil { + return err + } + body := map[string]any{"noDisplay": false, "vnc": vncPolicyDisabled} + status, payload, err := c.apiRaw(ctx, http.MethodPost, "/lume/vms/"+name+"/run", body) + if err != nil { + return err + } + message := apiErrorMessage(payload) + if status == http.StatusBadRequest && strings.Contains(strings.ToLower(message), vncConflictMarker) { + return nil + } + return vncUnsupported(fmt.Sprintf( + "lume serve answered the %q run-policy probe with HTTP %d %s", + vncPolicyDisabled, status, strings.TrimSpace(message))) +} + +func vncProbeName(inventory []lumeVM) (string, error) { + for range 16 { + id, err := randomID() + if err != nil { + return "", err + } + name := vncProbeVM + id + if _, exists := findVM(inventory, name); !exists { + return name, nil + } + } + return "", errors.New("unable to allocate a vnc run-policy probe name") +} + +func vncUnsupported(detail string) error { + return fmt.Errorf( + "lume on this host cannot start guests with VNC disabled: %s; install the build pinned in %s as %s and run lume serve from that executable", + detail, + vncPinFile, + lumeBin, + ) +} + +// requireNoVNCListener stops a guest that came up with a VNC endpoint anyway. +// Lume's listener binds every interface, so a running guest reporting one is +// an exposure, not a cosmetic mismatch. +func (c *Client) requireNoVNCListener(ctx context.Context, name string, vm lumeVM) error { + if vm.vnc() == "" { + return nil + } + failure := agentErrorf( + "vm %q started a VNC listener despite the %q run policy; it was stopped again", + name, vncPolicyDisabled) + if err := c.stopNamed(ctx, name); err != nil { + return errors.Join(failure, fmt.Errorf("stop %q: %w", name, err)) + } + return failure +} diff --git a/internal/lume/vnc_test.go b/internal/lume/vnc_test.go new file mode 100644 index 0000000..8533735 --- /dev/null +++ b/internal/lume/vnc_test.go @@ -0,0 +1,185 @@ +package lume + +import ( + "context" + "errors" + "io" + "strings" + "testing" + "time" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" + + "github.com/GilmanLab/agentcompute/internal/compute" +) + +func TestNewRefusesHostsThatCannotDisableVNC(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + opts tunnelOpts + }{ + { + // Lume 0.5.3 ships --vnc-port and --vnc-password, so a flag scan + // that stops at the "--vnc" prefix would pass this installation. + name: "released build without the run policy option", + opts: tunnelOpts{RunHelp: vncLegacyRunHelp}, + }, + { + // The binary on disk can be the source build while launchd still + // serves the old daemon, which drops the unknown key and accepts. + name: "daemon that ignores the run policy", + opts: tunnelOpts{LegacyDaemon: true}, + }, + { + name: "account-local executable missing", + opts: tunnelOpts{Host: func(script string, _ io.Reader, _, stderr io.Writer) (bool, error) { + if !strings.Contains(script, "run --help") { + return false, nil + } + _, _ = io.WriteString(stderr, "not installed or not executable") + return true, errors.New("exit status 1") + }}, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + _, err := dialTunnel(t, tt.opts) + require.Error(t, err) + }) + } +} + +func TestStartupProbesTheDaemonWithoutTouchingMappedVMs(t *testing.T) { + t.Parallel() + const seed = "ac-seed-macos-tahoe-desktop" + fixture := &lifecycleHost{vms: map[string]lumeVM{seed: seedVM()}} + state := startTunnel(t, tunnelOpts{API: fixture, Host: fixture.host}) + + probes := state.daemon.probeCalls() + require.Len(t, probes, 1) + // A disabled policy asked for alongside a display is rejected while the + // request is decoded, so the pinned daemon answers without reaching any VM. + assert.False(t, probes[0].NoDisplay) + assert.Equal(t, vncPolicyDisabled, probes[0].Policy) + assert.NotContains(t, probes[0].VM, ":", "a colon would send 0.5.3 down its image auto-pull path") + assert.NotContains(t, fixture.names(), probes[0].VM) + assert.Empty(t, fixture.apiCalls(), "the probe must not reach a real VM route") + assert.Empty(t, fixture.runCalls()) +} + +func TestEveryStartSendsTheDisabledRunPolicy(t *testing.T) { + t.Parallel() + const seed = "ac-seed-macos-tahoe-desktop" + const vmName = "ac-demo-web" + ref := compute.Ref{Sandbox: "demo", Name: "web"} + + tests := []struct { + name string + // cancelOnRun stops the readiness wait that follows a start, which + // needs a live guest this fixture cannot provide. + cancelOnRun bool + start func(ctx context.Context, client *Client) error + }{ + { + name: "create", + start: func(ctx context.Context, client *Client) error { + _, err := client.BeginCreateInstance(ctx, compute.CreateInstance{ + Ref: ref, + Start: true, + Image: compute.CatalogImage{Name: "macos/tahoe/desktop", Seed: seed}, + CPUs: 4, MemoryMB: 8192, DiskGB: 100, + }) + return err + }, + }, + { + name: "start", + cancelOnRun: true, + start: func(ctx context.Context, client *Client) error { + _, err := client.StartInstance(ctx, ref, false) + return err + }, + }, + { + name: "restart", + cancelOnRun: true, + start: func(ctx context.Context, client *Client) error { + _, err := client.RestartInstance(ctx, ref, false) + return err + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + t.Parallel() + fixture := &lifecycleHost{vms: map[string]lumeVM{seed: seedVM()}} + if tt.name != "create" { + fixture.vms[vmName] = lumeVM{ + Name: vmName, OS: "macOS", Status: vmStatusStopped, + CPUCount: 4, MemorySize: 8 << 30, DiskSize: lumeDisk{Total: 100 << 30}, + } + } + state := startTunnel(t, tunnelOpts{API: fixture, Host: fixture.host}) + client := state.client + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + if tt.cancelOnRun { + fixture.onRun = cancel + } + require.NoError(t, client.CreateSandbox(ctx, compute.Sandbox{ + Name: "demo", Platform: platformMac, ExpiresAt: time.Now().Add(time.Hour), + })) + if tt.name != "create" { + require.NoError(t, client.updateSandbox(ctx, "demo", func(rec *sandboxRecord) error { + rec.Instances["web"] = &instanceRecord{ + VM: vmName, Seed: seed, Prepared: true, CPUs: 4, MemoryMB: 8192, DiskGB: 100, + } + return nil + })) + } + + err := tt.start(ctx, client) + if tt.cancelOnRun { + require.ErrorIs(t, err, context.Canceled) + } else { + require.NoError(t, err) + } + + runs := fixture.runCalls() + require.Len(t, runs, 1) + assert.Equal(t, vmName, runs[0].VM) + assert.Equal(t, vncPolicyDisabled, runs[0].Policy) + // A display would make Lume reject the disabled policy outright. + assert.True(t, runs[0].NoDisplay) + }) + } +} + +func TestStartStopsAGuestThatKeptItsVNCListener(t *testing.T) { + t.Parallel() + const seed = "ac-seed-macos-tahoe-desktop" + fixture := &lifecycleHost{ + vms: map[string]lumeVM{seed: seedVM()}, + vncOnRun: "vnc://:secret@127.0.0.1:52397", + } + client := newTunneledClientWithHost(t, fixture, fixture.host) + require.NoError(t, client.CreateSandbox(t.Context(), compute.Sandbox{ + Name: "demo", Platform: platformMac, ExpiresAt: time.Now().Add(time.Hour), + })) + _, err := client.BeginCreateInstance(t.Context(), compute.CreateInstance{ + Ref: compute.Ref{Sandbox: "demo", Name: "web"}, + Start: true, + Image: compute.CatalogImage{Name: "macos/tahoe/desktop", Seed: seed}, + CPUs: 4, MemoryMB: 8192, DiskGB: 100, + }) + require.Error(t, err) + + fixture.mu.Lock() + defer fixture.mu.Unlock() + assert.Equal(t, vmStatusStopped, fixture.vms["ac-demo-web"].Status, + "an exposed guest must not be left running") +} diff --git a/pins/lume.yaml b/pins/lume.yaml new file mode 100644 index 0000000..1abb957 --- /dev/null +++ b/pins/lume.yaml @@ -0,0 +1,126 @@ +schema_version: 1 +recipe_revision: 1 +architecture: arm64 + +# The Lume the Mac host's agentcompute service account runs. +# +# Released Lume 0.5.3 always starts a VNC listener on a wildcard address and +# offers no switch to bind or disable it, which is the whole reason this file +# exists. Upstream merged the disable path as cua#3209, and no Lume release +# contains it yet: lume-v0.5.3 (2026-08-11) is still the newest tag, and the +# commit landed 2026-08-16. So this host builds that exact commit from source, +# account-locally, instead of waiting for a release or opening host ports. +# +# Scope of the source build: it installs ONLY under /Users/agentcompute. The +# global /usr/local/bin/lume (released, Developer ID signed, notarized 0.5.3) +# is untouched, and so are PF and Internet Sharing. + +lume: + source_build: true + # trycua/cua, the commit that merged "feat(lume): support running without a + # VNC listener (#3209)", committed 2026-08-16T21:02:51+02:00. + source_commit: ab957bdb7566f7e137b00654cc01167d9e42af38 + source_url: https://github.com/trycua/cua.git + source_subpath: libs/lume + upstream_change: https://github.com/trycua/cua/pull/3209 + # libs/lume/VERSION at this commit. It is the released version string with + # no suffix, so `lume --version` alone cannot tell this build apart from the + # global 0.5.3. Identify it by path and by installed_sha256, not by version. + version: 0.5.3 + license: MIT + + # Everything below is measured from the build that is installed on the host. + build_host: studio-1 (macOS 26.6.2 / 25G83, Apple M2 Max) + build_user: agentcompute + toolchain: Apple Swift 6.3.3 (swiftlang-6.3.3.1.3 clang-2100.1.1.101), Command Line Tools + build_command: swift build -c release --product lume --only-use-versions-from-resolved-file + # Dependency versions come from libs/lume/Package.resolved at the pinned + # commit and nothing else. Package.swift tracks two moving branches + # (mhdhejazi/Dynamic @ master, apple/swift-format @ release/6.0), so a free + # resolve here would silently pick up whatever those branches point at today. + dependency_resolution: Package.resolved only + source_dir: /Users/agentcompute/src/cua + app_install_dir: /Users/agentcompute/.local/share/lume + install_bin: /Users/agentcompute/bin/lume + + # Ad-hoc signature. `com.apple.vm.networking` is deliberately NOT requested: + # it is Apple-granted, needs an embedded provisioning profile, and macOS + # kills an ad-hoc binary that carries it (upstream says so in + # scripts/install-local.sh). The seed is NAT-only, which needs only + # com.apple.security.virtualization. + entitlements_file: resources/lume.local.entitlements + entitlements: + - com.apple.security.virtualization + signing: adhoc + bundle_id: com.trycua.lume + # SHA-256 of the installed Mach-O executable, + # /Users/agentcompute/.local/share/lume/lume.app/Contents/MacOS/lume, after + # signing. This is the artifact identity the host runs; the wrapper below is + # just a two-line exec shim. + installed_sha256: 5643bfdc54079cf0d94403a2d49c25a72c9759ee84029f871020a941851609df + # SHA-256 of /Users/agentcompute/bin/lume, the launcher the backend and the + # service invoke. + wrapper_sha256: e26d62a32f80598fd0204de79a072c3ebbbd5de7f72ada3a44c73d1040e2f9fc + provenance: measured + published_by: null + # Measured on this host, not assumed: re-assembling, re-signing, and + # reinstalling from the same SwiftPM build tree reproduces + # installed_sha256 byte for byte, but a rebuild after `rm -rf .build` + # produces a different Mach-O from the same commit and the same + # Package.resolved. So installed_sha256 identifies the artifact this host + # was given, not a value a clean rebuild will land on. + # + # images/macos/build-lume.sh refuses to install anything whose hash is not + # the one above, and refuses before it touches the install paths. After a + # deliberate clean rebuild the operator re-reads the commit, writes the new + # hash here, and says why in the commit message. Do not relax that check + # into a warning. + bitwise_reproducible: false + + # What this build adds over the released pin, and the exact shape of it. + vnc_policy: + cli: lume run --no-display --vnc disabled + http: 'POST /lume/vms//run with {"noDisplay": true, "vnc": "disabled"}' + # `disabled` conflicts with a display, so the HTTP body must also set + # noDisplay: true. The server validates before it starts anything + # (Handlers.swift handleRunVM -> validatedVNCPolicy(noDisplayDefault: + # false)), and answers 400 otherwise. + values: + - enabled + - disabled + default: enabled + rejects: + - --display vnc + - --vnc-port + - --vnc-password + + build_script: images/macos/build-lume.sh + +# The released pin this host would fall back to, and the only Lume the global +# install has ever been. Identical to images/macos/pins.lock.yaml `lume`; +# repeated here so a rollback needs one file, not two. +fallback: + version: 0.5.3 + archive: lume-0.5.3-darwin-arm64.tar.gz + archive_root: lume + url: https://github.com/trycua/cua/releases/download/lume-v0.5.3/lume-0.5.3-darwin-arm64.tar.gz + size: 6009673 + sha256: af5d0556763a7f0116153c220aaabe44974e775091ac57e38da2abb2959c63e8 + provenance: published + published_by: https://api.github.com/repos/trycua/cua/releases (asset digest) + installed_at: /usr/local/bin/lume + # Only a manual rollback reference: backend startup rejects this release. + # Never automatically fall back to a wildcard VNC listener. + vnc_policy: none available + +# Delete this source build and return to a released pin as soon as one carries +# cua#3209. +revisit: + condition: the first trycua/cua `lume-v*` release whose tree contains + ab957bdb7566f7e137b00654cc01167d9e42af38 (that is, the first release after + lume-v0.5.3 that includes cua#3209). + action: replace this file's source-build fields with the release archive and + published SHA-256, install it account-locally, and retire + images/macos/build-lume.sh. Keep the account-local path and disabled policy. + checked: 2026-09-15 + newest_release_at_check: lume-v0.5.3 (2026-08-11), does not contain the commit