diff --git a/.gitignore b/.gitignore index 82bd83e1e..982c5a8e9 100644 --- a/.gitignore +++ b/.gitignore @@ -43,3 +43,8 @@ result-* # NOTE: generated clients (go/gen, packages/compass-client/src/gen) # are intentionally committed and CI drift-gated — do not ignore them. + +# runner-image build outputs: the staged nix closure (which carries the +# entrypoint symlink) and the OCI layout, realised by tools/runner-image/build.ts. +/runner-image/store/ +/runner-image/out/ diff --git a/.moon/workspace.yml b/.moon/workspace.yml index a718eec44..773d9d18a 100644 --- a/.moon/workspace.yml +++ b/.moon/workspace.yml @@ -39,12 +39,23 @@ projects: # cache lane or V2a boot bring-up. Same affected-detection posture as # compass-agent-image (see guest-image/moon.yml). compass-guest-image: 'guest-image' + # The Runner container image: the compass-runner binary, the KVM userland and + # the guest assets on a minimal hardened base. Its `build` is runInCI:false (it + # needs a buildkitd and a multi-GB realise), so registration here is for graph + # membership and affected-detection, not a gate leg; the gate coverage is the + # runner-image tool's pure-core suite below. See runner-image/moon.yml. + compass-runner-image: 'runner-image' # The local microVM boot-test lane (RIG-2591): realises the guest image + VMM # stack from nix and execs the KVM-gated `go test -tags microvm` suite the # untagged compass-go:test lane never builds. Registered so its typecheck + # unit tests ride the moon-driven CI sweep; the boot lane itself lives on # compass-go:test-microvm (runInCI:false — it needs KVM + a nix build). microvm-boot-test: 'tools/microvm-boot-test' + # The runner-image build lane: realises the nix closure, stages it, and drives + # the Dockerfile build. Registered so its typecheck + pure-core unit tests ride + # the CI sweep — that suite is what catches a build-arg or closure-root drift, + # since no compass CI step can build the image itself today. + runner-image: 'tools/runner-image' # The Compass native-app release bundle: a heavy nix build (realises the # WebKitGTK cc/pkg-config closure) that stages the versioned tarball. Same # affected-detection posture as compass-agent-image — registered here so the diff --git a/.release-please-manifest.json b/.release-please-manifest.json index 9d1a25b76..efc4499a8 100644 --- a/.release-please-manifest.json +++ b/.release-please-manifest.json @@ -1,3 +1,3 @@ { - ".": "0.2.0" + ".": "0.2.1" } diff --git a/CHANGELOG.md b/CHANGELOG.md index 906d84434..1f134f9b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,22 @@ # Changelog +## [0.2.1](https://github.com/RigelBuild/compass/compare/v0.2.0...v0.2.1) (2026-09-12) + + +### Features + +* **compass-agent:** emit decoded traceparent on the SessionInjection observation (RIG-2894) ([#1132](https://github.com/RigelBuild/compass/issues/1132)) ([2392c9a](https://github.com/RigelBuild/compass/commit/2392c9adca70b3781e69ba2a408fe7fcedd2f8c5)) +* **compass-agent:** re-attach the turn's trigger traceparent on outbound posts (RIG-2894) ([#1139](https://github.com/RigelBuild/compass/issues/1139)) ([1dfcbfc](https://github.com/RigelBuild/compass/commit/1dfcbfc67417a158cd99ee5ecdd30317b66c0585)) +* **envelope:** AES-256-GCM seam for user-secret values (RIG-3655 T1) ([#1111](https://github.com/RigelBuild/compass/issues/1111)) ([d58e256](https://github.com/RigelBuild/compass/commit/d58e256efb135e34b8c88063e014d6844498ddfe)) +* **secrets:** add the DB-backed StoreResolver (RIG-3655 T3) ([#1138](https://github.com/RigelBuild/compass/issues/1138)) ([902feca](https://github.com/RigelBuild/compass/commit/902feca21b9a722673f34f4ed6eb197bba6d4741)) +* **server:** resolve the at-rest master key at boot (RIG-3655 T4) ([#1146](https://github.com/RigelBuild/compass/issues/1146)) ([5c00a05](https://github.com/RigelBuild/compass/commit/5c00a0515418b4abcbfefd56beb664bf4b061eeb)) +* **store:** scope and encrypt user secrets at rest (RIG-3655 T2) ([#1118](https://github.com/RigelBuild/compass/issues/1118)) ([c34e417](https://github.com/RigelBuild/compass/commit/c34e4175e4d23d843ffd054e17dd7b92e2787643)) + + +### Bug Fixes + +* **release:** put the pinned pkg-config on PATH for the app bundle (RIG-3731) ([#1171](https://github.com/RigelBuild/compass/issues/1171)) ([3692236](https://github.com/RigelBuild/compass/commit/36922361628173c366f826ea6496a84469f20e78)) + ## [0.2.0](https://github.com/RigelBuild/compass/compare/v0.1.0...v0.2.0) (2026-09-12) diff --git a/bun.lock b/bun.lock index 7e4f2f793..786a61944 100644 --- a/bun.lock +++ b/bun.lock @@ -236,6 +236,16 @@ "typescript": "catalog:", }, }, + "tools/runner-image": { + "name": "@compass/runner-image", + "bin": { + "runner-image-build": "./build.ts", + }, + "devDependencies": { + "@types/bun": "catalog:", + "typescript": "catalog:", + }, + }, "tools/sea-ref-gate": { "name": "@compass/sea-ref-gate", "bin": { @@ -491,6 +501,8 @@ "@compass/renovate-preflight": ["@compass/renovate-preflight@workspace:tools/renovate-preflight"], + "@compass/runner-image": ["@compass/runner-image@workspace:tools/runner-image"], + "@compass/sea-ref-gate": ["@compass/sea-ref-gate@workspace:tools/sea-ref-gate"], "@compass/sql-migration-gate": ["@compass/sql-migration-gate@workspace:tools/sql-migration-gate"], diff --git a/docs/designs/DECISIONS.md b/docs/designs/DECISIONS.md index 5a42e5a2c..da2a1b250 100644 --- a/docs/designs/DECISIONS.md +++ b/docs/designs/DECISIONS.md @@ -133,6 +133,7 @@ check enforces the mechanical half. Full rationale: | DL-361 | Secret resolution is most-specific-wins — `agent > user > tenant`, ONE value per name in the injected environment — collapsed in SQL (`DISTINCT ON` ordered by `scope_kind DESC`, the numeric encoding being the precedence) so shadowed rows never leave Postgres or get decrypted; `FetchSecrets` resolves per agent account using the identity the runnerhub authz maps (`sessionAccounts`/`containerAccounts`) already hold and previously discarded, the user tier reached through the single `agent_accounts.owner_user_id` FK hop. Scope is an ADDITIONAL filter inside a tenant — RLS tenant isolation stays the outer boundary, never replaced. The existing `SetSecret`/`DeleteSecret` verbs stay pinned to the tenant coordinate, preserving inject-all behavior until a scope wire surface is ruled | Active (Matt, 2026-09-11) | [user-secret store §A9](server/compass-user-secret-store.md#a9--scope-model-tenant--user--agent-most-specific-wins) | | DL-362 | The canonical user-secret AAD is the five-field tuple `"compass/user-secret/v1\x00" + tenantID + "\x00" + decimal(scopeKind) + "\x00" + scopeID + "\x00" + name + "\x00" + decimal(keyVersion)` (Go: `UserSecretAAD(tenantID string, scopeKind int16, scopeID, name string, keyVersion int16) []byte`; SMALLINTs rendered `strconv.FormatInt(int64(v), 10)`), every field bound unconditionally (a tenant row binds scopeID as the empty string) with `\x00` separators keeping the encoding injective. Fixed BEFORE any migration ships because the AAD is baked into every ciphertext — a scope field added later would force a re-encrypt of every row. Refines DL-351's four-field AAD clause; DL-351's other rulings stand | Active (Matt, 2026-09-11) | [user-secret store §A9](server/compass-user-secret-store.md#a9--scope-model-tenant--user--agent-most-specific-wins) | | DL-363 | Writing a tenant-scoped user-secret row requires an admin (`store.UserRoleAdmin`, `go/internal/store/types.go`), reusing the existing role elevation rather than introducing a permission concept: tenant (0) admin-only, user (1) and agent (2) writable by the owning user or an admin. The check lands in the store door inside the same writing transaction as DL-360's FK-substitute referential checks, so one place enforces both. READS are deliberately asymmetric — a plain user's agent resolves tenant rows, which is the point of a shared tenant value under DL-361; reading a shared secret is the feature, writing one is the privileged act. The wire surface for a scoped write stays undecided (a scope selector on `SetSecretRequest` is a public-proto fork) | Active (Matt, 2026-09-12) | [user-secret store §D8](server/compass-user-secret-store.md#resolved-decisions) | +| DL-366 | The `compass-agent` image publishes as a multi-arch OCI image index carrying exactly `linux/amd64` + `linux/arm64` under the existing `:git-`/`:latest`/`:vX.Y.Z` tags, each arch built NATIVELY on its own runner (`ubuntu-latest` + the free public-repo `ubuntu-24.04-arm`, never QEMU) and composed by member digest in a serialized manifest job. Consumer-facing tags stay bare, so every puller relies on engine platform negotiation and no consumer changes; the per-arch `:git--` tags are internal immutable building blocks. Every single-digest guard is carried forward at manifest-list level rather than dropped — immutability, two-tag coherence and the semver re-tag compare list digests, and the `linux/amd64` platform tripwire becomes an exact platform-SET assertion plus per-member arch/os checks, a stricter contract than the one it replaces. Supersedes the `linux/amd64` single-arch platform contract of the agent-image publish record; that record's tag names, immutability posture, auth pin and build-once mechanism all stay live | Active (Matt, 2026-09-12) | [arm64 agent image §Decision A](infra/ci/compass-agent-arm64-image/design.md#decision-a--consumer-facing-tags-are-a-manifest-list-per-arch-tags-are-internal) | > Note (2026-07-31, RIG-1570 R5): DL-065's retired internal `ResumeContext > resume = 12` envelope field is NOT silently reintroduced — the collapsed @@ -287,6 +288,7 @@ check enforces the mechanical half. Full rationale: | DL-208 | Both Solid-2 blocker deps are FORKED into the RigelBuild org and consumed as externalized pinned artifacts, never vendored into the compass tree (matching the org-wide fork-externalization ruling): `@tanstack/solid-virtual` is forked with Solid-2 support + an upstream PR to TanStack/virtual (a 136-line thin wrapper over the framework-agnostic `@tanstack/virtual-core`, likely to land), consumed via a `package.json` git ref (`github:RigelBuild/virtual`) until upstream merges then dropped; `solid-markdown` (effectively abandoned upstream) is forked-and-owned, published as `@rigelbuild/solid-markdown` with the `solid-js` 2.x peer bump + Solid-2 codemods (imports + effect lifecycle, no algorithmic rewrite, no in-repo AST renderer). Overrides this record's earlier adapter-over-core virtualizer recommendation | Active (Matt, 2026-08-18) / solid-markdown clause superseded by DL-218 (Matt, 2026-08-19) | [solid v2 migration §Resolved decisions](ui/compass-ui-solid-v2/design.md#resolved-decisions) | | DL-209 | The Wails packaged-shell seam under Solid 2 is low-risk-but-verified (no automation): the seam is Solid-free plain TS (`daemon-transport.ts` string-name `Call`/`Events`; `MarkdownText.tsx:1` `Browser.openExternal`), so effect-flush timing is the lesser risk; the packaged shell is the first place the babel-preset-solid 2 / vite-plugin-solid 3 / `@solidjs/web` production-build compilation and v2's render-root-owned delegated events run outside dev/happy-dom, so S7's manual packaged-shell smoke (RPC round-trip + openExternal) stays the mandatory gate — no automation justified for a one-shot migration | Active (Matt, 2026-08-18) | [solid v2 migration §Resolved decisions](ui/compass-ui-solid-v2/design.md#resolved-decisions) | | DL-218 | The owned `RigelBuild/solid-markdown` fork adopts upstream `andi23rosca/solid-markdown` #44 (the react-markdown-10 API rewrite — a third-party draft, upstream effectively inactive on it) + #45 (the bundle-deps jsdom-compat fix), re-ported to Solid 2 via a fork-local `src/jsx-runtime.ts` `jsx`/`jsxs`/`Fragment` triple over `createComponent`/`merge` + `@solidjs/web` `Dynamic` (solid-jsx dropped as Solid-1-bound), published as a clean `@rigelbuild/solid-markdown@3.0.0` (no `-rigel.N` suffix — the fork is ours and sets its own versions); this retires the fork's in-repo AST renderer in favor of the maintained `hast-util-to-jsx-runtime`. The apps/ui consumer injects @mention chips via a consumer-side rehype plugin over the existing `mentionRuns` splitter (the removed `text` component override), derives inline-vs-block code structurally via a `pre`/`code` override split (the removed `inline` prop), keeps `renderingStrategy="reconcile"` (un-deprecated in the owned fork), and adds a synchronous consumer-side `(lang, code)` highlight cache that `CodeBlock` reads at initial render (seeding highlighted HTML and bypassing the debounce + async `createResource` on a hit) to suppress the code-highlight flash caused by #44's per-tick component teardown (accepting marginal within-stream selection loss; the fine-grained-walker re-port is the escalation if it bites). Supersedes DL-208's `solid-markdown` clause ONLY (no-algorithmic-rewrite/no-in-repo-AST-renderer) — DL-208's `@tanstack/solid-virtual` clause and its org-wide fork-externalization stance stay Active | Active (Matt, 2026-08-19) | [markdown react10 §Approach](ui/compass-ui-markdown-react10/design.md#approach) | +| DL-367 | Compass UI chrome glyphs are dot-matrix `crispEdges` SVG on an 11×11 grid, adopting the technique DL-150/DL-199 froze at 9×9 for `StateDot`/`BadgeGlyph` (which stay 9×9 — two grid sizes coexist, the components are separate). `ActivityBarItem.icon: string` is split AT THE ITEM into `GlyphTabItem { kind: "glyph"; name: GlyphName }` and `AvatarTabItem { kind: "avatar"; letter: string }` — the field conflated a fixed symbol with an agent handle's initial, and a field-level union was rejected because it preserves exactly that confusion. The avatar initial is NFKD-normalized and combining-marks-stripped before an ASCII clamp (`É`→`E`), falling back to `?` only for scripts no Latin letter represents — a bare clamp would collapse distinct non-ASCII handles to identical `?` tabs, defeating the per-agent distinguishability the initial exists for. 11×11 does NOT resolve the pre-existing 15px slot violation (that rule governs Departure Mono font sizes via UPM, not SVG boxes); the real hazard is half-pixel placement in the 34px tab, so the glyph box is pinned to a whole-pixel offset. Conversion is wholesale — every chrome symbol becomes a ``, including ones Space Mono covers — accepting baseline churn for one vocabulary. This is the path that retires the ~14 MB Unifont pin, in its own stacked PR after the glyph lane recaptures baselines | Active (Matt, 2026-09-12) | [glyph primitives](ui/compass-glyph-primitives/design.md) | ## Threading & rendering diff --git a/docs/designs/infra/ci/compass-agent-arm64-image/design.md b/docs/designs/infra/ci/compass-agent-arm64-image/design.md new file mode 100644 index 000000000..8d36deff5 --- /dev/null +++ b/docs/designs/infra/ci/compass-agent-arm64-image/design.md @@ -0,0 +1,543 @@ +# Publish the compass-agent image for linux/arm64 + +Status: Draft +Tracking: RIG-3625 + +> **Design record.** This designs the multi-arch (linux/amd64 + linux/arm64) +> publish lane for the `compass-agent` runtime image. Every path cited below is +> a path in `RigelBuild/compass` at `origin/main` as of 2026-09-12; line numbers +> drift as the code evolves, so resolve them against that ref. It extends, and +> partially supersedes, the platform contract in +> [`compass-agent-image-publish`](../compass-agent-image-publish/design.md) +> ("Decision: platform contract — `linux/amd64` single-arch"), whose other +> decisions (tag names, immutability posture, auth pin, build-once mechanism) +> all stay live and are carried forward here. + +## Problem / Intent + +The Apple-container runtime tier on macOS runs agent containers in a real +Linux arm64 guest, so the `compass-agent` image must exist for linux/arm64. +Today the publish lane builds and ships linux/amd64 only, and CI actively +asserts that: `.github/workflows/release.yml:333-335` fails the publish run +when the pushed image is not `linux/amd64`. This record designs the arm64 +build, the multi-arch tag layout, and the multi-arch form of every existing +integrity guard. Matt has authorized building the arm image (RIG-3625). + +## Context: how the image is built and guarded today + +### The build is native, single-arch, and independent of the root flake + +`agent-image/publish.sh:65-68` builds the image spec through the shared devenv +fork, resolved from `agent-image/devenv.lock` — never from the root +`flake.nix`: + +```bash +DEVENV_SRC="$(bun "$SCRIPT_DIR/../tools/toolchain/devenv-cli/index.ts" --lock "$SCRIPT_DIR/devenv.lock" --mode flakeref)" +log "Building image spec: nix run $DEVENV_SRC -- container build agent" +BUILD_OUT="$(nix run "$DEVENV_SRC" -- container build agent)" +SPEC="$(printf '%s\n' "$BUILD_OUT" | tail -n 1)" +``` + +`nix run` evaluates the devenv fork's flake for the *builder's* system and +nix2container emits a spec for that system — so the build is native: an arm64 +image needs an arm64 builder (or emulation). The root `flake.nix:33` +(`systems = [ "x86_64-linux" ];`) is **not in this build's dependency graph**: +the inputs are `agent-image/devenv.yaml`'s own `nixpkgs`, `nix2container`, +`mk-shell-bin`, and `devenv` fork inputs, pinned in `agent-image/devenv.lock`. +The moon gate build (`agent-image/moon.yml`, `build` task) runs the identical +two-step resolve-then-run: + +```text +script: 'src=$(bun tools/toolchain/devenv-cli/index.ts --lock agent-image/devenv.lock --mode flakeref) && cd agent-image && nix run "$src" -- container build agent' +``` + +So the blast radius of arm64 support is `agent-image/` plus the publish jobs in +`release.yml` — the root flake's `systems` list does not change. The +`flake.nix:138-141` TODO ("TODO(aarch64-darwin follow-up): the darwin app links +system WebKit via frameworks … add a darwin branch when the systems list +grows") is about aarch64-**darwin** and the gtk/WebKit app; it is unrelated to +the agent image's aarch64-**linux** need and stays untouched. + +### The single-digest assumptions a manifest list breaks + +Three guards assume one image config per tag. All three are deliberate +security/correctness properties, and each needs a multi-arch equivalent (never +deletion — `rule://no-inert-gating`). + +1. **`guard_immutable` + `LOCAL_DIGEST`** (`agent-image/publish.sh:81`, + `:92-127`): the local identity is one config digest, + + ```bash + LOCAL_DIGEST="$("${SKOPEO[@]}" inspect --raw "nix:$SPEC" | jq -r .config.digest)" + ``` + + and the guard compares it to the remote tag's + (`remote_digest="$(printf '%s' "$remote_raw" | jq -r .config.digest)"`, + `publish.sh:102`), aborting on any ambiguous registry error. The post-copy + assert (`publish.sh:153`) re-inspects the pushed tag against the same + digest. On a manifest list, `skopeo inspect --raw` returns an OCI image + index — it has **no** `.config.digest`, so `jq -r .config.digest` yields + `null` and every compare is meaningless. + +2. **The amd64 tripwire** (`.github/workflows/release.yml:330-336`): + + ```bash + # Cheapest platform-contract-regression tripwire. + arch="$(jq -r .Architecture "$inspect_json")" + os="$(jq -r .Os "$inspect_json")" + if [ "$arch" != "amd64" ] || [ "$os" != "linux" ]; then + echo "platform contract violated: got $os/$arch, want linux/amd64" >&2 + exit 1 + fi + ``` + + `skopeo inspect` on a manifest list resolves one member (or errors, + depending on flags); `.Architecture` is not a property of the list. The + tripwire must become an assertion about the new contract — the index + contains exactly the expected platform set — not disappear. + +3. **The `:latest` / `:git-` coherence check** + (`.github/workflows/release.yml:338-345`): + + ```bash + git_digest="$(skopeo inspect --raw --authfile "$REGISTRY_AUTH_FILE" "$ref:git-$sha12" | jq -r .config.digest)" + latest_digest="$(skopeo inspect --raw --authfile "$REGISTRY_AUTH_FILE" "$ref:latest" | jq -r .config.digest)" + if [ "$git_digest" != "$latest_digest" ]; then + ``` + + Same breakage: no `.config.digest` on an index. The property itself + (the two tags are the same artifact) survives — compared as manifest-list + digests instead. + + The semver re-tag job carries the same shape: `release.yml:874-885` copies + `:git-` to `:vX.Y.Z` with `skopeo copy` and verifies + `.config.digest` coherence. Two multi-arch breaks there: a bare + `skopeo copy` of a list copies a **single resolved image**, not the list + (it needs `--multi-arch all`), and the digest compare needs the same + list-digest form. + +### Where publish runs in CI + +The `publish-image` job (`release.yml:94-129`) runs on `runs-on: ubuntu-latest` +(`release.yml:96`) with `permissions: contents: read / packages: write` +(`release.yml:98-100`), serialized under +`concurrency: group: publish-agent-image, cancel-in-progress: false, queue: max` +(`release.yml:110-113`), gated to `github.ref == 'refs/heads/main'` +(`release.yml:117`), `working-directory: agent-image` (`release.yml:127-129`). +Its bootstrap, per step: + +- an in-job changed-path gate over `IMAGE_CLOSURE_PATHS` (`release.yml:49`, + consumed at `:204`) decides `should_publish`; +- `cachix/install-nix-action` with reviewed substituters (`release.yml:209-222`); +- pinned bun via `nix eval -f tools/toolchain/gate-tools.nix langs.bun` + (`release.yml:224-244`) — publish needs the devenv-CLI resolver under bun; +- the fork's patched skopeo via + `nix build -f tools/toolchain/skopeo-nix2container-env.nix skopeo` + (`release.yml:246-278`), prepended to `PATH` — it understands the `nix:` + transport stock skopeo lacks; +- `REGISTRY_AUTH_FILE=$RUNNER_TEMP/ghcr-auth.json` pinned to `GITHUB_ENV` + (`release.yml:287`) so login and copy resolve the same creds file; +- `skopeo login ghcr.io` with `GITHUB_TOKEN` via `--password-stdin` + (`release.yml:302-305`); +- `run: ./publish.sh` with no args (`release.yml:312`) — the default two-tag + set, `publish.sh:54-59`: `TAGS=("git-${SHA}" "latest")`, immutable pin + first; +- the verify step (`release.yml:314-346`) quoted above. + +## Approach + +Fan the build out to one native job per arch, keep every per-arch artifact +under an immutable per-arch tag with the existing single-digest guards intact, +then compose and push an OCI image index under the existing consumer-facing +tags (`:git-`, `:latest`, and at release time `:vX.Y.Z`), with each +guard translated to its list-level equivalent. + +### Decision A — consumer-facing tags are a manifest list; per-arch tags are internal + +**Options.** + +1. **Manifest list under the existing tags** (recommended). `:git-` and + `:latest` become OCI image indexes listing exactly + `{linux/amd64, linux/arm64}`. Per-arch tags `:git--amd64` / + `:git--arm64` exist as immutable internal building blocks the index + references by digest. +2. **Per-arch tags only** (`:git--arm64`, `:latest-arm64`). Keeps every + existing guard byte-for-byte, but pushes arch selection onto every consumer + forever. +3. **Separate arm64 repository** (`compass-agent-arm64`). Same consumer burden + as 2 plus a second package's visibility/auth surface. No advantage. + +**Consumers found** (search: `compass-agent` refs across the repo): + +- `go/cmd/compass-app/embedded.go:43` — + `const defaultAgentImage = "ghcr.io/rigelbuild/compass-agent:latest"`, the + embedded stack's default when no `--image`/`$COMPASS_AGENT_IMAGE` is given. +- `go/internal/stack/stack.go:316` — + `s.deps.Images.EnsureImage(ctx, s.cfg.AgentImage)`; the ensurer pulls by + tag: `go/internal/stack/adapters/image.go` `imageCLI` is + `ImageExists(ctx, image)` + `Pull(ctx, image)`, backed by + `go/internal/runtime/podman.go:651-652` — `p.run(ctx, "podman pull", + []string{"pull", image})`. A plain tag pull, no digest, no platform flag. +- `go/cmd/compass-stack/main.go:273` — `AgentImage: f.image` plumbs the + `--image` flag into that config. +- The Apple-container tier (the motivating consumer): + `docs/designs/platform/apple-container-macos-runner/design.md` OQ-8 states + "the compass-agent image is built for the host arch" — i.e. its pull path + also consumes the plain ref and expects the registry to serve the host arch. + +Every consumer pulls a bare tag and relies on the container engine's default +platform negotiation. With a manifest list, **zero consumer changes**: podman +(and Apple `container`) resolve the index to the host-native member. With +per-arch tags, `defaultAgentImage`, the baked `--image` pin, the stack config, +and every doc naming the ref would all need arch-switching logic — permanent +complexity in many places to avoid one-time complexity in the publish lane. + +**Recommendation: option 1.** The publish lane is the single place that knows +the platform set; keep the arch knowledge there. + +### Decision B — native `ubuntu-24.04-arm` runner, not QEMU + +**Options.** + +1. **GitHub-hosted `ubuntu-24.04-arm` runner** (recommended). Free-tier Linux + arm64 runners are GA for public repos (labels `ubuntu-24.04-arm` / + `ubuntu-22.04-arm`); `RigelBuild/compass` is public. The arm64 job is a + near-clone of the amd64 job: same nix install, same bootstrap steps, same + `publish.sh` invocation. The build is native, so nix2container is expected + to emit an aarch64-linux spec with no cross machinery. That is + designed-to-be-true, not yet observed (UNVERIFIED until T2; OQ-1). +2. **binfmt/QEMU emulation on `ubuntu-latest`.** One runner, but the image + closure is the dominant CI cost already (the 90-minute `timeout-minutes` + at `release.yml:120` is sized by it); emulating a full nix build of that + closure multiplies it several-fold and adds a binfmt setup step as a new + trust surface. Rejected. +3. **Self-hosted arm64 (the mattmini).** The mac mini is committed to darwin + spike/contract work (apple-container record, OQ-9 ruling); a Linux arm64 + build lane on it would need a Linux VM and a standing runner registration. + Unnecessary while hosted arm64 runners are free. + +**Recommendation: option 1**, with these grounded facts and stated unknowns: + +- **bun pin: verified present for arm64.** + `tools/toolchain/versions/bun.nix:9-11` already carries an + `"aarch64-linux"` entry (`bun-linux-aarch64.zip` + hash), so the + `gate-tools.nix langs.bun` bootstrap resolves on the arm runner. +- **FOD hash: verified single-platform — a real change.** + `agent-image/entrypoint.nix` pins the bundled entrypoint's `node_modules` + tree as one fixed-output hash + (`outputHash = "sha256-JbgM44AwH7/b3Y/2T44+eBXwyvMi8owXToGVspEeCk4="`), and + its own comments state the hash covers "the INSTALLED TREE … as it lands on + THIS build platform" including "platform-specific optional dependencies". + An aarch64 install produces a different tree, so the single hash must + become a per-system attrset keyed like `versions/bun.nix` (T1). +- **UNVERIFIED: the devenv fork + nix2container fork + patched skopeo + toolchain has never been run on aarch64-linux.** Nobody has executed + `nix run -- container build agent` on an arm64 builder. Upstream + devenv, nix2container, and nixpkgs all support aarch64-linux, and + `tools/toolchain/skopeo-nix2container-env.nix` imports nixpkgs + system-implicitly ("the system is implied by the nixpkgs it is imported + with"), so nothing *pins* x86_64 — but this record does not claim it works. + T2 is a spike that proves or disproves it before any workflow change. +- **UNVERIFIED: binary-cache coverage on aarch64-linux.** The reviewed + substituters (`devenv.cachix.org`, `cachix.cachix.org`, + `release.yml:219-222`) may hold few aarch64 artifacts. The risk is + build-from-source time, not correctness: a cache miss falls back to source + builds within the 90-minute ceiling or fails it visibly. If T2 shows the + wall-clock is unacceptable, populating a cache is a follow-up, not a design + change. +- **The `@oh-my-pi` native-addon copy block is x64-hardcoded and must be + edited, independent of whether an aarch64 prebuilt exists.** + `agent-image/entrypoint.nix:218-220` names the arch three times: + `natives=node_modules/.bun/node_modules/@oh-my-pi/pi-natives-linux-x64`, + then `cp $natives/pi_natives.linux-x64-modern.node` and + `pi_natives.linux-x64-baseline.node`. The package name, both filenames, and + the variant scheme itself all change on arm64: the surrounding comment + (`:214-217`) states the loader picks `modern` when the host has AVX2 else + `baseline`, and AVX2 is an x86 feature with no arm64 analogue, so the + two-variant copy is not portable as written. T1 owns this edit. +- **UNVERIFIED: whether that aarch64 prebuilt exists at all, and under which + variant names.** If absent, the entrypoint bundle fails at build time on + arm64 — a loud, pre-push failure. T2 surfaces both the existence and the + real filenames the copy block must use. + +### Decision C — the guards' multi-arch forms + +Every guard survives; none is deleted (`rule://no-inert-gating`). + +1. **Per-arch immutability: unchanged code, new tag names.** `publish.sh` runs + once per arch job and pushes only that arch's tag + (`git--`). Single-arch manifests still have exactly one + `.config.digest`, so `LOCAL_DIGEST` (`publish.sh:81`), `guard_immutable` + (`publish.sh:92-127`) — including its "ambiguous registry error → abort, + never overwrite" posture — and the post-copy assert (`publish.sh:153`) + work verbatim. The script grows a tag-suffix/skip-latest mode (T3); its + guard logic does not change. + +2. **Index immutability: the same guard shape one level up.** The compose + step's local identity is the index's **manifest-list digest** (the sha256 + of the raw index bytes, `skopeo inspect --raw docker://…:git- | + sha256sum`, or `skopeo inspect --format '{{.Digest}}'`). Before pushing + `:git-`: if the remote tag exists and its list digest equals the + locally composed one → idempotent skip; if it exists and differs → hard + fail; if inspect fails with anything but a definitive manifest-unknown → + abort. Identical decision table to `guard_immutable`, with + `.config.digest` replaced by the list digest. + + **This guard's skip arm depends on byte-deterministic index composition, + which is UNVERIFIED (OQ-5).** An OCI image index is not canonicalized by + the spec, so a recomposed index could differ in member order or carry an + injected annotation and hash differently while describing the same two + images. If that happens, the digest-equality skip never fires and a + re-run after a mid-compose failure hits the "exists and differs → hard + fail" arm against a tag that is immutable by design, wedging that sha's + publish with no clean recovery. T2 closes this by composing, pushing to a + scratch tag, recomposing from the same member digests, and comparing the + two list digests. If composition proves non-deterministic, the guard's + identity becomes the **member digest set** (assert the remote index's + members are exactly the two per-arch digests) rather than the list digest + — same immutability property, no dependence on byte-stable serialization. + +3. **The platform tripwire becomes a platform-set assertion.** Replacement + for `release.yml:330-336`: fetch the raw index for `:git-`, assert + `mediaType` is an image index, and assert the platform set is **exactly** + `{linux/amd64, linux/arm64}` — no members missing, none extra: + + ```bash + platforms="$(jq -r '[.manifests[].platform | "\(.os)/\(.architecture)"] | sort | join(",")' "$index_json")" + [ "$platforms" = "linux/amd64,linux/arm64" ] || fail + ``` + + Then, for each member, resolve its digest-addressed manifest and assert + its config's `architecture`/`os` match the entry's declared platform — + the direct descendant of the old tripwire, now per member. This is a + stronger contract than today's, not a weaker one: it also fails when the + arm64 half silently vanishes. + +4. **Two-tag coherence compares list digests.** Replacement for + `release.yml:338-345`: `:latest`'s manifest-list digest must equal + `:git-`'s. Same property ("the moving tag is the pinned artifact"), + same hard-fail, one level up. + +5. **The semver re-tag copies the whole list.** `release.yml:874-875`'s + `skopeo copy "$ref:git-$resolved_sha12" "$ref:$tag"` gains + `--multi-arch all`, and the coherence verify at `:880-885` switches from + `.config.digest` to the manifest-list digest. The §A4 ancestor-walk + resolver (`release.yml:830-870`) is digest-agnostic (it only probes tag + existence) and needs no change. + +### Decision D — ordering and partial-failure posture + +**Order (three phases, strictly sequenced):** + +1. **Per-arch build+push, in parallel.** `publish-image-amd64` + (`ubuntu-latest`) and `publish-image-arm64` (`ubuntu-24.04-arm`) each run + the full bootstrap and `publish.sh` in per-arch mode, pushing only + `:git--`. They share no tag, so they need no mutual + serialization. +2. **Compose+push the index.** A third job, `needs:` both, composes the index + from the two per-arch tags **by digest** (re-inspect each per-arch tag, + pin the member digests into the index — never by tag, so a race cannot + swap a member) and pushes `:git-` first, then `:latest`, preserving + `publish.sh:54-59`'s pin-before-moving-tag order. +3. **Verify.** The platform-set assertion and list-digest coherence check + (Decision C.3/C.4), in the compose job. + +Only the compose job carries the `publish-agent-image` concurrency group +(`release.yml:110-113` semantics: `cancel-in-progress: false`, `queue: max`) — +it is the only writer of shared tags, and the release-time `release-image` job +already shares that group (`release.yml:723-726`). + +**Partial-failure analysis, preserving the "immutable `:git-*`, abort on +ambiguity" posture:** + +- One arch job fails → the compose job never runs; `:git-` and + `:latest` do not move. Consumers see the previous coherent state. A + stranded `:git--` tag is harmless: it is immutable, + digest-pinned, and consumer-invisible (nothing pulls `-` tags). + Re-running the workflow is idempotent — the stranded tag hits + `guard_immutable`'s matching-digest skip. +- Compose pushes `:git-` but fails before `:latest` → exactly today's + failure mode between the two `skopeo copy` iterations of + `publish.sh:128-159`; the re-run's index guard (C.2) skips the pin and + moves `:latest`. No new window is introduced. **This recovery is only as + good as C.2's skip arm, which is UNVERIFIED pending OQ-5**: if index + composition is not byte-deterministic the re-run hard-fails instead of + skipping, and the fallback identity in C.2 (assert the member digest set + rather than the list digest) is what restores the clean re-run. +- A registry blip during any guard probe → abort without pushing, verbatim + `guard_immutable` posture. + +**Index-composition tool.** `skopeo` cannot assemble an index. Options: +`podman manifest create/add/push` (preinstalled on GitHub runners, honors +`REGISTRY_AUTH_FILE`), `buildah manifest`, or writing the OCI index JSON and +pushing it raw. **Recommendation: `podman manifest`**, digest-pinned members, +with the pushed bytes re-inspected for the guard digest. UNVERIFIED: that the +GitHub-runner podman version pushes an OCI-mediaType index GHCR serves +correctly to both podman and Apple `container`; T2's spike includes this +end-to-end pull check. + +### What does not change + +- The root `flake.nix` (`systems = [ "x86_64-linux" ]`, `flake.nix:33`) — the + agent image does not build through it (see Context). The + `flake.nix:138-141` aarch64-darwin TODO is out of scope. +- `agent-image/devenv.yaml` / `devenv.lock` inputs — same fork revs, evaluated + for a second system. +- `guard_immutable`'s logic and the auth/`REGISTRY_AUTH_FILE` pin. +- The moon gate (`agent-image/moon.yml`) stays amd64-only as the PR-time + build-health signal; an arm64 PR gate would double the dominant CI cost for + drift classes T2 shows are rare. Revisit only if arm64-only breakage + recurs (Open Questions). + +## Plan + +### Global Constraints + +- Tag contract: `:git-` (immutable) and `:latest` (moving) remain the + only consumer-facing per-push tags; `:vX.Y.Z` remains release-time. Per-arch + tags `:git--{amd64,arm64}` are internal and immutable; no `-latest` + per-arch moving tags. +- Platform set: exactly `{linux/amd64, linux/arm64}`, asserted, not implied. +- No guard is deleted or weakened; every existing property gets its + list-level equivalent (`rule://no-inert-gating`). +- All registry writes stay `GITHUB_TOKEN` + `REGISTRY_AUTH_FILE`-pinned, + least-privilege `packages: write`, main-ref-guarded, inside the + `publish-agent-image` concurrency group for shared-tag writers. +- Runners: GitHub-hosted only (`ubuntu-latest`, `ubuntu-24.04-arm`); no + self-hosted, no QEMU. + +### T1 — per-system `entrypoint.nix` (FOD hash and native-addon copy) + +Two edits in `agent-image/entrypoint.nix`, both required before an arm64 build +can succeed. + +First, make `nodeModules.outputHash` a per-system attrset keyed by +`pkgs.stdenv.hostPlatform.system`, following the shape of +`tools/toolchain/versions/bun.nix` (`"x86_64-linux"` / `"aarch64-linux"` +entries). The aarch64 hash is obtained the way the file's own comment +prescribes (set `lib.fakeSha256`, take the reported value) — on the T2 spike +runner, since the hash is what the arm64 install tree produces. + +Second, parameterize the native-addon copy block at `:218-220` per system. It +hardcodes the arch three times: the `pi-natives-linux-x64` package path and +both `pi_natives.linux-x64-{modern,baseline}.node` filenames. The variant +scheme is also not portable: per the block's own comment (`:214-217`) the +loader picks `modern` on an AVX2 host else `baseline`, and AVX2 is x86-only. +So arm64 needs its real variant names rather than a renamed pair, and the +"copy both" rule holds only if arm64 ships two. T2 reports the actual package +contents; this task consumes that answer. + +**T1 does not merge on its own.** Both edits need a real aarch64 value that +only the T2 runner can produce, so landing the seam alone would put a +`lib.fakeSha256` placeholder and an unresolved copy block on `main` where +nothing selects them until the arm64 lane exists — config that provably does +nothing, which `rule://no-inert-gating` forbids. T1 is authored against the +T2 spike and lands with T2's measured hash and variant names in the same +change, or it waits for the T4 cutover. T2 itself is dispatch-only and writes +no tags, so it is not a merge gate for anything else. + +Interfaces: consumes `pkgs` (already in scope); produces the same `outputHash` +string and the same two `.node` files in `$out` per system. The amd64 hash and +copied filenames stay byte-identical to today's. + +### T2 — arm64 build spike (workflow_dispatch, no tag writes) + +A temporary `workflow_dispatch`-only job on `ubuntu-24.04-arm`: full bootstrap +(nix, pinned bun, patched skopeo), then +`nix run -- container build agent` in `agent-image/`, then +`skopeo inspect --raw nix:$SPEC` asserting `architecture == arm64`. No +registry writes. Also: compose a throwaway index in a scratch tag under the +actor's namespace and pull it with podman to close the UNVERIFIED +index-serving question. Success criteria: spec builds, config architecture is +arm64, wall-clock recorded against the 90-minute ceiling, native addon +present in the bundle. + +Interfaces: consumes T1; produces a go/no-go plus the aarch64 FOD hash and a +wall-clock number that sizes the arm64 job's timeout. + +### T3 — `publish.sh` per-arch mode + +Add flags (e.g. `--arch-suffix ` implying suffix-tagged pushes and no +`:latest`): the default tag computation (`publish.sh:54-59`) becomes +`git--` only. Guard logic untouched. Bare invocation keeps +today's behavior until T4 cuts over, then bare invocation is removed with the +cutover (no dead mode left behind). + +Interfaces: consumes `REGISTRY_AUTH_FILE`, arch flag; produces the immutable +per-arch tag, guard-verified. + +### T4 — `release.yml` fan-out + compose job + +Split `publish-image` into `publish-image-amd64` / `publish-image-arm64` +(identical steps, `runs-on` differs, both gated by the same +`IMAGE_CLOSURE_PATHS` in-job gate) and add `publish-image-manifest` +(`needs:` both, `ubuntu-latest`, `publish-agent-image` concurrency group): +digest-pinned `podman manifest create/add`, push `:git-` then +`:latest`, with the C.2 index-immutability guard before the pin push and the +C.3/C.4 verify replacing `release.yml:314-346`. + +Interfaces: consumes T3's script mode; produces the two index tags plus the +verify assertions. The old single `publish-image` job is removed in this same +change. + +### T5 — `release-image` multi-arch re-tag + +`skopeo copy --multi-arch all` at `release.yml:874-875`; coherence verify at +`:880-885` switches to manifest-list digests. Ancestor-walk resolver +unchanged. + +Interfaces: consumes T4's published index; produces `:vX.Y.Z` as the same +index. + +### T6 — docs + record cross-reference sweep + +Update `docs/architecture`/record references to the `linux/amd64 single-arch` +contract where they describe the *current* lane (not frozen decisions), and +note in the Apple-container record's OQ-8 successor context that the arm64 +image now exists. Frozen records are not rewritten; this record is the new +authority for the platform contract. + +## Tasks + +- [ ] T1 — per-system `entrypoint.nix`: FOD hash + native-addon copy block +- [ ] T2 — arm64 build spike on `ubuntu-24.04-arm` (dispatch-only, no pushes) +- [ ] T3 — `publish.sh` per-arch mode +- [ ] T4 — `release.yml` fan-out + index compose/verify jobs +- [ ] T5 — `release-image` `--multi-arch all` re-tag + list-digest verify +- [ ] T6 — docs sweep + +Ordering: T1 → T2 (spike needs the per-system hash seam to obtain the arm64 +hash) → T3 → T4 → T5; T6 with T4/T5. T2 is the gate: if the toolchain does not +bootstrap on aarch64-linux, findings come back to this record's Open Questions +before any workflow change lands. + +## Open Questions + +- **OQ-1 [load-bearing until T2] — does the pinned toolchain run on + aarch64-linux?** Nobody has run the devenv fork + nix2container fork + + patched skopeo on an arm64 Linux host. Nothing found pins x86_64 + (`skopeo-nix2container-env.nix` is system-implicit; `versions/bun.nix` + carries an aarch64-linux entry), but this is designed-to-be-true, not + observed-true. T2 resolves it empirically; the recommendation stands only + if T2 is green. +- **OQ-2 [load-bearing until T2] — GHCR index serving to both consumers.** + That a podman-composed OCI index at GHCR resolves correctly for podman on + linux/amd64 hosts and Apple `container` on arm64 macOS guests. T2's scratch + pull closes it. +- **OQ-3 [non-load-bearing] — arm64 in the PR-time moon gate.** Deferred: + the moon `build` gate stays amd64. If T2/production show arm64-only + breakage classes (FOD drift, addon prebuilt gaps), an affected-scoped arm64 + gate is a later add; the publish lane's own arm64 job reds post-merge + either way. +- **OQ-4 [non-load-bearing] — aarch64 binary-cache population.** If T2's + wall-clock is painful, publishing the arm64 closure to a Rigel cachix cache + is a follow-up optimization; correctness does not depend on it. +- **OQ-5 [load-bearing until T2] — is index composition byte-deterministic?** + C.2's immutability guard and Decision D's clean re-run both assume that + recomposing the index from the same two member digests produces identical + bytes, and so an identical list digest. The OCI spec does not canonicalize + an index, so member order or an injected annotation could break it. T2 + composes, pushes to a scratch tag, recomposes, and compares list digests. + If it is not deterministic, C.2's identity becomes the member digest set + instead of the list digest — the immutability property is preserved either + way, so this changes the guard's mechanism, not the design. diff --git a/docs/designs/ui/compass-glyph-primitives/design.md b/docs/designs/ui/compass-glyph-primitives/design.md new file mode 100644 index 000000000..33dceb2ba --- /dev/null +++ b/docs/designs/ui/compass-glyph-primitives/design.md @@ -0,0 +1,498 @@ +# Compass glyph primitives (RIG-3603) + +Status: Active +Ledger: DL-367 +Owner lane: compass-ux (design) → compass-ui (execution) +Refs: RIG-3603. Adopts the technique frozen by DL-150 (state dot: 9×9 +`crispEdges` 1-bit SVG) and DL-199 (badge: pixel-art 1-bit glyph + 2-char mono +axis code). All code references are to main `97741c5ce2f4`. + +## Problem / Intent + +UI chrome renders ~23 non-ASCII glyphs (`▦`, `🗀`, `⎇`, `⇄`, `▸`, `◆`, `★`, …) +that neither brand face (Departure Mono, Space Mono) covers, so the e2e font +stack pins a ~14 MB Unifont fallback (`tools/toolchain/chromium-e2e-env.nix`) +and end users get whatever their OS substitutes. Separately, +`ActivityBarItem.icon: string` conflates two different primitives — a fixed +chrome symbol and a person's initial — behind one string field. This record +replaces chrome glyphs with dot-matrix `crispEdges` SVG per DL-150/DL-199, +splits the item type at the item per Matt's frozen ruling, and states the +conditions under which the Unifont pin can be retired. + +## Global Constraints + +- **Solid v2 (`solid-js@2.0.0-rc.1`), TS strict, Biome (tabs).** Props are + never destructured in components. +- **Technique is frozen, not chosen here.** DL-150/DL-199 already ruled: 1-bit + whole-cell grids, `shape-rendering="crispEdges"`, one `` per lit cell, filled with `currentColor` so tier colors keep + applying through the wrapper (`StateDot.tsx`, `BadgeGlyph.tsx` are the + shipped exemplars). +- **Night Owl palette via `--cx-*` tokens only.** No raw hex; glyphs carry no + color of their own — `currentColor` end to end. +- **The even-11px-multiple rule governs Departure Mono font sizes, not SVG + boxes** (`tokens.css:51-57` — it is about UPM font-pixels). The glyph grid + therefore neither inherits nor fixes the pre-existing 15px `.r-tab-icon` + violation (`app.css` `.r-tab .r-tab-icon`). What the grid MUST satisfy is + whole-pixel placement inside the 34px `.r-tab`, so its 1px cells do not + straddle device pixels. +- **Item-level union, per Matt's frozen ruling.** `ActivityBarItem` splits AT + THE ITEM: `{ kind: "glyph"; … }` vs `{ kind: "avatar"; … }`. A field-level + union (one `icon` field with two shapes) was explicitly rejected. Both the + refactor and the glyph work land in RIG-3603. +- **Out of scope for SVG conversion:** the 4 comment-only glyphs (`─` U+2500, + `≙` U+2259, `⌘` U+2318, `═` U+2550); math/punctuation glyphs in comments and + test names (`§`, `⇒`, `≥`, …), which never render. +- **The `stub-data.ts` fixture log glyphs are not converted, but they are not + free either.** `➜` (U+279C) and `▪▪▪▪` render through `.term-body`, which + uses `var(--cx-font-ui)` (`app.css:1273`) — there is NO separate terminal + font stack — and `agent.png` captures that exact fixture + (`AGENT_ID = "acc-compass-ui"`). So after the pin is removed they would + rasterize as tofu. They stay text, but T8 must first replace the two fixture + strings with ASCII equivalents (`->`, `####`); the module hand-fakes a + representative fleet, so the strings are not a contract. +- **Unifont-pin removal is a separate stacked PR**, and it is NOT the only + recapture. T4/T5 change pixels inside existing baselines, so the glyph lane + recaptures at T6; T8 then recaptures again against the shrunk font set. +- Markdownlint-clean record; Simple Technical English. + +## Approach + +### The `` primitive + +Adopt the DL-150/DL-199 technique as-is. A new +`apps/ui/src/components/Glyph.tsx` renders a fixed **11×11** 1-bit grid at one +CSS px per cell: + +```tsx + +``` + +11×11 (not StateDot's 9×9) for two reasons: an odd grid has a true center cell, +matching the 9×9 exemplars, and pictographs (a folder, a branch, a gear) need +more cells than a state dot. + +This does **not** reconcile the pre-existing 15px slot violation — that rule +governs Departure Mono font sizes via UPM font-pixels, and an SVG box has no +UPM (D2). The real hazard is placement: an 11px box flex-centered in the 34px +`.r-tab` lands on a half-pixel offset, smearing every 1px cell across two +device pixels, so the box is pinned to a whole-pixel offset (T3+T4). + +Bitmap storage follows `BadgeGlyph.tsx` exactly: a module-level +`GLYPH_CELLS: Record>` +of `[x, y]` lit cells, transcribed from frozen ASCII grids added to +`design/components.md` §Glyphs (`#` = lit). Keying the table on the exhaustive +`GlyphName` union makes a name without a bitmap a compile error, not a runtime +blank — the same guard `BadgeGlyph`'s `GlyphKey` provides. + +`GlyphName` starts as the closed set of the four static tabs — +`"status" | "files" | "vcs" | "pr"` — and grows semantic names (never +character names) as the chrome audit (T5) converts further sites. + +### The `ActivityBarItem` split (Matt's frozen ruling) + +The union splits **at the item**, in `apps/ui/src/constants.ts`: + +```ts +interface ActivityBarItemBase { + title: string; + group: RightTabGroup; +} + +/** A static tab: a fixed chrome symbol from the closed glyph set. */ +export interface GlyphTabItem extends ActivityBarItemBase { + kind: "glyph"; + id: StaticRightTab; + name: GlyphName; +} + +/** A fleet agent tab: a person's initial, derived once from the handle. */ +export interface AvatarTabItem extends ActivityBarItemBase { + kind: "avatar"; + id: `agent:${string}`; + letter: string; + group: "fleet"; + agentId: string; + unreachable?: boolean; +} + +export type ActivityBarItem = GlyphTabItem | AvatarTabItem; +``` + +The split also moves `agentId`/`unreachable` onto the avatar arm only — today +they are optional on every item, but only fleet agent tabs ever carry them +(`RightSidebar.tsx` `FleetPane`/`AgentUnreachable` and the `StateDot` badge all +key off `agentId`). `FleetPane`, `AgentUnreachable`, and `activeFleetItem` +narrow from `ActivityBarItem` to `AvatarTabItem`, deleting their +`agentId ? … : undefined` dances. + +The avatar arm owns the initial derivation **once**. Both +`fleetItemForAgent` and `unreachableFleetItem` currently duplicate +`(handle.at(0) ?? "?").toUpperCase()`, which splits a surrogate pair and can +expand to multiple characters for some scripts. A single helper replaces both: + +```ts +/** First grapheme of the handle, NFKD-normalized with combining marks + * stripped (É→E), uppercased, then clamped to one printable ASCII + * character; anything else degrades to "?" (see D1). */ +export function avatarInitial(handle: string): string; +``` + +Implementation, in order: `Array.from(handle)[0]` (code-point safe, not +`.at(0)`, which splits a surrogate pair) → `.normalize("NFKD")` and drop +combining marks → `.toUpperCase()` → return `"?"` unless the result is exactly +one character in the printable-ASCII range. + +The NFKD step is what keeps accented-Latin handles distinguishable (`Émile`→`E` +rather than `?`); the clamp is what lets the Unifont pin retire. Both are D1. + +### The render site and the CSS split + +The single render site (`RightSidebar.tsx`, the `{tab.icon}` span inside the +`.r-tab` button) branches on the discriminant. `tab` is a plain `For` item, so +a ternary narrows cleanly: + +```tsx + +``` + +The shared `.r-tab .r-tab-icon { font-size: 15px }` rule splits by `data-kind`: + +- `.r-tab .r-tab-icon[data-kind="glyph"]` — an 11×11 box (`display: inline-flex`, + no font properties; the SVG is the content), centered in the unchanged + `.r-tab` button. +- `.r-tab .r-tab-icon[data-kind="avatar"]` — text, DS mono face at 15px + (Space Mono is legal at ≤16px), `line-height: 1` as today. + +The `StateDot` sibling and its absolute bottom-right badge CSS are untouched. + +### Chrome glyph audit + +Beyond the activity bar, rendered chrome glyphs (`▸` disclosure, `◆`/`★`/`☆` +markers, `✓`/`✗` status, `⚙` settings, `⊞` grid, `✕` close, `⟨⟩` framing, and +the rest of the ~23-site census) convert to `` inline at their +sites, extending `GlyphName` per adopted symbol (T5). Fixture log content in +`stub-data.ts` and comment-only glyphs stay untouched (Global Constraints). +Whether typographic marks that a brand face already covers (e.g. `−`, `•`) may +stay as text is settled by D3. + +### Unifont retirement conditions + +The pin (`tools/toolchain/chromium-e2e-env.nix`, fontDirs: unifont + +unifont_upper) can be removed only when **no rendered character falls outside +brand-face coverage**, which requires ALL of: + +1. Every rendered chrome glyph converted or verified covered (T5 audit). +2. The avatar arm clamped to printable ASCII (`avatarInitial`, D1) — handles + are charset-unconstrained (`agent.proto` `from_handle`, no schema + validation), so without the clamp an arbitrary initial re-opens the gap. +3. Fixture log glyphs (`➜`, `▪▪▪▪`) replaced with ASCII in `stub-data.ts` + (T8 step 1). They are content, not chrome, so they are never converted to + SVG — but they are NOT exempt from coverage: `.term-body` renders through + `var(--cx-font-ui)` (`app.css:1273`), the same stack as everything else, + and `agent.png` captures them. There is no terminal font stack to fall back + on. + +Removal changes the fontconfig nix store path and invalidates all 11 visual +baselines in `apps/ui/e2e/__screens__/`, so it lands as its own stacked PR +with its own recapture (T8) — never mixed into a glyph slice. Note this is the +SECOND recapture: T4/T5 already move pixels inside the same baselines, which +T6 recaptures first. + +## Alternatives considered + +- **Field-level union (keep one `icon` field, vary its type).** REJECTED by + Matt, explicitly and finally. A field union leaves every consumer testing the + field's shape instead of the item's kind, keeps `agentId`/`unreachable` + optional on items that can never carry them, and gives the avatar arm no + natural owner for the initial derivation. Do not re-litigate. +- **Keep the Unifont pin permanently.** Rejected: ~14 MB of toolchain weight + to paper over a small closed set of chrome symbols, and it only fixes the + e2e environment — end-user machines never had the pin, so real browsers + already render these glyphs from arbitrary OS fallback fonts, off-brand and + inconsistent. The pin hides the problem from the one environment that could + catch it. +- **A webfont/icon-font of custom glyphs.** Rejected: re-introduces font + loading, hinting, and anti-aliasing where DL-150/DL-199 already shipped a + crisper, zero-asset answer; a font also cannot guarantee whole-cell 1-bit + rendering at exactly one CSS px per cell. +- **9×9 grid matching StateDot.** Rejected for chrome glyphs: it has no + center-cell advantage over 11×11 (both are odd), and it gives pictographs — + gear, folder, branch — too few cells to read at tab size. The baseline + argument does NOT apply: neither 9 nor 11 resolves the 15px slot violation, + because that rule governs Departure Mono font sizes, not SVG boxes (D2). + +## Plan + +Tasks in dependency order. T1–T7 are one PR lane (RIG-3603); T8 is a separate +stacked PR. + +### T1 — `` primitive and bitmap data + +Add `apps/ui/src/components/Glyph.tsx`: the `GlyphName` union, the +`GLYPH_CELLS` table for the four static-tab glyphs (status grid, files folder, +vcs branch, pr arrows), and the component rendering the 11×11 `crispEdges` +SVG on `currentColor`, following `BadgeGlyph.tsx`'s shape. Transcribe the four +frozen ASCII grids into `design/components.md` §Glyphs; `GLYPH_CELLS` cites +them. Unit-test that every `GlyphName` key yields a non-empty cell list and +all cells lie in `0..10`. + +Interfaces: + +```ts +export type GlyphName = "status" | "files" | "vcs" | "pr"; +export const Glyph: Component<{ name: GlyphName }>; +``` + +### T2 — `avatarInitial` helper + +In `apps/ui/src/constants.ts`: the single owner of initial derivation, today +duplicated at `:137` and `:154`. Take `Array.from(handle)[0]` (never `.at(0)`, +which splits a surrogate pair), NFKD-normalize it and strip combining marks so +`É`→`E`, uppercase, then clamp to one printable ASCII character, else `"?"` +(D1). +Unit-test: plain handle, empty string, surrogate-pair-leading handle (e.g. an +emoji → `?`), a lowercase handle, an accented Latin handle (`Émile` → `E`), a +non-Latin script (`Живко` → `?`), and a script whose uppercase expands to +multiple characters. + +Interfaces: + +```ts +export function avatarInitial(handle: string): string; +``` + +### T3+T4 — `ActivityBarItem` union, constructors, render site and CSS (ONE commit) + +**These cannot be separate commits.** `GlyphTabItem` has neither `icon` nor +`agentId`, and the activity-bar loop reads BOTH on the un-narrowed union: +`{tab.icon}` at `RightSidebar.tsx:689` and +`tab.agentId ? store.agentById(tab.agentId) : undefined` at `:662`. Landing the +union without the render branch is a TS-strict error, so a split leaves a red +commit mid-stack and breaks bisection. + +In `apps/ui/src/constants.ts`: replace the single interface with the +`GlyphTabItem | AvatarTabItem` union (shapes as frozen in `## Approach`). +`RIGHT_SIDEBAR_TAB_BY_ID` entries become `kind: "glyph"` with `name` replacing +`icon`; `fleetItemForAgent`/`unreachableFleetItem` return `AvatarTabItem` with +`letter: avatarInitial(…)`. Narrow `FleetPane`, `AgentUnreachable`, and +`activeFleetItem` in `RightSidebar.tsx` and the `rightTabGroups` memo typing in +`store.ts` to the arm they actually handle. + +At the render site: the `.r-tab-icon` span gains `data-kind={tab.kind}` and +branches `tab.kind === "glyph" ? : tab.letter`; the +`:662` agent lookup moves behind a `tab.kind === "avatar"` guard. In `app.css` +split `.r-tab .r-tab-icon` into the `[data-kind="glyph"]` box (no font +properties) and the `[data-kind="avatar"]` 15px mono rule; `.r-tab +.cx-state-dot` untouched. The glyph box MUST sit at a whole-pixel offset inside +the 34px `.r-tab` — an 11px box flex-centered there lands on a half pixel and +smears every 1px cell across two device pixels (see D2). + +Unit-test both constructors: a resolvable agent yields `kind: "avatar"` with no +`unreachable`; a pin yields `unreachable: true`; both route the initial through +`avatarInitial`. Component-test BOTH arms and the whole-pixel offset — no test +covers this surface today. + +Interfaces: + +```ts +export interface GlyphTabItem { + kind: "glyph"; + id: StaticRightTab; + name: GlyphName; + title: string; + group: RightTabGroup; +} +export interface AvatarTabItem { + kind: "avatar"; + id: `agent:${string}`; + letter: string; + title: string; + group: "fleet"; + agentId: string; + unreachable?: boolean; +} +export type ActivityBarItem = GlyphTabItem | AvatarTabItem; +export function fleetItemForAgent(agent: Agent): AvatarTabItem; +export function unreachableFleetItem(pin: PinnedAgent): AvatarTabItem; +``` + +```tsx +// RightSidebar.tsx render-site contract + +``` + +Component tests (the missing regression net): a static tab renders an SVG with +`shape-rendering="crispEdges"` and no text; a fleet tab renders the initial as +text plus its `StateDot`; an unreachable tab renders no `StateDot`. + +### T5 — chrome glyph audit and conversion (THREE commits) + +Sweep the rendered non-ASCII chrome sites (census at main `97741c5ce2f4`: +`LeftSidebar.tsx` `◆ ★ ☆ ▸ ▼ ◉ ○ ▦ ▤ ✓ ⚙`, `RightSidebar.tsx` +`▸ ▾ 🗀 ⎇ ✓ ✗ • − ·`, `App.tsx` `◇ ▦ ▐ ▌`, `AgentView.tsx` `⊞ ▁ ▏ ✕`, +`UsageBar.tsx` `⎇`, `LogPanel.tsx` `■ ⟨⟩`, `SessionTrace.tsx` `↗`, +`BacklogView.tsx` `▸`, `IssueCard.tsx`/`DoneView.tsx` `−`). The `·` in +`RightSidebar.tsx:32` `FILE_ICON` is part of the census — Space Mono covers +it, so it is a "kept, covered" row, not a conversion. + +**Split into three commits by surface**, each independently eyeball-able: +T5a `LeftSidebar` + `App`; T5b `RightSidebar` + `AgentView`; T5c the rest. +Pixel-art authorship for ~15 bitmaps is judgment work, and one review of 10 +files × 15 grids is past the size where a reviewer checks each grid against +its rendering. The audit table accretes per commit. + +Per site: convert to `` with a new semantic `GlyphName` + +bitmap, or record it in the audit table as covered-by-brand-face/text-kept +(per D3: convert wholesale). Extend the T1 cell-validity test automatically via the +keyed table. + +The audit table carries an **a11y column** per site: *decorative* +(`aria-hidden`, no change) or *name-bearing* (the site must gain or keep a +text/`aria-label` name). Without it each conversion silently decides whether a +glyph carried meaning — e.g. `LogPanel.tsx` `■ stop` keeps its text name, but +bare `✓`/`✗` verdict marks read as nothing once hidden. + +Its completeness criterion is checkable, not trusted: **every match of a +non-ASCII sweep over `apps/ui/src` at the pinned rev** appears as a row. + +Deliverable includes the audit table appended as a section in this record, not +a new file elsewhere. + +Interfaces: + +```ts +// GlyphName grows per adopted symbol; names are semantic ("disclosure", +// "check"), never character names ("triangle-right"). Example after T5: +export type GlyphName = + | "status" | "files" | "vcs" | "pr" + | "disclosure" | "disclosure-open" | "check" | "cross" | "close" + | "gear" | "star" | "star-outline"; // …exact set fixed by the audit table +``` + +### T6 — baseline recapture for the glyph conversion + +T3+T4 and T5 change rendered pixels inside committed baselines: +`right-sidebar.png` clips `aside.right`, which holds the activity bar T3+T4 +rewrites, and the seven `fullPage: true` captures (`bridge`, `bridge-empty`, +`agent`, `backlog`, `done`, `settings`, `bridge-prs`) hold the LeftSidebar and +App glyphs T5 converts. So this lane must recapture before it can go green; +without this task the T1–T5 PR fails the visual gate with no step that fixes +it. + +Recapture via the CI regen lane (`ci.yml` `workflow_dispatch`, `regen: visual`), +then eyeball every diff: only glyph cells may move. `state-dot.png` is the +positive control and MUST NOT change — it is textless, so any diff there means +the conversion leaked into unrelated rendering. + +Interfaces: + +```text +apps/ui/e2e/__screens__/ — baselines recaptured for the glyph conversion +state-dot.png — positive control, byte-identical before and after +``` + +### T7 — docs + +Update `design/components.md` §Glyphs with the full frozen grid set and the +two-arm activity-bar item contract; note the DL-367 adoption in the section +header. No changelog beyond the record. + +Interfaces: + +```text +design/components.md §Glyphs — frozen ASCII grids, one per GlyphName ("#" = lit) +``` + +### T8 — Unifont pin removal + baseline recapture (SEPARATE STACKED PR) + +Gated on T6 complete; D1's clamp is what makes it possible. Three steps, in order: + +1. Replace the `stub-data.ts` fixture log glyphs with ASCII (`➜` → `->` at + :525-526, `▪▪▪▪` → `####` at :535-537 and :722). Without this the recapture + bakes tofu into `agent.png`, because `.term-body` has no separate font stack. +2. Remove `unifont` and `unifont_upper` from + `tools/toolchain/chromium-e2e-env.nix` fontDirs. +3. Recapture all 11 baselines; eyeball each diff for tofu/substitution before + accepting. + +The PR carries nothing beyond these, so a rendering regression bisects to the +font change alone — relative to the post-T6 baselines, not to today's. + +Before opening it, re-run the census: no rendered character may fall outside +SpaceMono + departure-mono coverage. That is the real retirement condition; T6 +and D1 are necessary, not sufficient. + +Interfaces: + +```text +apps/ui/src/stub-data.ts — fixture log strings ASCII-only +tools/toolchain/chromium-e2e-env.nix — fontDirs shrinks to SpaceMono + departure-mono +apps/ui/e2e/__screens__/ — 11 baselines recaptured against the shrunk font set +``` + +## Tasks + +- [ ] T1: `` primitive + 4 static-tab bitmaps + `components.md` grids + cell-validity test +- [ ] T2: `avatarInitial` helper + unit tests (surrogate pair, empty, accented Latin, non-Latin, multi-char uppercase) +- [ ] T3+T4 (ONE commit): `ActivityBarItem` → `GlyphTabItem | AvatarTabItem` union, both constructors, the render branch for `{tab.icon}` (`:689`) AND `tab.agentId` (`:662`), the `.r-tab-icon` CSS split with a whole-pixel glyph box, and tests for BOTH arms +- [ ] T5a: chrome glyph conversion — `LeftSidebar` + `App` (audit table rows, a11y column) +- [ ] T5b: chrome glyph conversion — `RightSidebar` + `AgentView` +- [ ] T5c: chrome glyph conversion — remaining surfaces; audit table complete against a non-ASCII sweep +- [ ] T6: baseline recapture for the glyph conversion (`state-dot.png` unchanged) +- [ ] T7: `design/components.md` §Glyphs update (DL-367) +- [ ] T8: Unifont pin removal + 11-baseline recapture — SEPARATE stacked PR, gated on T6 + D1 + +## Resolved decisions + +- **D1 — avatar fallback: NFKD-normalize, then clamp to ASCII.** Ruled by Matt. + `avatarInitial` takes the first grapheme, NFKD-normalizes it and strips + combining marks (`É`→`E`, `Ó`→`O`), uppercases, then clamps to one printable + ASCII character, falling back to `?` only for scripts no Latin letter + represents (CJK, Cyrillic, emoji). Handles are charset-unconstrained (proto + `from_handle`, no schema validation), and the initial exists to tell agents + apart (`constants.ts:127-130`: "a per-agent glyph, no hardcoded Supervisor + ◆") — a bare clamp would collapse distinct non-ASCII handles to identical `?` + tabs. A font fallback was rejected: it keeps the pin forever and makes T8 + impossible. The tab's `title`/`aria-label` carries the full handle in every + case. **This is what lets the Unifont pin retire.** +- **D2 — glyph cell grid: 11×11.** Ruled by Matt. An odd grid gives a true + center cell like the 9×9 exemplars, and the extra resolution matters for + pictographs (gear, folder, branch). StateDot and BadgeGlyph stay frozen at + 9×9 per DL-150/DL-199; they are physically separate components, so two grid + sizes coexisting is fine. + 11×11 does **not** resolve the pre-existing 15px slot violation — the + even-11px-multiple rule (`tokens.css:51-57`) governs Departure Mono font + sizes via UPM font-pixels, and an SVG box has no UPM. An earlier draft of + this record claimed it did and was wrong. + The real hazard is placement: an 11px box flex-centered in the 34px `.r-tab` + lands on a half-pixel offset, smearing every 1px `crispEdges` rect across two + device pixels. **T3+T4 MUST pin the glyph box to a whole-pixel offset and + assert it.** A 12×12 grid would center integrally for free but loses the + center cell; rejected on that trade. +- **D3 — audit breadth: convert wholesale.** Ruled by Matt. Every chrome symbol + becomes a ``, including ones Space Mono already covers. One + vocabulary is cheaper to hold than a coverage table, and a per-character + check against two font files is fragile and invisible in review. The cost is + accepted: converting covered glyphs adds baseline churn without adding + coverage. Typographic marks inside prose (`−` in a diff stat, `•` as a + separator, `·` in `FILE_ICON`) stay text where the brand face covers them — + the audit table records each keep with its reason. +- **D4 — avatar slot type size: keep 15px.** Non-load-bearing, deferred by + design. The avatar arm keeps 15px mono; a later pass may move it to 16px + (still ≤16px Space Mono) or an 11px-multiple display treatment. This record + changes the avatar arm's type, not its look. diff --git a/runner-image/.dockerignore b/runner-image/.dockerignore new file mode 100644 index 000000000..4080d8467 --- /dev/null +++ b/runner-image/.dockerignore @@ -0,0 +1,6 @@ +# Context prune for the runner image build. The context is runner-image/ itself, +# which holds only the Dockerfile, the staged store/ closure and the +# compass-runner entrypoint symlink — so this excludes the build OUTPUT (out/), +# which would otherwise be re-transferred into every rebuild's context and grow +# it without bound. +out diff --git a/runner-image/Dockerfile b/runner-image/Dockerfile new file mode 100644 index 000000000..166872001 --- /dev/null +++ b/runner-image/Dockerfile @@ -0,0 +1,101 @@ +# syntax=docker/dockerfile:1.7-labs +# The compass-runner container image (R1 of the Compass Runner containerization +# record). Carries the Runner binary, the KVM userland it exec's +# (cloud-hypervisor, virtiofsd, passt) and the three guest assets the microVM +# backend direct-boots, so a Runner pod needs no host toolchain — only /dev/kvm +# and the session-volume mount. +# +# A Dockerfile on a minimal base, not a nix2container image: the mechanism +# follows the image's RUNTIME, and the Runner is a prebuilt application that +# runs no toolchain. Full rationale in the design record's R1 task; nix still +# builds every carried artifact, and build.ts COPYs the result. + +# Distroless STATIC, not :base and not Alpine. The carried binaries are +# dynamically linked but nix-closed: each names an ABSOLUTE /nix/store +# interpreter and resolves every NEEDED library through its own RPATH, so the +# closure supplies its own ld.so and glibc and the base is never consulted for a +# library. That makes the base's own libc dead weight, and `static` is the +# smallest base that still provides what IS needed from it: /etc/passwd + +# /etc/group (the nonroot uid), CA certificates (the Runner dials the Server +# over TLS), and /tmp. Verified: with the closure present all four binaries +# execute; with ONLY cloud-hypervisor's own store path copied — its glibc +# omitted — the same exec fails "missing dynamic library", which is the control +# proving the loader comes from the closure rather than the base. +# +# The pinned digest is the nonroot variant (uid/gid 65532). The Runner needs no +# root: /dev/kvm access is granted by supplementary group at the pod layer. +# Pinned by digest, never by tag — GHCR-style tag mutability gives no +# immutability guarantee. +FROM gcr.io/distroless/static-debian12@sha256:c0f429e16b13e583da7e5a6ec20dd656d325d88e6819cafe0adb0828976529dc + +# The realised nix closure: the Runner binary, the KVM userland and the guest +# assets, each at its own absolute /nix/store path. ONE COPY of a +# build.ts-staged directory rather than a COPY per component, because the store +# paths reference each other by absolute path — splitting them across layers by +# component would interleave a binary and its glibc into different layers for no +# cache benefit, since a Go rebuild moves the Runner path and nothing else. +# +# Ownership is left at the default (root-owned, world-readable) and the image +# runs as nonroot: the closure is read-only at runtime, so a non-root process +# needs no write access to it. This also keeps the layer bit-identical to the +# staged tree, which is what makes the digest reproducible across rebuilds. +COPY store /nix/store + +# The stable entrypoint name is /nix/store/.compass-runner, a RELATIVE symlink +# build.ts stages inside store/ (so it arrives with the COPY above — there is no +# second COPY). ENTRYPOINT is exec-form and cannot expand a build ARG, while the +# Runner's own store path carries a hash that moves on every Go rebuild; the +# symlink gives the image a fixed entrypoint whose target moves with the build. +# +# It must be relative and inside the staged tree. An absolute symlink at the +# context root dangles on the BUILD HOST, and BuildKit checksums context entries +# before any COPY runs, so such a link fails the build outright with "not found" +# rather than resolving later inside the image. + +# Absolute store paths, resolved at build time from the staged closure by +# build.ts and passed in as build args. They are baked into ENV rather than +# hardcoded because every path carries a content hash that moves on any rebuild +# of its input. +# +# The Runner reads each of these as the documented env fallback for the matching +# --microvm-* flag (main.go registerBackendFlags), so an operator can still +# override any one on the command line without rebuilding the image. Setting +# them here is what makes the image self-describing: the microVM backend's +# preflight is fail-closed (it requires both a static support probe and a real +# boot canary), so a missing or wrong path fails at startup, loudly, rather than +# at first session. +ARG VMM_BIN +ARG VIRTIOFSD_BIN +ARG GUEST_KERNEL +ARG GUEST_ROOTFS +ARG GUEST_INITRD + +ENV COMPASS_RUNTIME_BACKEND=microvm \ + COMPASS_MICROVM_VMM=${VMM_BIN} \ + COMPASS_MICROVM_VIRTIOFSD=${VIRTIOFSD_BIN} \ + COMPASS_MICROVM_KERNEL=${GUEST_KERNEL} \ + COMPASS_MICROVM_ROOTFS=${GUEST_ROOTFS} \ + COMPASS_MICROVM_INITRD=${GUEST_INITRD} + +# passt is exec'd by name, not by configured path (unlike the VMM and virtiofsd, +# which have --microvm-* flags), so its bin dir must be on PATH. Prepended to +# the base PATH rather than replacing it. +# Consumed by the dockerfile frontend (paired with the export's +# rewrite-timestamp) to fix the config `created` field and the layer mtimes, so +# two builds of the same inputs yield the same digest. +ARG SOURCE_DATE_EPOCH + +ARG STACK_BIN_DIR +ENV PATH=${STACK_BIN_DIR}:/usr/local/bin:/usr/bin:/bin + +# ENTRYPOINT, not CMD: the image runs exactly one program, and an ENTRYPOINT +# lets a pod spec's `args` add flags without re-stating the binary path. Exec +# form, so the Runner is pid 1 and receives SIGTERM directly at pod shutdown — +# a shell wrapper would swallow it and leave the graceful drain to the kill +# timeout. +# Explicit, rather than inherited from the base's own config: a future digest +# bump that landed on the root `static` variant would otherwise silently produce +# a root-running container, and nothing here would catch it. +USER 65532:65532 + +ENTRYPOINT ["/nix/store/.compass-runner"] diff --git a/runner-image/moon.yml b/runner-image/moon.yml new file mode 100644 index 000000000..a96169df3 --- /dev/null +++ b/runner-image/moon.yml @@ -0,0 +1,72 @@ +# yaml-language-server: $schema=https://moonrepo.dev/schemas/project.json +# +# compass-runner-image (runner-image/): the Runner container image — the +# compass-runner binary, the KVM userland it exec's (cloud-hypervisor, +# virtiofsd, passt) and the three guest assets the microVM backend direct-boots. +# This project carries the Dockerfile and the staged build context; the +# realise-and-stage logic is tools/runner-image (TypeScript, not a shell script). +# +# A Dockerfile project, not a nix one like the sibling agent-image/ and +# guest-image/: their artifacts ARE nix derivations, this image's runtime is a +# distroless base. See the design record's R1 task. +# +# WHY THERE IS NO `ci` TASK. Unlike agent-image/ and guest-image/, this project +# deliberately does NOT ride the pre-merge gate: the build needs a reachable +# buildkitd (which no compass CI step carries today) and it +# stages a multi-GB closure dominated by the guest rootfs. Registering a `ci` +# task now would add an unrunnable task to the gate, so the gate coverage this +# project has today is the pure-core suite in tools/runner-image, which is where +# a mapping drift would actually be caught. The publish lane adds the first CI leg that builds this image. +layer: 'application' +language: 'bash' +# ci-group.nix with the sibling image projects: the CI matrix requires every +# project to carry exactly one group tag, and this project's real build closure +# is the nix one those legs already provision. It contributes no CI task today +# (`build` is runInCI:false), so the tag places it in the graph rather than +# adding a leg. +tags: ['ci-group.nix'] + +workspace: + inheritedTasks: + # Not a bun package — no package.json, no bun.lock — so the tag-bun `install` + # must never run here; `lint` and `format` are whole-repo tasks on the root + # project, never inherited. The same defensive guard the sibling image + # projects carry. + exclude: ['install', 'lint', 'format'] + +tasks: + build: + # Realise the closure and build the image. runInCI:false — the build needs a + # buildkitd and a multi-GB nix realise, so this is the dev-box/publish entry + # point, not a gate task. cache:false — nix owns the store-path caching and + # buildkit owns the layer caching, so moon must not false-green a build it + # did not re-run (the posture the sibling image projects take). + command: 'bun tools/runner-image/build.ts' + options: + cache: false + runInCI: false + runFromWorkspaceRoot: true + # The image's true build closure. A leading `/` is workspace-root-relative + # (the convention the sibling image projects use); the bare glob is this + # project's own tree. Each entry is an input whose change alters the built + # image: the Dockerfile and the build script directly, the Go module + + # runner command for the binary, and the guest-image/VMM-env closures for + # the carried assets. A glob that misses one would leave a stale image + # passing as current. + # Named tracked files, NOT a `**/*` glob. moon hashes with its native walker, + # which reads contents directly and does not skip gitignored paths — a bare + # glob here would hash the multi-GB staged closure this task itself writes + # into store/, on every affected-computation. + inputs: + - 'Dockerfile' + - '.dockerignore' + - 'moon.yml' + - '/tools/runner-image/**/*' + - '/flake.nix' + - '/flake.lock' + - '/go/go.mod' + - '/go/go.sum' + - '/go/cmd/compass-runner/**' + - '/go/internal/**' + - '/guest-image/**' + - '/tools/toolchain/microvm-vmm-env.nix' diff --git a/tools/runner-image/biome.json b/tools/runner-image/biome.json new file mode 100644 index 000000000..ece11dd9b --- /dev/null +++ b/tools/runner-image/biome.json @@ -0,0 +1,4 @@ +{ + "extends": "//", + "linter": { "rules": { "suspicious": { "noConsole": "off" } } } +} diff --git a/tools/runner-image/build-core.test.ts b/tools/runner-image/build-core.test.ts new file mode 100644 index 000000000..32fe265d0 --- /dev/null +++ b/tools/runner-image/build-core.test.ts @@ -0,0 +1,164 @@ +// Unit tests for the runner-image build lane's pure core. Each case pins a +// behaviour the image's correctness depends on, and each would fail on a +// plausible bug — not on a restatement of the implementation. + +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { + buildArgs, + buildctlArgs, + closureRoots, + kernelImagePath, + outputSpec, + parseOutPaths, + type RunnerImageOutputs, +} from "./build-core.ts"; + +const outputs: RunnerImageOutputs = { + runner: "/nix/store/aaa-compass-runner", + stack: "/nix/store/bbb-compass-stack-env", + kernelDir: "/nix/store/ccc-linux", + rootfs: "/nix/store/ddd-rootfs.erofs", + initrd: "/nix/store/eee-initrd", +}; + +describe("parseOutPaths", () => { + test("drops the blank trailing line nix emits", () => { + expect(parseOutPaths("/nix/store/a\n/nix/store/b\n")).toEqual([ + "/nix/store/a", + "/nix/store/b", + ]); + }); + + test("yields nothing for empty stdout, so a silent build failure cannot read as one path", () => { + expect(parseOutPaths("")).toEqual([]); + expect(parseOutPaths("\n \n")).toEqual([]); + }); +}); + +describe("kernelImagePath", () => { + // The kernel derivation is a directory (bzImage + System.map); the rootfs and + // initrd derivations ARE files. Booting the directory would fail at the VMM. + test("points at the bzImage inside the kernel derivation directory", () => { + expect(kernelImagePath("/nix/store/ccc-linux")).toBe( + "/nix/store/ccc-linux/bzImage", + ); + }); +}); + +describe("buildArgs", () => { + test("maps each artifact to the env the Runner reads, kernel suffixed and the file assets bare", () => { + expect(buildArgs(outputs)).toEqual({ + VMM_BIN: "/nix/store/bbb-compass-stack-env/bin/cloud-hypervisor", + VIRTIOFSD_BIN: "/nix/store/bbb-compass-stack-env/bin/virtiofsd", + STACK_BIN_DIR: "/nix/store/bbb-compass-stack-env/bin", + GUEST_KERNEL: "/nix/store/ccc-linux/bzImage", + GUEST_ROOTFS: "/nix/store/ddd-rootfs.erofs", + GUEST_INITRD: "/nix/store/eee-initrd", + }); + }); +}); + +describe("closureRoots", () => { + // The roots are derivation out-paths: `nix path-info -r` takes store paths, + // and passing the bzImage FILE instead of its directory would stage a partial + // closure whose binaries fail to exec. + test("uses the kernel DIRECTORY, not the bzImage inside it", () => { + const roots = closureRoots(outputs); + expect(roots).toContain("/nix/store/ccc-linux"); + expect(roots).not.toContain("/nix/store/ccc-linux/bzImage"); + }); + + test("covers all five artifacts, so no carried binary is staged without its dependencies", () => { + expect(closureRoots(outputs).sort()).toEqual( + [ + outputs.runner, + outputs.stack, + outputs.kernelDir, + outputs.rootfs, + outputs.initrd, + ].sort(), + ); + }); +}); + +describe("outputSpec", () => { + test("oci writes a browsable layout, which is what the publish lane scans before pushing", () => { + expect(outputSpec("oci", "ignored:tag", "/tmp/out")).toBe( + "type=oci,dest=/tmp/out,tar=false,rewrite-timestamp=true", + ); + }); + + test("image names the tag for a local load", () => { + expect(outputSpec("image", "compass-runner:dev", "/tmp/out")).toBe( + "type=image,name=compass-runner:dev,rewrite-timestamp=true", + ); + }); + + // The digest-stability property the publish lane depends on: without this, + // two builds of a bit-identical staged tree still export different layer + // digests, because the context's mtimes ride into the layer tar. + test("both modes rewrite layer timestamps, so a rebuild is digest-stable", () => { + expect(outputSpec("oci", "t", "/tmp/out")).toContain( + "rewrite-timestamp=true", + ); + expect(outputSpec("image", "t", "/tmp/out")).toContain( + "rewrite-timestamp=true", + ); + }); + + test("oci mode never names the tag, so a dev tag cannot leak into a layout build", () => { + expect(outputSpec("oci", "compass-runner:dev", "/tmp/out")).not.toContain( + "compass-runner:dev", + ); + }); +}); + +describe("buildctlArgs", () => { + test("passes every build-arg the Dockerfile declares", () => { + const args = buildctlArgs( + "/repo/runner-image", + outputs, + "linux/amd64", + "type=oci,dest=/repo/runner-image/out,tar=false", + ); + const joined = args.join(" "); + for (const name of Object.keys(buildArgs(outputs))) { + expect(joined).toContain(`build-arg:${name}=`); + } + }); + + test("build-arg order is deterministic, so two identical builds produce identical argv", () => { + const once = buildctlArgs("/ctx", outputs, "linux/amd64", "type=oci"); + const twice = buildctlArgs("/ctx", outputs, "linux/amd64", "type=oci"); + expect(once).toEqual(twice); + }); + + // The load-bearing drift test: buildArgs is only correct RELATIVE to the + // Dockerfile's own ARG declarations, and nothing else compares the two. It is + // what caught a RUNNER_BIN that build-core supplied and the Dockerfile had + // stopped consuming. + test("supplies exactly the build-args the Dockerfile declares", () => { + const dockerfile = readFileSync( + join(import.meta.dir, "..", "..", "runner-image", "Dockerfile"), + "utf8", + ); + const declared = new Set( + [...dockerfile.matchAll(/^ARG\s+([A-Z_][A-Z0-9_]*)/gm)].map( + (m) => m[1] as string, + ), + ); + // SOURCE_DATE_EPOCH is consumed by the frontend itself and supplied by + // buildctlArgs directly, not through the artifact mapping. + declared.delete("SOURCE_DATE_EPOCH"); + expect(new Set(Object.keys(buildArgs(outputs)))).toEqual(declared); + }); + + test("uses the dockerfile.v0 frontend against the staged context", () => { + const args = buildctlArgs("/ctx", outputs, "linux/amd64", "type=oci"); + expect(args.slice(0, 3)).toEqual(["build", "--frontend", "dockerfile.v0"]); + expect(args).toContain("context=/ctx"); + expect(args).toContain("dockerfile=/ctx"); + }); +}); diff --git a/tools/runner-image/build-core.ts b/tools/runner-image/build-core.ts new file mode 100644 index 000000000..67fd4f838 --- /dev/null +++ b/tools/runner-image/build-core.ts @@ -0,0 +1,156 @@ +// The pure core of the runner-image build lane (R1). Every function here is a +// total map over its inputs with no I/O, so build-core.test.ts can drive each +// mapping — and each fail-closed edge — without nix, buildkit, or a subprocess. +// +// The lane's job is to turn realised nix store paths into the two things the +// container build needs: the set of paths to stage into the build context, and +// the build-args that bake those absolute paths into the image's env. + +/** The fixed build epoch. Nix normalises every store mtime to 1, so using the + * same value keeps the rewritten layer timestamps equal to what the staged tree + * already carries. */ +export const SOURCE_DATE_EPOCH = 1; + +/** The six artifacts the image carries, as realised store paths. */ +export interface RunnerImageOutputs { + /** The `compass-runner` package out-path (the binary is at `bin/compass-runner`). */ + runner: string; + /** The `compass-stack-env` symlinkJoin (cloud-hypervisor, virtiofsd, passt under `bin/`). */ + stack: string; + /** The guest kernel DERIVATION dir; the bootable artifact is `/bzImage`. */ + kernelDir: string; + /** The guest rootfs image — the derivation IS the file. */ + rootfs: string; + /** The guest initramfs — the derivation IS the file. */ + initrd: string; +} + +/** Split `nix build --print-out-paths` stdout into trimmed, non-empty store + * paths, one per line. Mirrors the microvm-boot-test lane's parser: the same + * command shape deserves the same reader, not a second convention. */ +export function parseOutPaths(stdout: string): string[] { + return stdout + .split("\n") + .map((line) => line.trim()) + .filter((line) => line !== ""); +} + +/** + * The absolute in-image path of the bootable kernel. The kernel derivation is a + * DIRECTORY (it also carries System.map); cloud-hypervisor direct-boots the + * bzImage inside it, with no bootloader. The rootfs and initrd derivations are + * files, so only this one gains a suffix — the asymmetry is the derivations', + * not a convention. + */ +export function kernelImagePath(kernelDir: string): string { + return `${kernelDir}/bzImage`; +} + +/** + * The `--opt build-arg:` values the Dockerfile bakes into ENV. Every value is an + * absolute /nix/store path resolved at build time, because each carries a + * content hash that moves whenever its input is rebuilt — hardcoding any of them + * in the Dockerfile would pin a stale artifact that the staged closure no longer + * contains. + * + * These land on the documented `COMPASS_MICROVM_*` env fallbacks for the + * matching `--microvm-*` flags, so an operator can still override any one at + * runtime without rebuilding. + */ +export function buildArgs(outputs: RunnerImageOutputs): Record { + return { + VMM_BIN: `${outputs.stack}/bin/cloud-hypervisor`, + VIRTIOFSD_BIN: `${outputs.stack}/bin/virtiofsd`, + // passt is exec'd by NAME rather than by a configured path (it has no + // --microvm-* flag), so the image needs its bin dir on PATH. + STACK_BIN_DIR: `${outputs.stack}/bin`, + GUEST_KERNEL: kernelImagePath(outputs.kernelDir), + GUEST_ROOTFS: outputs.rootfs, + GUEST_INITRD: outputs.initrd, + }; +} + +/** + * The closure ROOTS whose transitive dependencies must be staged into the build + * context. + * + * The roots are the derivation out-paths, NOT the inner file paths: `nix + * path-info -r` takes store paths, and the kernel's root is its directory even + * though the image references the bzImage inside it. + * + * Staging the transitive closure — rather than just these five — is + * load-bearing. Each carried binary names an absolute /nix/store interpreter and + * resolves every NEEDED library through its own RPATH, so a context missing the + * transitive glibc produces an image whose binaries fail to exec with "missing + * dynamic library". That is the measured negative control behind the base-image + * choice, not a hypothetical. + */ +export function closureRoots(outputs: RunnerImageOutputs): string[] { + return [ + outputs.runner, + outputs.stack, + outputs.kernelDir, + outputs.rootfs, + outputs.initrd, + ]; +} + +/** The buildctl `--output` spec for each supported output mode. `oci` writes a + * browsable local layout (what the publish lane scans BEFORE deciding to push); + * `image` names a tagged image for a local dogfood load. */ +export function outputSpec( + mode: "oci" | "image", + tag: string, + ociDir: string, +): string { + // rewrite-timestamp normalises every layer entry's mtime to SOURCE_DATE_EPOCH. + // Without it two builds of a BIT-IDENTICAL staged tree still produce different + // layer digests, because BuildKit carries the context's mtimes into the layer + // tar. Measured: only the `COPY store` layer differed between runs; with this + // set, two builds yield the same manifest digest. + const rewrite = "rewrite-timestamp=true"; + return mode === "oci" + ? `type=oci,dest=${ociDir},tar=false,${rewrite}` + : `type=image,name=${tag},${rewrite}`; +} + +/** + * The full `buildctl` argv (after the binary). Rootless BuildKit is the ruled + * mechanism for a prebuilt-application image; this lane never shells out to + * `docker build` and never mounts a host docker socket, which would hand the + * build the daemon's blast radius. + */ +export function buildctlArgs( + contextDir: string, + outputs: RunnerImageOutputs, + platform: string, + output: string, +): string[] { + const args = [ + "build", + "--frontend", + "dockerfile.v0", + "--local", + `context=${contextDir}`, + "--local", + `dockerfile=${contextDir}`, + "--opt", + "filename=Dockerfile", + "--opt", + `platform=${platform}`, + // Pairs with rewrite-timestamp: this fixes the image config's `created` + // field and is the epoch every layer mtime is rewritten to. 1, not 0, + // matching the mtime nix normalises its store paths to. + "--opt", + `build-arg:SOURCE_DATE_EPOCH=${SOURCE_DATE_EPOCH}`, + ]; + // Sorted, so the argv is deterministic across runs: an unstable arg order + // would make two otherwise-identical builds diff in logs for no reason. + for (const [name, value] of Object.entries(buildArgs(outputs)).sort( + ([a], [b]) => a.localeCompare(b), + )) { + args.push("--opt", `build-arg:${name}=${value}`); + } + args.push("--output", output); + return args; +} diff --git a/tools/runner-image/build.ts b/tools/runner-image/build.ts new file mode 100755 index 000000000..658e5edd3 --- /dev/null +++ b/tools/runner-image/build.ts @@ -0,0 +1,268 @@ +#!/usr/bin/env bun +// Build the compass-runner container image (R1 of the Compass Runner +// containerization record): realise the nix closure, stage it into a build +// context, and hand that plain directory to a rootless BuildKit Dockerfile +// build. +// +// THE MECHANISM IS THE RULED ONE, and it is part of the reviewed surface rather +// than an implementation detail. The fleet's first-party image-builds spec +// splits mechanisms by what the image's RUNTIME is, not by what built the +// artifact: an image that IS a Nix environment earns nix2container; a PREBUILT +// APPLICATION on a minimal base is a Dockerfile built by rootless BuildKit. The +// Runner is the latter — it runs no toolchain, it exec's four binaries — so nix +// is the BUILDER here and never the runtime. The prescribed shape is `nix +// build` then COPY, which is exactly the two halves below. +// +// TypeScript, not bash: this has real logic — four nix builds whose out-paths +// are parsed and mapped to distinct build-args, a transitive closure staged +// path-by-path, and a fail-fast on every missing output — so the parsing and +// mapping core is pure and unit-tested (./build-core.test.ts) while this file is +// the thin I/O shell. +// +// Usage: +// bun tools/runner-image/build.ts [--tag ] [--output oci|image] + +import { spawnSync } from "node:child_process"; +import { + chmodSync, + cpSync, + existsSync, + mkdirSync, + readdirSync, + rmSync, + symlinkSync, +} from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { + buildctlArgs, + closureRoots, + outputSpec, + parseOutPaths, + type RunnerImageOutputs, +} from "./build-core.ts"; + +// This file is tools/runner-image/build.ts, so `../..` is the workspace root. +const workspaceRoot = join(dirname(fileURLToPath(import.meta.url)), "..", ".."); +const imageDir = join(workspaceRoot, "runner-image"); +const stageDir = join(imageDir, "store"); +const ociDir = join(imageDir, "out"); + +// Single-arch linux/amd64: the Runner's microVM backend needs KVM on the node, +// and the cluster nodes are amd64. A second leg would need its own KVM-capable +// builder, so it is added when a node arch is, not speculatively. +const IMAGE_PLATFORM = "linux/amd64"; + +function parseArgs(argv: readonly string[]): { + tag: string; + mode: "oci" | "image"; +} { + let tag = "compass-runner:dev"; + let mode: "oci" | "image" = "oci"; + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]; + const value = argv[i + 1]; + if (arg === "--tag" && value !== undefined) { + tag = value; + i += 1; + } else if (arg === "--output" && value !== undefined) { + if (value !== "oci" && value !== "image") { + console.error(`--output must be 'oci' or 'image', got: ${value}`); + process.exit(2); + } + mode = value; + i += 1; + } else { + console.error(`unknown argument: ${arg}`); + process.exit(2); + } + } + return { tag, mode }; +} + +/** Run a nix build and return its realised out-paths in argument order. Exits on + * failure or on an out-path count that does not match what was asked for, so a + * drifted attr set is a named, fail-closed abort rather than a silently + * incomplete image. */ +function nixBuild( + args: readonly string[], + expected: number, + cwd: string, +): string[] { + const result = spawnSync( + "nix", + ["build", "--no-link", "--print-out-paths", ...args], + { + cwd, + encoding: "utf8", + stdio: ["ignore", "pipe", "inherit"], + }, + ); + if (result.status !== 0) { + console.error(`nix build ${args.join(" ")} failed (exit ${result.status})`); + process.exit(1); + } + const paths = parseOutPaths(result.stdout); + if (paths.length !== expected) { + console.error( + `nix build ${args.join(" ")} produced ${paths.length} out-paths, expected ${expected}`, + ); + process.exit(1); + } + return paths; +} + +/** Restore owner-write on a staged tree so it can be removed. Copying from the + * nix store preserves its read-only directory modes, which would otherwise make + * the stage dir undeletable on the next run. A no-op when the path is absent. */ +function makeWritable(dir: string): void { + if (!existsSync(dir)) return; + chmodSync(dir, 0o755); + for (const entry of readdirSync(dir, { withFileTypes: true })) { + // readdirSync withFileTypes uses lstat semantics, so a symlink already + // reports isDirectory() === false and is never followed here. + if (entry.isDirectory()) { + makeWritable(join(dir, entry.name)); + } + } +} + +const { tag, mode } = parseArgs(process.argv.slice(2)); + +// The platform below is a manifest LABEL; BuildKit applies it without checking +// what the COPY'd files actually are. Today only flake.nix's systems list keeps +// the two honest — a separate file this lane never reads — so assert it here +// rather than inherit an unchecked invariant. +if (process.arch !== "x64") { + console.error( + `runner-image targets ${IMAGE_PLATFORM}, but this host is ${process.arch}.\n` + + " Building here would label the image amd64 while staging this host's binaries.", + ); + process.exit(1); +} + +// buildctl is a CLIENT; it needs a reachable buildkitd. This only checks the +// var is SET, so an unreachable daemon still fails at the build step after the +// realise — it catches the common "forgot to start one" case early, not a dead +// socket. There is deliberately no `docker build` fallback. +const buildkitHost = process.env.BUILDKIT_HOST; +if (buildkitHost === undefined || buildkitHost === "") { + console.error( + "BUILDKIT_HOST is unset — start a buildkitd and point it here.\n" + + " This lane never falls back to `docker build`: rootless BuildKit is the\n" + + " ruled mechanism for a prebuilt-application image, not a preference.", + ); + process.exit(1); +} + +// --------------------------------------------------------------------------- +// 1. Realise every carried artifact. +// --------------------------------------------------------------------------- +// The Runner and the VMM env are flake outputs; the three guest assets come from +// guest-image/default.nix, which is a bare nix file and NOT a flake, so it takes +// the `-f` form (no `#attr` flake-ref exists for it) and runs from guest-image/. +console.error("runner-image: realising closure…"); +const [runner] = nixBuild([".#compass-runner"], 1, workspaceRoot) as [string]; +const [stack] = nixBuild([".#compass-stack-env"], 1, workspaceRoot) as [string]; +const guest = nixBuild( + [ + "-f", + "default.nix", + "compass-guest-kernel", + "compass-guest-rootfs", + "compass-guest-initrd", + ], + 3, + join(workspaceRoot, "guest-image"), +); +const [kernelDir, rootfs, initrd] = guest as [string, string, string]; +const outputs: RunnerImageOutputs = { + runner, + stack, + kernelDir, + rootfs, + initrd, +}; + +// --------------------------------------------------------------------------- +// 2. Stage the transitive closure. +// --------------------------------------------------------------------------- +// `nix path-info -r` over all five roots emits the DEDUPED union, so a path +// shared by several roots (glibc, most obviously) is staged once. The transitive +// set — not just the five roots — is what makes the image runnable: see +// closureRoots' note on the measured "missing dynamic library" control. +console.error("runner-image: staging closure…"); +// Nix store paths are read-only, and a recursive copy preserves that — so a +// previously staged tree cannot be removed until its directories are made +// writable again. Without this, the SECOND run of this lane fails EACCES on its +// own leftovers. +makeWritable(stageDir); +rmSync(stageDir, { recursive: true, force: true }); +rmSync(ociDir, { recursive: true, force: true }); +mkdirSync(stageDir, { recursive: true }); + +const closure = spawnSync( + "nix", + ["path-info", "-r", ...closureRoots(outputs)], + { + cwd: workspaceRoot, + encoding: "utf8", + stdio: ["ignore", "pipe", "inherit"], + }, +); +if (closure.status !== 0) { + console.error(`nix path-info -r failed (exit ${closure.status})`); + process.exit(1); +} +const closurePaths = parseOutPaths(closure.stdout); +for (const path of closurePaths) { + // Each basename is unique by construction (it carries a content hash), and + // the store's symlinks are preserved verbatim so the symlinkJoin'd stack env + // still resolves inside the image. + // + // preserveTimestamps is load-bearing for the DIGEST, not just tidiness. Nix + // normalises every store mtime to 1; without this, cpSync stamps wall-clock + // mtimes into the staged tree, they land in the layer tar, and two builds of + // identical inputs produce different digests. The publish lane asserts a + // stable digest, so this is part of what makes that assertion meaningful. + cpSync(path, join(stageDir, path.replace(/^\/nix\/store\//, "")), { + recursive: true, + verbatimSymlinks: true, + preserveTimestamps: true, + }); +} +console.error(`runner-image: staged ${closurePaths.length} store paths`); + +// The Dockerfile's ENTRYPOINT is exec-form and so cannot expand a build ARG, +// while the Runner's own store path carries a hash that moves on every Go +// rebuild. A stable relative symlink INSIDE the staged tree resolves both: the +// entrypoint is always /nix/store/.compass-runner, and what it points at moves +// with the build. +// +// It must live inside store/ and be RELATIVE. An absolute symlink at the +// context root dangles on the host (nothing is mounted at the target yet), and +// BuildKit checksums context entries before any COPY runs, so it fails the +// build with "not found" rather than deferring to runtime. Relative-and-inside +// keeps the link resolvable in both places. +rmSync(join(stageDir, ".compass-runner"), { force: true }); +symlinkSync( + join(runner.replace(/^\/nix\/store\//, ""), "bin", "compass-runner"), + join(stageDir, ".compass-runner"), +); + +// --------------------------------------------------------------------------- +// 3. Build the image from the staged directory. +// --------------------------------------------------------------------------- +mkdirSync(ociDir, { recursive: true }); +console.error(`runner-image: building ${tag}…`); +const build = spawnSync( + "buildctl", + buildctlArgs( + imageDir, + outputs, + IMAGE_PLATFORM, + outputSpec(mode, tag, ociDir), + ), + { cwd: workspaceRoot, stdio: "inherit" }, +); +process.exit(build.status ?? 1); diff --git a/tools/runner-image/moon.yml b/tools/runner-image/moon.yml new file mode 100644 index 000000000..8e6826d14 --- /dev/null +++ b/tools/runner-image/moon.yml @@ -0,0 +1,31 @@ +# yaml-language-server: $schema=https://moonrepo.dev/schemas/project.json +# +# runner-image (R1): the build lane for the compass-runner container image. +# Realises the nix closure, stages it, and drives the rootless-BuildKit build. +# TypeScript rather than a shell script (the no-bash-gate CI task); +# runner-image/ carries the Dockerfile and the staged context. +# +# The BUILD itself needs a reachable buildkitd and realises a multi-GB nix +# closure, so it is not a CI gate task (see runner-image/moon.yml for the build +# task and why it is runInCI:false). The typecheck/test tasks here ARE ordinary +# bun gates that ride CI: they cover the pure core — the out-path parsing, the +# build-arg mapping, and the closure-root selection whose drift would silently +# produce an unrunnable image. +# +# A bun/TypeScript CLI, hoisted root-workspace member (`bun` tag): install is +# inherited via .moon/tasks/tag-bun.yml and lint/format are whole-repo root +# tasks, so this leaf carries no own bun.lock. +layer: 'tool' +language: 'typescript' +tags: ['bun', 'ci-group.bun'] + +tasks: + typecheck: + command: 'bunx tsc --noEmit' + deps: ['install'] + inputs: ['*.ts', 'tsconfig.json', '/tsconfig.base.json', 'package.json', '/bun.lock'] + test: + # The pure core (build-core.ts) over fixtures: the out-path parsing, the + # artifact→build-arg mapping (incl. the kernel's bzImage suffix, which the + # file-shaped rootfs/initrd must NOT gain), and the closure-root selection. + inputs: ['*.ts', 'tsconfig.json', '/tsconfig.base.json', 'package.json', '/bun.lock'] diff --git a/tools/runner-image/package.json b/tools/runner-image/package.json new file mode 100644 index 000000000..33cc0cc07 --- /dev/null +++ b/tools/runner-image/package.json @@ -0,0 +1,14 @@ +{ + "name": "@compass/runner-image", + "private": true, + "type": "module", + "description": "Build the compass-runner container image (R1): realise the nix closure, stage it into a build context, and build a Dockerfile on a minimal hardened base with rootless BuildKit. Nix is the builder, never the runtime.", + "module": "build.ts", + "bin": { + "runner-image-build": "./build.ts" + }, + "devDependencies": { + "@types/bun": "catalog:", + "typescript": "catalog:" + } +} diff --git a/tools/runner-image/tsconfig.json b/tools/runner-image/tsconfig.json new file mode 100644 index 000000000..d40cc9e50 --- /dev/null +++ b/tools/runner-image/tsconfig.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.base.json", + "compilerOptions": { + "lib": ["ES2022"], + "moduleDetection": "force", + "allowJs": true, + "allowImportingTsExtensions": true, + "noUncheckedIndexedAccess": true, + "types": ["bun"] + } +} diff --git a/version.txt b/version.txt index 0ea3a944b..0c62199f1 100644 --- a/version.txt +++ b/version.txt @@ -1 +1 @@ -0.2.0 +0.2.1