Real-node integration suite: the wallet engine against a spawned rand-node (CI job real-node on fullnode v0.7.1)

.github/workflows/node-integration.yml

@@ -0,0 +1,79 @@
+# The wallet against a real node: the wallet's real transport, reply validators and flows (the wasm
+# core and engine under Node, real `fetch`) against a real `rand-node` from the fullnode v0.7.1
+# release, on a Test-profile chain the test cuts itself. See scripts/e2e-node.sh for what it covers
+# and web/wallet/test/node.e2e.test.mjs for the cases.
+#
+# The node binaries are the release's, not built here: that is what makes this a check of the
+# wallet against the node that is deployed, and it keeps the job to the wasm core's build plus about
+# a minute of tests with `rand-prover`'s one real bundle proof on top (a few minutes on a shared
+# runner). To test against another release set FULLNODE_TAG; nothing the suite asserts may depend on
+# a node feature newer than v0.7.1.
+name: node-integration
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: node-integration-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  CARGO_TERM_COLOR: always
+  RUST_TOOLCHAIN: 1.98.1
+  FULLNODE_TAG: v0.7.1
+
+jobs:
+  real-node:
+    name: real-node
+    runs-on: ubuntu-24.04
+    timeout-minutes: 60
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: recursive
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+      - name: Rust ${{ env.RUST_TOOLCHAIN }}
+        run: |
+          rustup toolchain install "$RUST_TOOLCHAIN" --profile minimal --target wasm32-unknown-unknown
+          rustup default "$RUST_TOOLCHAIN"
+      - name: Cache cargo registry and wasm-bindgen
+        uses: actions/cache@v4
+        with:
+          path: |
+            ~/.cargo/registry/index
+            ~/.cargo/registry/cache
+            ~/.cargo/bin/wasm-bindgen*
+          key: cargo-wasm-${{ env.RUST_TOOLCHAIN }}-${{ hashFiles('core/Cargo.lock') }}
+      - name: Cache the wasm core's build
+        uses: actions/cache@v4
+        with:
+          path: core/target
+          key: core-target-${{ env.RUST_TOOLCHAIN }}-${{ hashFiles('core/Cargo.lock') }}
+          restore-keys: core-target-${{ env.RUST_TOOLCHAIN }}-
+      - name: The wasm core
+        run: core/scripts/build-wasm.sh
+      - name: Download the fullnode ${{ env.FULLNODE_TAG }} release
+        env:
+          GH_TOKEN: ${{ github.token }}
+        run: |
+          mkdir -p "$RUNNER_TEMP/fullnode"
+          gh release download "$FULLNODE_TAG" --repo randprotocol/fullnode \
+            --pattern rand-node --pattern rand-prover --pattern SHA256SUMS \
+            --dir "$RUNNER_TEMP/fullnode"
+          (cd "$RUNNER_TEMP/fullnode" && sha256sum -c --ignore-missing SHA256SUMS)
+          chmod +x "$RUNNER_TEMP/fullnode/rand-node" "$RUNNER_TEMP/fullnode/rand-prover"
+          "$RUNNER_TEMP/fullnode/rand-node" --version
+          {
+            echo "RAND_NODE_BIN=$RUNNER_TEMP/fullnode/rand-node"
+            echo "RAND_PROVER_BIN=$RUNNER_TEMP/fullnode/rand-prover"
+          } >> "$GITHUB_ENV"
+      - name: The wallet against the real node
+        run: scripts/e2e-node.sh

extension/README.md

@@ -144,3 +144,10 @@ RAND_BRIDGE_URL=http://localhost:3000 node --test extension/test/bridge-chrome.e
 The withdrawal itself — a burn proved by a paired `rand-prover`, against a real node with a
 bridged genesis — is `web/wallet/test/withdraw.e2e.test.mjs`, run by `scripts/e2e-withdraw.sh`
 (the transfer's counterpart is `scripts/e2e-prover.sh`).
+
+The wallet's whole client side of the node's JSON-RPC contract is tested against a real node by
+`web/wallet/test/node.e2e.test.mjs`, run by `scripts/e2e-node.sh` (set `RAND_NODE_BIN`, and
+`RAND_PROVER_BIN` for the spend round trip; its header lists what it covers). Unlike the two above
+it runs in CI, as the `real-node` job of `.github/workflows/node-integration.yml`, against the
+fullnode v0.7.1 release binaries; the spawn, polling and genesis helpers it shares with
+`prover.e2e.test.mjs` live in `web/wallet/test/node-harness.mjs`.

scripts/e2e-node.sh

@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# The wallet-against-a-real-node end-to-end suite: the wallet's real transport, reply validators
+# and flows (the wasm core and engine under Node, real `fetch`) against one real one-validator
+# `rand-node` on a Test-profile chain. It covers chain identity, the whole client-side RPC contract
+# (every method in ui/lib/rpc-methods.js, empty and populated), the faucet → scan → balance →
+# activity → rescan flow, failover and submit rules, and a node restart; with a `rand-prover` it
+# also sends from one wallet to another through one real bundle proof (a couple of minutes on a
+# laptop — do not run it beside another proving job). Everything else takes about a minute.
+#
+#   RAND_NODE_BIN=… scripts/e2e-node.sh                       # all but the spend round trip
+#   RAND_NODE_BIN=… RAND_PROVER_BIN=… scripts/e2e-node.sh     # and the spend round trip
+#
+#   RAND_NODE_BIN    a `rand-node`: a fullnode release asset (CI downloads v0.7.1) or a build.
+#                    Required — without it the suite would only report itself skipped.
+#   RAND_PROVER_BIN  a `rand-prover` from the same release; without it the spend case is skipped.
+#
+# Nothing is built here except the wasm core (extension/shared/core/rand_wallet_bg.wasm), when
+# missing — as chrome/pack.sh does. The node binaries are never built by this script: take a
+# release (`gh release download v0.7.1 -R randprotocol/fullnode`) or see scripts/e2e-prover.sh for
+# building them from the vendored fullnode. The suite must hold on v0.7.1, so it uses no node
+# feature newer than that.
+set -euo pipefail
+cd "$(dirname "$0")/.."
+
+[ -f extension/shared/core/rand_wallet_bg.wasm ] || core/scripts/build-wasm.sh
+
+[ -n "${RAND_NODE_BIN:-}" ] || { echo "RAND_NODE_BIN is not set: point it at a rand-node binary (see the header)" >&2; exit 1; }
+for var in RAND_NODE_BIN RAND_PROVER_BIN; do
+  bin="${!var:-}"
+  [ -z "$bin" ] || [ -x "$bin" ] || { echo "$var is not an executable: $bin" >&2; exit 1; }
+done
+echo "rand-node:   $RAND_NODE_BIN"
+echo "rand-prover: ${RAND_PROVER_BIN:-(not set: the spend round trip is skipped)}"
+exec node --test web/wallet/test/node.e2e.test.mjs

ui/engine/wallet.js

@@ -1576,7 +1576,7 @@ export function makeWallet({ core, store, rpc, settings, annotate = true, onRese
   }
 
   return {
-    scan, rescan, send, burn, completeSend, completeBurn, quoteInvoke, invoke, completeInvoke, faucet, loadStore, persist, core: c, rpcFor, activity, balanceOf, isSpendable,
+    scan, rescan, send, burn, completeSend, completeBurn, quoteInvoke, invoke, completeInvoke, faucet, loadStore, persist, core: c, rpcFor, activity, balanceOf, isSpendable, anchorAndWitnesses,
     // Exposed so a backend can prove the chain WITHOUT scanning — the gate in front of send and
     // faucet is two RPC calls, not a page of leaves.
     chainIdentity, chainVerdict,

web/wallet/test/node-harness.mjs

@@ -0,0 +1,185 @@
+// Shared pieces of the real-node end-to-end tests (prover.e2e.test.mjs, node.e2e.test.mjs): the
+// wasm core under Node, an in-memory storage, a fetch that remembers what it was asked, a free
+// loopback port, a bounded `until`, child-process handling and the genesis flags a given
+// `rand-node` understands. Nothing here is a test; importing it starts nothing.
+import { spawn, execFileSync } from 'node:child_process';
+import { readFileSync } from 'node:fs';
+import { createServer } from 'node:net';
+import { makeWasmBackend } from '../../../ui/engine/backend-wasm.js';
+
+// A release `rand-node` (fullnode v0.6.8 and later, #94) refuses a Test-profile genesis at `init`
+// and `run` unless told it is a test harness; child processes inherit this. Older nodes ignore it.
+process.env.RAND_ALLOW_TEST_FRI_PROFILE ??= '1';
+
+export const CORE_JS = new URL('../../../extension/shared/core/rand_wallet.js', import.meta.url);
+export const CORE_WASM = new URL('../../../extension/shared/core/rand_wallet_bg.wasm', import.meta.url);
+
+/** The wasm core, initialised from disk, behind the `call(method, params)` every binding has. */
+export async function realCore() {
+  const mod = await import(CORE_JS.href);
+  mod.initSync({ module: readFileSync(CORE_WASM) });
+  return {
+    async call(method, params = {}) {
+      const reply = JSON.parse(mod.call(method, JSON.stringify(params)));
+      if (!reply.ok) throw new Error(reply.error);
+      return reply.value;
+    },
+  };
+}
+
+export function mapStorage() {
+  const local = new Map();
+  const session = new Map();
+  const clone = (v) => (v === undefined ? undefined : JSON.parse(JSON.stringify(v)));
+  return {
+    local,
+    sessionMap: session,
+    async get(k) { return clone(local.get(k)); },
+    async set(k, v) { local.set(k, clone(v)); },
+    async remove(k) { local.delete(k); },
+    async clear() { local.clear(); session.clear(); },
+    session: {
+      async get(k) { return session.get(k); },
+      async set(k, v) { session.set(k, v); },
+      async remove(k) { session.delete(k); },
+    },
+  };
+}
+
+/** Real `fetch`, with every request's URL and body kept (for the key-leak scan, for replays). */
+export function recordingFetch() {
+  const requests = [];
+  const wrapped = async (url, init = {}) => {
+    requests.push({ url: String(url), body: typeof init.body === 'string' ? init.body : JSON.stringify(init.body ?? null) });
+    return fetch(url, init);
+  };
+  wrapped.requests = requests;
+  return wrapped;
+}
+
+/** A wasm backend over the real core and real `fetch`, as a user's wallet is built. */
+export async function wallet(core) {
+  const storage = mapStorage();
+  const fetchImpl = recordingFetch();
+  const backend = makeWasmBackend({
+    core, storage, fetch: fetchImpl,
+    platform: { name: 'node-e2e', openExternal() {}, copy() {} },
+    locks: null, broadcast: null,
+  });
+  return { backend, storage, fetch: fetchImpl };
+}
+
+export function freePort() {
+  return new Promise((resolve, reject) => {
+    const srv = createServer();
+    srv.unref();
+    srv.on('error', reject);
+    srv.listen(0, '127.0.0.1', () => {
+      const { port } = srv.address();
+      srv.close(() => resolve(port));
+    });
+  });
+}
+
+/** One bare JSON-RPC call, outside the wallet's own client. */
+export async function rpc(url, method, params = []) {
+  const res = await fetch(url, {
+    method: 'POST',
+    headers: { 'content-type': 'application/json' },
+    body: JSON.stringify({ jsonrpc: '2.0', id: 1, method, params }),
+  });
+  const reply = await res.json();
+  if (reply.error) throw Object.assign(new Error(reply.error.message), { code: reply.error.code });
+  return reply.result;
+}
+
+export const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+
+/** Polls `check` until it returns something truthy, or throws after `timeoutMs`. */
+export async function until(what, check, { timeoutMs, everyMs = 1000 }) {
+  const deadline = Date.now() + timeoutMs;
+  let last;
+  while (Date.now() < deadline) {
+    try {
+      const value = await check();
+      if (value) return value;
+    } catch (err) {
+      last = err;
+    }
+    await sleep(everyMs);
+  }
+  throw new Error(`timed out waiting for ${what}${last ? ` (last error: ${last.message})` : ''}`);
+}
+
+/** A child process whose output is kept (and printed if the test fails), killed in `after`. */
+export function launch(bin, args, label, options = {}) {
+  const child = spawn(bin, args, { stdio: ['ignore', 'pipe', 'pipe'], ...options });
+  child.log = [];
+  const keep = (chunk) => {
+    child.log.push(chunk.toString());
+    if (child.log.length > 400) child.log.splice(0, child.log.length - 400);
+  };
+  child.stdout.on('data', keep);
+  child.stderr.on('data', keep);
+  child.label = label;
+  return child;
+}
+
+export function stop(child) {
+  if (child && child.exitCode === null && child.signalCode === null) child.kill('SIGTERM');
+}
+
+export const running = (child) => !!child && child.exitCode === null && child.signalCode === null;
+
+/** Resolves when `child` has exited, or after `ms` — whichever is first; `true` if it exited. */
+export function exited(child, ms) {
+  if (!running(child)) return Promise.resolve(true);
+  return new Promise((resolve) => {
+    const timer = setTimeout(() => { child.off('exit', done); resolve(false); }, ms);
+    function done() { clearTimeout(timer); resolve(true); }
+    child.once('exit', done);
+  });
+}
+
+/** SIGTERM, wait up to `ms` for the exit, then SIGKILL and wait for that exit too. */
+export async function stopAndWait(child, ms = 10_000) {
+  if (!running(child)) return;
+  stop(child);
+  if (await exited(child, ms)) return;
+  child.kill('SIGKILL');
+  await exited(child, 5_000);
+}
+
+/**
+ * What chain 18's genesis has beyond the defaults (fullnode v0.6.7): the `gas` section — the
+ * prices, the bundle guest's pinned gas (20 479, the one value genesis accepts) and the in-circuit
+ * meter (constraint set 8, spec 2026-09-28 §4.2) — and **split authorisation** (v0.6.3): bundle
+ * guest v3 with the auth guest, which needs a block cap of at least three proof caps plus a MiB
+ * (chain 18's 4 MiB proofs and 20 MiB blocks). So the node holds every transaction this wallet
+ * makes to the live rules: the bundle proof to the gas pin, and an auth proof beside it made by
+ * the wallet from its spend key. Only when the node knows the flags: an older `rand-node` (a
+ * v0.6.2 build, say, passed as RAND_NODE_BIN) cuts a plain chain and the test still runs against
+ * it — on a spend-key job, which then needs a prover started with --accept-spend-key.
+ */
+export function chainArgs(nodeBin) {
+  const help = execFileSync(nodeBin, ['genesis', '--help'], { stdio: 'pipe' }).toString();
+  const gas = help.includes('--bundle-gas-limit') ? ['--gas-price', '100', '--byte-price', '800', '--bundle-gas-limit', '20479'] : [];
+  const auth = help.includes('--auth-guest')
+    ? ['--bundle-guest', 'v3', '--auth-guest', '--max-proof-bytes', '4194304', '--max-block-bytes', '20971520']
+    : [];
+  // BIND-1: a chain after 19 binds its genesis hash, and this wallet builds every transaction for
+  // one that way — so the test chain must say so, or its node expects the chain-id binding.
+  const bind = help.includes('--binding-domain') ? ['--binding-domain', '1'] : [];
+  // Extra `rand-node genesis` arguments, to run the same send on a chain with more sections — say
+  // chain 21's `--fees F.json --multisig M.json` (fullnode v0.8.0). Space-separated; none by default.
+  const extra = (process.env.RAND_E2E_GENESIS_ARGS || '').split(/\s+/).filter(Boolean);
+  return [...gas, ...auth, ...bind, ...extra];
+}
+
+/** Whether the node cuts a split-authorisation chain (see `chainArgs`). */
+export function splitAuthorisation(nodeBin) {
+  return chainArgs(nodeBin).includes('--auth-guest');
+}
+
+/** Spendable RAND notes of a scan result. */
+export const spendableRand = (scan) => (scan.notes || []).filter((n) => !n.spent && !n.pending && Number(n.asset) === 0);