Skip to content

ci: faster CI and release runs, and the two flaky Auth tests - #191

Merged
sanjevirau merged 7 commits into
mainfrom
ci/faster-runs
Nov 12, 2026
Merged

sanjevirau merged 7 commits into
mainfrom
ci/faster-runs

Conversation

@sanjevirau

Copy link
Copy Markdown
Collaborator

Faster CI and release runs, and the two Auth tests that kept making a re-run necessary. Part of the 0.2.4 milestone.

What changes

  • The engine no longer rebuilds on every cargo build in a checkout without a console build. crates/console-front/build.rs watched console/dist/index.html. Cargo re-runs a build script on every build while a watched path is missing, so console-front, suite-runtime and the CLI were recompiled each time: about 43 s on a CI runner, in most steps of the Differential harness and the client replays. Locally, a no-change build goes from 5.6 s to 0.15 s; adding, changing or removing a console file still re-embeds it.
  • CI: the sign-in p99 test fails now and then on the Intel macOS runner #190, the sign-in p99 test measures up to three rounds and passes on the first within both bounds. It failed release 0.2.3's first attempt and chore(release): prepare 0.2.3 (engine 2294132, the Indexes page, three reported bugs fixed, a container image) #184's and chore(deps): bump notify from 8.2.1 to 8.2.2 in the rust-dependencies group #185's packages runs. The persistence test flushes before removing the state directory, so the writer thread can't add a file to it meanwhile ("Directory not empty").
  • darwin-x64 is built on Apple silicon for x86_64-apple-darwin (checked with lipo), and its release-mode tests, npm and Bun installs and suite smoke run the x86_64 engine with an x64 Node under Rosetta 2. The Intel runner took 20–47 min for this job, twice the next slowest platform, and set the length of every packages and release run.
  • The draft GitHub release gets this version's changelog section (packaging/release-notes.mjs), not the whole changelog. On 0.2.3 it gives exactly the notes that were published.

Checked locally: the Auth tests in release (82 passed, the two changed tests 5 more times each), node --test packaging/*.test.mjs (195 passed), node packaging/check-release.mjs, the independence check, and the build script's rebuild behaviour in a fresh checkout, after a console build appears, after a nested asset changes and after it is removed.

Before/after run timings follow in a comment once this PR's runs finish.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2

sanjevirau and others added 4 commits November 12, 2026 15:32
The build script watched console/dist/index.html, which is missing until
the console is built. Cargo re-runs a build script on every build while a
watched path is missing, so in such a checkout every `cargo build`
recompiled console-front, suite-runtime and the CLI: about 43 s each time
on a CI runner, in most steps of the Differential harness and of the
client replays. It now watches the folder only (Cargo walks it), and dates
a folder it creates to the script, so a fresh checkout builds once.

A no-change build goes from 5.6 s to 0.15 s locally; adding, changing or
removing a console file still re-embeds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
The sign-in p99 test measures up to three rounds and passes on the first
within both bounds; the p99 of 200 sign-ins is the third slowest, and a
busy Intel macOS runner failed its ratio about one run in ten. A sign-in
that waited for the encode still fails every round. Fixes #190.

The persistence test flushes before it removes the state directory, so
the writer thread cannot create a temporary file in it meanwhile
("Directory not empty").

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
The Intel macOS runner spent 18-25 minutes on the release build alone and
20-47 on the job, twice the next slowest platform, and it set the length
of every packages and release run. The engine is now built for
x86_64-apple-darwin on the Apple silicon runner (checked with lipo), and
its release-mode tests, the npm and Bun installs and the suite smoke run
the x86_64 engine with an x64 Node under Rosetta 2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
The draft took the whole changelog, and the release owner cut it down to
the version's section by hand. packaging/release-notes.mjs takes that
section; on 0.2.3 it gives exactly the notes that were published.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
@sanjevirau sanjevirau added this to the 0.2.4 milestone Nov 12, 2026
sanjevirau and others added 3 commits November 12, 2026 15:54
Under Rosetta, Bun's npm installer picks its Apple silicon build, which an
x64 Node did not install, and fails. The x64 baseline build is installed
directly (Rosetta offers no AVX), so Bun installs the darwin-x64 package
as on an Intel Mac; the step checks that Bun reports x64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
The journeys need the engine, not the console checks: the engine is built
in a job of its own beside them, so the journeys start 2.5-4.5 minutes
sooner. The six short journey projects (3-6 minutes each) run in pairs and
the Functions journeys on Node 22 and 26 in one job, so a run asks for
four fewer runners and waits less for them. Shell stays alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
The Functions, Cloud Tasks, Auth and Pub/Sub replays were the harness's
longest steps, about 9 of its 19 minutes once the engine stopped
rebuilding. They compare recorded answers and start no Java, so they run
in a job of their own with their own ledger records and evidence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GojiTpvSY8YywZFwh2ZJg2
@sanjevirau

Copy link
Copy Markdown
Collaborator Author

Before and after. Wall time is from the run's start to its last job. "Before" is the last five runs of each workflow on main or a code PR before this one.

Before After
CI 25.1–31.0 min (median 30.6) 13.9 and 13.4 min (two runs)
— the build-script fix alone 20.1 min
Differential harness 24.6–30.5 min 9.8 and 8.3 min, with the oracle replays (12.0 and 11.7) beside it
Client replays and real SDK checks 15.3–18.0 min 8.2 and 8.9 min
CI runner time 155.7–167.5 min 143.9 and 144.7 min
Installable packages 35.4–38.5 min (median 37.0) 17.6 min
— darwin-x64 job 30.1–37.9 min (46.8 in release 0.2.3) 17.2 min; its release build 8 min 40 s, against 18–25 on the Intel runner
Packages runner time 81.5–93.2 min 68.7 min

A release runs CI and the packages together, as a pull request does, so its build phase before the approval should go from 34.8 min (0.2.2) and 35.4 (0.2.1) to about 18. For 0.2.3 that phase took 76.4 min, because #190 failed the first attempt. That figure is an estimate from this PR's runs; a release can't be dry-run.

The darwin-x64 job on this PR ran on the macos-15-arm64 image with Node darwin-x64 v24.20.0. lipo reported x86_64, the release-mode tests passed on the x86_64 binaries (82 Auth, 1 suite control), and the npm, Bun and suite smokes installed and ran @firenook/cli-darwin-x64.

Two failures here were not caused by this change. linux-x64's old-glibc step hit a Docker Hub 502 Bad Gateway pulling node:24.20.0-alpine; it passed on a re-run. CI's second run failed functions-invocations.spec.ts:38 once ("Show" detached while the feed re-rendered); the same test failed on 2026-10-27 in run 40599798543.

@sanjevirau
sanjevirau merged commit ba4bb3d into main Nov 12, 2026
70 of 72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant