Skip to content

Add a nightly Beta channel for the Windows installer - #139

Open
whoisaldo wants to merge 8 commits into
mainfrom
feat/nightly-channel
Open

whoisaldo wants to merge 8 commits into
mainfrom
feat/nightly-channel

Conversation

@whoisaldo

@whoisaldo whoisaldo commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Adds a Beta channel for Windows: a nightly build of the newest fixes, published as a GitHub pre-release at a download link that never changes, with a Stable/Beta switch on the download page. It follows T3 Code's download page. Users only ever see the word "Beta"; internal names (the workflow file, scripts/nightly.py) say nightly.

Design

.github/workflows/nightly.yml (shown in Actions as Beta builds), with the build in the reusable .github/workflows/beta-installer.yml:

  • Plan (contents: read). scripts/nightly.py plan picks the source branch: the NIGHTLY_BRANCH repository variable if set, otherwise the newest release/X.Y.Z-candidate with commits main lacks, otherwise main. Today that is release/0.3.4-candidate. It also computes the version and tag, and skips the build when the branch head is the commit named by the completion marker (<!-- beta-commit: <sha> -->) in the beta release body. Publish writes that marker last, after everything is verified, so a run that stops partway is redone the next night. The tag position is not used.

  • Windows installer (contents: read, id-token: write for signing; scheduled and manual runs only) calls beta-installer.yml with secrets: inherit. A pull request instead runs Windows installer (pull request, no secrets), which calls the same workflow with no secrets and grants only contents: read. beta-installer.yml sets no permissions of its own, so it runs with exactly what the calling job grants. The scheduled and manual caller grants contents: read and id-token: write, so signing works there. The pull request caller grants only contents: read, so that run gets no OIDC token. GitHub's Reusing workflow configurations says: "The GITHUB_TOKEN permissions passed from the caller workflow can be only downgraded (not elevated) by the called workflow." In practice a called job that asks for more than its caller grants doesn't just get less: the whole run fails at startup ("The nested job 'build' is requesting 'id-token: write', but is only allowed 'id-token: none'"). That's why the build job declares nothing, rather than declaring id-token: write and relying on the caller to cap it. Its diagnostics env is empty, signing and Azure steps are skipped, and the artifact is still scanned. scripts/test_workflow_secrets.py fails if any job a pull request can reach gets a secret other than GITHUB_TOKEN, a secrets: pass-through or id-token: write. It also checks the grants: beta-installer.yml declares no permissions, the trusted caller grants contents: read + id-token: write, and the pull request caller grants contents: read alone and passes no secrets. beta-installer.yml has the same steps as release.yml on the candidate: the same FFmpeg, VDD and signing-client pins (test_nightly.py fails if they drift from release.yml), the same repo-level diagnostics secrets, the same optional Azure Artifact Signing (on only when the release environment has ARTIFACT_SIGNING_ACCOUNT, and only for runs on main), the update helper's import check, and scripts/artifact_secrets.py scan before the artifact uploads. The version is stamped into host/Cargo.toml and Cargo.lock in the runner's checkout only; nothing is committed. ETERNAL_BUILD_REVISION is set to the commit, so the stamped checkout doesn't report +dirty.

  • Publish (contents: write, the only job that can write). It runs only on main, never for a dry run. It checks the downloaded artifact against the build's SHA-256, then runs nightly.py publish:

    1. Create the dated pre-release with its asset (--prerelease --latest=false), or on a retry update the one that exists. Check it with an anonymous HEAD: HTTP 200 and the right size.
    2. On the moving beta release: upload the new installer as EternalMonitor-Setup-beta.next.exe and check the uploaded size.
    3. Rename the old asset to …old.exe, then rename …next.exe to EternalMonitor-Setup-beta.exe. These are two consecutive API calls, so the fixed link is missing for one rename call, about a second.
    4. Check the fixed link anonymously: 200, the size, and the SHA-256 of the downloaded bytes, then delete the old asset.
    5. Move the beta tag, and last write the release title, notes and completion marker.

    Every call in steps 2–4 sits inside one recovery path, reconcile(). A call can fail after GitHub has already applied it, for example a rename that went through before the connection dropped, so recovery does not assume what happened. It lists the beta release's assets again and keeps the asset the fixed link serves, unless that asset is the new build and fails its check. Otherwise the new build takes the fixed name if its size and the SHA-256 of an anonymous download match, else the old build does, and the fixed link is checked again. Every publish starts with the same reconcile, which puts the old build back if a run stopped inside recovery.

    Then it prunes old builds. Checks retry for up to a minute, since a new asset can take a few seconds to show.

Schedule: 07:41 UTC daily, plus workflow_dispatch with a dry_run input. A pull request that changes nightly.yml, beta-installer.yml or scripts/nightly.py also gets a dry run, without secrets. Concurrency is one run at a time (group: nightly).

Names

Version <next>-beta.<YYYYMMDD>.<run_number>, e.g. 0.3.4-beta.20261030.12
Dated tag beta-<YYYYMMDD>-<shortsha>, e.g. beta-20261030-3dc6b1e
Release title EternalMonitor 0.3.4 beta (Oct 30, 2026)
Moving tag and release beta
Asset EternalMonitor-Setup-beta.exe
Fixed link https://github.com/sidebandstudio/EternalMonitor/releases/download/beta/EternalMonitor-Setup-beta.exe

<next> is the host's Cargo version when it is ahead of the stable version in docs/latest.json on main, else that stable version with the patch number plus one. Today the candidate's host is 0.3.3 and stable is 0.3.3, so <next> is 0.3.4. If the host were bumped to 0.3.5, it would be 0.3.5. The date makes each build sort above the hand-made 0.3.4-beta.1 and below 0.3.4.

Retention: the newest 7 dated builds are kept. nightly.py expired selects only pre-releases whose tag fully matches beta-\d{8}-[0-9a-f]{7,40}, so it never selects beta itself, windows-0.3.4-beta.1 or any v* tag (tested).

Release notes (nightly.py notes) give the version, the source branch and commit, a compare link from the last stable tag, the line "the newest fixes, built every night. For testing; you can go back to Stable anytime.", how to go back to Stable, that iPad betas come through TestFlight, and the installer's SHA-256.

Download page: a Stable | Beta switch under the heading. ?channel=beta is the shareable link, and ?channel=nightly is accepted too. Stable is the default and unchanged: the same card, links, SHA-256 and latest.json reading, and the ids that check-versions.sh and latest_json.py edit are untouched. The Beta card has the one-line explanation, the fixed download link, and the version, build date and commit read from the beta release through the GitHub API (cached for 30 minutes; the link works without it). It also shows the SHA-256 when GitHub provides it, how to go back to Stable, and the existing unsigned-installer note. The iPad card (TestFlight) is the same in both channels. A line under both cards says every build is on the GitHub releases page. Without JavaScript the switch is hidden and the page shows Stable.

Night look: switching to Beta turns the download page to night, after T3's Nightly page but with original artwork:

  • a sky band fades in behind the header and title, with stars, a crescent moon that rises into place, and one shooting star;
  • the shared cards and buttons get a moonlit tint (the lime buttons become periwinkle);
  • the nav icon changes to a night version (docs/assets/icon-night.svg), and the Beta card's monitor icon shows a moon.

It takes about 0.9 s with an ease-out, and the same back to Stable. Only opacity and transform animate. Two star layers twinkle as whole layers, so no SVG repaints. Each visit to Beta twinkles them for about a minute (14 and 9 cycles), after which they hold still at full brightness. The shooting star runs once per switch. All sky animations pause (html.sky-paused, animation-play-state: paused) while the tab is hidden (visibilitychange) or the sky is scrolled out of view (IntersectionObserver). With prefers-reduced-motion the switch is instant, with no twinkle or shooting star. The head script sets the class before first paint, so ?channel=beta loads straight into night. The night layers aren't rendered at all until the first visit to Beta, so a Stable visitor who never touches the switch gets a pixel-identical page.

Docs: a "Beta channel" section in docs/beta-testing.md: what it is, where it comes from, the version rule, the fixed link, and how support can send download.html?channel=beta.

Why Stable users are unaffected

  • Release and TestFlight workflows: no tag starts with v, so release.yml and testflight.yml (push: tags: v*) never run for a Beta build. GitHub also doesn't start workflows for tags or releases created with GITHUB_TOKEN.
  • latest.json and the website's Stable download: nothing writes docs/latest.json. The Stable card still reads latest.json.
  • GitHub's latest release: every Beta release is a pre-release created and edited with --latest=false, so /releases/latest stays on v0.3.3.
  • In-app updates: the apps read latest.json, which rejects pre-release versions. newer_stable() never offers a pre-release, and the release check refuses pre-releases and any asset not named EternalMonitor-Setup.exe. A new test in host/src/updates/tests.rs pins the Beta case: 0.3.4-beta.20261030.12 is never offered to 0.3.3, and 0.3.4 is offered to the Beta build.
  • Publishing: a dispatch on any branch other than main is a dry run, and so is any pull request. Pull request builds get no secrets.

Dry run

Manual run with dry_run: true on this branch (run number 2): success. The plan step chose release/0.3.4-candidate at 3dc6b1e, with published stable 0.3.3, and computed:

  • version 0.3.4-beta.20261030.2
  • tag beta-20261030-3dc6b1e
  • title EternalMonitor 0.3.4 beta (Oct 30, 2026)

The build stamped eternal-host version 0.3.4-beta.20261030.2, Inno Setup compiled it, and the update helper import test passed. The artifact scan found no secret values in the artifact (6 secrets checked; the 3 Azure ones are optional and unset). The installer is 60.16 MB, unsigned (signing is not configured), SHA-256 1c7e56bc…b5155b. It was uploaded as the run artifact EternalMonitor-Setup-beta. Publish was skipped, and the repository still has no beta or beta-* tag or release.

This PR's own dry run, from the pull_request trigger, also succeeded with Publish skipped. That run predates the secret-free split; the current head's pull request run uses the no-secrets build.

Screenshots and checks of the page

Taken in headless Chrome from docs/ served locally:

  • Stable: unchanged except for the switch and the releases line.
  • Beta as it is live now: GitHub has no beta release yet, so the card says "The first Beta build is on its way. Use Stable until then."
  • Beta with a mocked beta release response at 390, 1024, 1280 and 1920 px, showing the version, build date, commit and SHA-256.
  • Frames 110 ms into each switch, and a recording of Stable → Beta → Stable → Beta.

What was measured:

  • Stable: full-page Stable screenshots at 1280 and 390 px are byte-identical before and after the night work.
  • Frame rate: with Chrome's CPU throttled 4×, each switch held 60 fps (worst frame 16.8 ms, none over 20 ms). Each switch does 2–5 layouts, about 3 ms in total.
  • Reduced motion: the night state applies within one frame, with no animations.
  • Stars: none overlap the heading, lead or switch at 390, 768, 1024, 1280, 1920 or 2560 px.
  • Contrast (WCAG AA): text over the sky and the night cards measures 5.8:1 to 16:1.

Checks

  • python3 -m unittest discover -s scripts -p 'test_*.py' passes: 169 tests. These include scripts/test_nightly.py and scripts/test_workflow_secrets.py, the pull request secrets check (with fixtures that must fail).
  • The publish tests drive nightly.py publish through a fake GitHub. Each call can fail in two ways: it fails before doing anything, or it applies the change and then the request fails. They cover a first build and an update in order. One at a time, they fail every call of an update (also starting with a leftover …old.exe) and of a first build. Every call is failed the first way, and every write (create, edit, upload, rename, delete, tag move) the second way. In every update case, once publish fails, the fixed link already serves a whole build, old or new. Then the next run finishes, ending with one asset, the right bytes, the tag moved and the marker written, and the link is never missing across two writes in a row. One test reproduces the review's case: the old build's rename applies and the request fails; the new build takes the fixed name before publish fails. A second fault inside recovery is covered for both failure modes; the link then serves a build at the latest after the next run's reconcile. The tests also cover a failed swap with every later call failing (the next run's reconcile restores the link before anything else), wrong bytes at the fixed link (the old build is put back after retries), and a dated build that never appears (the beta release is not touched). A command test checks that the real gh calls are pre-releases with --latest=false and never name a v tag.
  • cargo test -p eternal-host --lib updates:: passes: 39 tests.
  • actionlint (with shellcheck) passes. scripts/check-versions.sh passes.
  • A test merge with release/0.3.4-candidate (EternalMonitor 0.3.4 #114) has no conflicts. On the merged tree, 222 script tests pass, including the candidate's test_workflow_artifacts.py, and actionlint passes.

Follow-up (not in this PR): opt-in Beta updates in the app

An opt-in "Beta updates" setting would need:

  • a signed or hash-pinned Beta manifest (for example beta.json next to latest.json, written by this workflow through a PR or Pages), since the app must not trust the moving release alone;
  • newer_stable widened to accept -beta. pre-releases only when the setting is on;
  • check_github_release accepting beta-* tags and the EternalMonitor-Setup-beta.exe asset, with the SHA-256 still required from the notes;
  • the setting stored per install, with turning it off returning to Stable on the next Stable release (no downgrade);
  • diagnostics recording the channel.

A daily workflow builds the installer from the newest release candidate
(or main), the same way release.yml does, and publishes it as a GitHub
pre-release tagged beta-<YYYYMMDD>-<sha>. The moving beta pre-release holds
the newest build at a fixed download link. The download page gets a
Stable/Beta switch (?channel=beta), and beta-testing.md describes the channel.
Switching to Beta fades in a night sky with stars and a moon, gives the
shared cards and buttons a moonlit tint, and swaps in a night version of the
app icon. ?channel=beta loads straight into it. Everything animates opacity
or transform only, switches instantly with reduced motion, and the night
layers are not rendered until the first visit to Beta, so Stable renders as
before.
The star layers twinkle for about a minute per visit to Beta instead of
forever, and every sky animation pauses while the tab is hidden or the sky
is scrolled out of view. The shooting star still runs once per switch, and
reduced motion is unchanged.
The build moves to beta-installer.yml, a reusable workflow. Scheduled and
manual runs call it with the repository's secrets and an OIDC token; a pull
request, which runs its own code, calls it with neither.
test_workflow_secrets.py fails when a job a pull request can reach gets a
secret or id-token: write.

nightly.py publish replaces the shell steps. It creates the dated release
and checks it downloads anonymously, uploads the new installer beside the
working one, swaps their names in two consecutive calls, checks the fixed
link byte for byte, deletes the old asset, moves the beta tag, and last
writes the commit into the beta release body. Plan skips a commit only when
that marker names it, so a run that stops partway is redone. A failed swap
or check puts the old installer back first. Tests drive it through a fake
GitHub, failing each call in turn and rerunning.

The workflow's name is now Beta builds, as the docs say.
Every call of the asset swap now sits inside the recovery path, and the
recovery re-lists the beta release instead of assuming which calls went
through: the new build takes the fixed name if its size and SHA-256 check
out, else the old one does, and the link is checked anonymously. Every
publish starts with the same reconcile, so a run that stopped inside it is
repaired by the next one. The fake GitHub can now apply a change and then
fail the request, and the tests run that mode at every write.
The reusable build job asks for contents: read and id-token: write. A
called workflow gets at most its caller's permissions, so the scheduled
and manual runs can sign while the pull request run still gets no OIDC
token. The secrets test now checks both callers' grants.
GitHub refuses to start a run whose called job asks for a permission its
caller does not grant, so the build job's id-token: write broke every pull
request run. The reusable workflow now sets no permissions at all and runs
with exactly what the calling job grants: contents: read and id-token: write
from the scheduled and manual caller, contents: read alone from the pull
request one.
@whoisaldo
whoisaldo marked this pull request as ready for review October 30, 2026 04:18

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants