Skip to content

Reuse a complete runtime verification between builds - #48

Merged
grawish merged 3 commits into
mainfrom
perf/runtime-verification-reuse
Oct 15, 2026
Merged

grawish merged 3 commits into
mainfrom
perf/runtime-verification-reuse

Conversation

@grawish

@grawish grawish commented Oct 15, 2026 •

Copy link
Copy Markdown
Owner

Requested by Grawish · project thread

Before: every warm build, every compiler status check and each self-test compile rehashed all ~3,982 runtime files (~372 MB). That accounted for roughly 0.7–0.85 s of a 0.85–1.0 s warm build, and it was the largest main-process CPU context in the long-session V8 diagnostic. First-run preparation also hashed the new copy three or four times and read whole engine binaries (up to 60 MB each, four at a time) into main-process memory.

After: a complete verification is reused while nothing about the runtime has changed. In a synthetic benchmark with the real file count and sizes, a reused check takes 0.13–0.14 s instead of 1.09–1.14 s for a full one. Preparation hashes the new copy once, and it hashes the installer during the copy instead of in a separate full read beforehand. Installer files are streamed through a 1 MiB buffer per copy worker. In the same synthetic setup (self-test stubbed), complete preparation went from 6.1–6.7 s to 4.7–4.8 s.

A result is reused only when all of the following still hold. Otherwise every byte is rehashed.

  • The manifest still produces the exact same runtime pin, and the project's recorded selection still matches it.
  • A fresh walk still finds only regular files named in the manifest, with no links, special files or extras.
  • Every file and folder keeps the same device, inode, size, mode, link count and nanosecond mtime and ctime. A write, replacement, rename or chmod always changes ctime, and ordinary programs cannot set ctime back.
  • The result is less than 10 minutes old (runtimeReusePolicy.maxAgeMs).

Each new process starts with nothing to reuse. A failure is never remembered. A result is stored only when the identity walk after hashing matches the walk before it. Install, repair, resource-pack assembly and the installer-copy check still use the always-complete verifyRuntime. The bibliography self-test is unchanged.

How: electron/core/runtime.ts splits verifyRuntime into a tree walk and a byte-hash step and adds verifyRuntimeReusing, which inspectRuntime now calls. In runtime-manager.ts, the post-copy check uses the reusing variant so that it seeds the cache, and the staging copy streams through a bounded buffer. The staging copy also hashes every inventoried installer byte, so a damaged installer fails before any file is renamed into place, and the first failure stops the other copy workers. All file and folder flushes are unchanged. A new runtime-manager test covers a same-size damaged installer: nothing is published and a repair works once the installer is restored. Four new tests in tests/runtime-verification.test.ts cover the following: no re-hashing on a hit, same-size and same-mtime byte changes, rename-replaced, added and symlinked files, expiry, and changes made during hashing. The design and its limits are written up in docs/PERFORMANCE_IMPROVEMENT_PLAN.md.

Tested in a Linux container: typecheck, npm run build, and npm test (508 pass). The one failure is the existing test that requires macOS sandbox-exec, and it also fails on main there. This has not been measured on an Apple silicon Mac or in the packaged app yet. Re-running scripts/profile-preview.mjs would confirm the warm-build gain before any release claim changes.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BgdVQjEFKCM4UR2MTytQgv


Generated by Claude Code


Generated by Claude Code

@grawish grawish self-assigned this Oct 15, 2026
Every build, status check and self-test compile rehashed all ~3,982 runtime
files (~372 MB), which was most of a warm build and the largest main-process
CPU consumer. inspectRuntime now reuses a complete verification while the
exact pin, the file set and every file's identity (dev, inode, size, mode,
nlink, ns mtime/ctime) are unchanged, for at most ten minutes. Any change,
a new process, a failure or an expiry rehashes every byte. Install, repair,
pack assembly and installer checks keep the always-complete verifyRuntime.

Preparation also streams installer files through a bounded buffer instead
of reading whole engine binaries into memory.

Claude-Session: https://claude.ai/code/session_01BgdVQjEFKCM4UR2MTytQgv
@grawish
grawish force-pushed the perf/runtime-verification-reuse branch from 6ea9378 to bcfdc67 Compare October 15, 2026 19:51
First-run preparation read and hashed the whole 372 MB installer runtime,
then read it again to copy it. Staging now hashes each inventoried file
against the manifest as it streams the copy, after a layout and identity
check of the installer. A mismatch fails before the file is renamed into
the private generation, which is then removed. The first copy failure also
stops the remaining workers.

Claude-Session: https://claude.ai/code/session_01BgdVQjEFKCM4UR2MTytQgv
- Age reused results with a monotonic clock, so moving the system clock
  back cannot extend reuse past ten minutes.
- Reject inherited object names (constructor, toString, ...) as unexpected
  runtime resources in every walk.
- Hash source bytes during staging only for the trusted installer; pack
  bases are already fully verified, so packs no longer hash them twice.
- Batch folder identity lookups with file lookups, and build verifyRuntime
  on checkRuntimeLayout.

Claude-Session: https://claude.ai/code/session_01BgdVQjEFKCM4UR2MTytQgv
@grawish
grawish marked this pull request as ready for review October 15, 2026 20:18
@grawish
grawish merged commit f5e23d8 into main Oct 15, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants