Repository navigation
fix(sandbox): no swap on cloud boxes; memguard reaps early enough without it - #9924
Merged
saddlepaddle merged 1 commit intoNov 18, 2026
Conversation
…hout it memguard no longer grows swapfiles. A snapshot is a full block copy of the disk, so swap written once stayed in every later snapshot (up to 1.5x box RAM, 6-16 GB). At start it still turns off and removes /var/swap/superset-* from an earlier bundle. Without swap a late reap is a stall, not a slow box, so the loop changes: - check MemAvailable every 0.25 s with no forks (rank still once a second) - reap when the next check at the current rate of fall would be under the floor, with a fresh scan before choosing the victim - after a kill, wait for the victim to exit (at most 3 s), not a fixed 3 s - kill before logging Measured on 4 and 8 GB boxes under four kinds of pressure: host-service's longest silence 314 ms, no kernel OOM kill. Main (swap, 1 s check) let a 2 GB/s allocator win in 6 of 7 runs, with silences up to 94 s.
Contributor
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Contributor
🧹 Preview Cleanup CompleteThe following preview resources have been cleaned up:
Thank you for your contribution! 🎉 |
saddlepaddle
marked this pull request as ready for review
November 18, 2026 02:59
saddlepaddle
added a commit
that referenced
this pull request
Nov 18, 2026
* chore(sandbox): remove the two spent probe scripts; add the review page snapshot-probe.ts created swapfiles to measure what a snapshot keeps, and drive-fork-probe.ts measured drive forks on throwaway drives. Both results are in plans/20261116-thin-sandbox-findings.md, boxes have no swap since #9924, and the real-sandbox drive pass now times a fork from a golden with content. Nothing imports either script; the workflow option for the fork probe goes with it. * docs(plans): the review page says swap removal shipped, and when an agent can be reaped
saddlepaddle
added a commit
that referenced
this pull request
Nov 18, 2026
Since #9924 cloud boxes have no swap and memguard reaps early enough without it. memguard is main's as is: no swap directory, on the drive or anywhere. The layout no longer creates /mnt/superset-drive/swap, and the "memguard swaps on the drive" checks leave boot-twice and real-sandbox. main's memguard already removes the old /var/swap files. No drive box can hold an old drive swap directory: no environment has been in drive mode, and every box the real check made was deleted. So nothing more is cleaned up.
10 tasks
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft for the product owner's review. It replaces the closed draft #9843: it re-applies that change on current main and adds the memguard changes that the measurements below showed are needed.
What changed
superset-memguard: no swap. Removedgrow_swap/shrink_swap, the swap constants,box_mband the swap cap. At start it still turns off and removes any/var/swap/superset-*an earlier bundle left, then removes/var/swap.MemAvailableevery 0.25 s (was 1 s), and nothing between checks forks: the pause isread -ton a pipe, notsleep. Ranking still runs once a second.available − (fall since the last check)is under the floor, so a fast allocator is caught one check early. Before choosing the victim it does a fresh scan.date). The reap line gainsdrop=…MB; host-service's parser reads only the prefix, so the pressure banner is unchanged.superset-boot: comment only. It still removes the old fixed/swapfile.boot.test.ts: the swap-cap test is replaced. New tests check that nothing turns swap on, that the old swapfiles are removed, the reap condition, that there is nosleepin the loop, and that it kills before it logs.cloud-sandbox-considerations.mdnow has "No swap (2026-11-17)" in place of "Swap is on demand", with the measurements and the open item for Vercel's OOM webhooks.cloud-sandbox-mismatches.md: the open item no longer says "swap included".superset-bootreads only a comment change, so--skip-imageis fine for that file.memguard's loop after the change
Why the loop changed (what the first runs showed)
With and without swap: four cases at 4 GB and 8 GB
"yes (main)" is memguard from main with on-demand swap. "no (branch)" is this PR. Each (d) row has three or four runs, because (d) is a race.
/trpc/health.checkanswers, probed every 250 ms from outside the box's memory (a C prober,mlockall).claude, rank −300) died. None died in any run. A 300 MB "dev server" (rank 500) also survived every run, because the reaps always picked the larger allocator.comm=MainThread. See "What the user sees".The workloads, run as the sandbox user: (a) one node process allocating 100 MB/s until it is killed. (b) 2×vCPU workers, each growing at 100 MB/s to a share of 1.5× the box, holding 20 s, then exiting, like a parallel build. (c) 0.75× the box of files written, then read in a loop together with every small file under
/usrand/opt, while an allocator grows at 80 MB/s, like a large install. (d) one node process allocating 128 MB chunks as fast as it can, about 2.2 GB/s.How this was measured, and what that means for trust
domain threaded, so no child cgroup can enable the memory controller, anddocker run --memoryfails with "cannot enter cgroupv2 … in threaded mode". Anyone repeating this on a cloud workspace will hit the same wall.vm.nr_hugepages) took RAM out of the VM so that what was left was 4 or 8 GB. The real image (superset-sandbox:local) ran in a privileged container pinned to 2 or 4 CPUs, booted with the realsuperset-boot: host-service, pty-daemon and the Xfce desktop all came up. Then the memguard under test replaced boot's copy. The only edit to memguard was to make it see the box size:MemTotalis set to 4 or 8 GB after the read, and on main the cgroup read for the swap cap is replaced by the same value. Swap on main was real (swapfiles on this VM's disk,vm.swappiness=10as memguard sets it). The kernel reclaim, page cache and OOM killer were the VM's own, which matches a real box better than a cgroup limit would./tmp/memtest; I can attach it on request.What the user sees when something is reaped
BoxOutOfMemoryBannershows: "Out of memory: the box stopped {process} ({n} GB) to keep running. {size} has {m} GB." Beside it is a "Resize to {next size}" button, which opens the machine-size menu with the suggested size, and a dismiss button. The sidebar shows a pressure dot on that workspace in every window.Killedfrom the shell, with exit 137. An agent sees exit 137 from its tool.commtoMainThread, so a reaped Vite or Next.js dev server shows as "the box stopped MainThread (2.4 GB)". Next.js shows asnext-server (v1(cut to 15 characters). memguard could log the script from argv (it already reads it for ranking), and the banner could show that.nextMachineSizereturns the current size, so no button appears and the text gives no next step, such as "run fewer jobs in parallel".cloud-sandbox-considerations.md. The API would report them to the realtime hub beside host-service's report (runtime/sandbox-box-pressure). They would also cover a box whose host-service is itself stalled, which host-service's own report cannot.Product implications
Snapshot saving
Swap was capped at 1.5× box RAM: 6 GB on a 4 GB box, 12 GB on 8 GB, 16 GB on 16 GB. A snapshot is a full block copy with no discard, so a workspace that ever came under pressure carried up to that much in every later snapshot. In these runs main wrote 3.6–6.0 GB of swap at 4 GB and 8.2–12.0 GB at 8 GB. After this change, any workspace that hits pressure from now on saves that, up to 6–16 GB per workspace. Workspaces that never grew swap save nothing. Snapshots that already hold swap keep it: removing the file does not zero its blocks.
Decisions that are yours
Checks
Ran:
bun testinpackages/sandbox: 66 pass, 0 fail.bash -n superset-memguard: passed.bun run lint:fix: passed, no fixes.packages/sandbox(bun run typecheckin the package): passed.startand ran; the fork-free pause measured 251 ms for 0.25 s.bun run --cwd packages/sandbox image --localon this branch: builtsuperset-sandbox:local(4.89 GB); the image's memguard is this PR's.src/runner-check.ts: all expectations met (30 ok).src/boot-twice.ts: 3 expectations failed, the rest pass. Boot, host-service (ready in 1193 ms, then 899 ms), the desktop,memguard.startwithmemguard.pid, the second-boot skips, and the drive cases all pass. The failures arefirst boot: checkout done,start hook: the runner asked host-service once the push landedandsecond boot: checkout marker respected. All three come fromcheckout.clone.failed: inside the container,git ls-remote https://github.com/paradevai/paradev.gitfails with "server certificate verification failed", because this workspace's egress needs a CA the image does not have. The same command on the host succeeds. These are the same three failures fix(sandbox): drop on-demand swap from memguard (draft: trade-off for review) #9843 reported, and memguard is not involved.Not run:
bun run typecheck. It needs about 8 GB, and this workspace's own memguard is on the old bundle, so it would grow swap on this disk. The change touches onlypackages/sandboxand docs.mainin this workspace, for a side-by-side comparison of the three clone failures above (their cause is shown directly instead).