Skip to content

feat(sandbox): release agent-tools drives every box can rely on (PR A, draft) - #9929

Merged
saddlepaddle merged 1 commit into
mainfrom
superset/agent-tools-drive-release
Nov 18, 2026
Merged

saddlepaddle merged 1 commit into
mainfrom
superset/agent-tools-drive-release

Conversation

@saddlepaddle

@saddlepaddle saddlepaddle commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Draft. PR A of two. This one changes only the release and the drive name a wake mounts; the box side is #9911 (PR B), stacked on this branch.

What it does

  • The release publishes the bundle's agents drive in all 19 regions, and fails before the image, the golden or any row if a region with an environment has no drive, or a drive without claude, codex or pdev.
  • Pruning is safe for rollback and for other databases: every environment row (archived too) moves to the new bundle, the 3 newest released bundles keep their drives in every region, and only --production prunes.
  • A wake mounts the drive of the box's region (not the environment row's), and the golden, probe and real-sandbox boxes mount the drive and check that the agents run from it.

Pseudocode

release:
  needed = {sfo1} ∪ {row.region for every environment row, archived too} ∩ known regions   # B10
  results = publish drive in SANDBOX_REGION_IDS (6 at a time)
  for each needed region: no drive, or manifest lacks claude|codex|pdev -> FAIL               # B4
  kept-previous -> ::warning:: "<region>: <agent> <version> failed; <drive> keeps <sha12> from <parent>"
  image, golden (agentTools: true), probe (agentTools: true; presets resolve into /opt/agents; wake relinks)
  rows: every environment row -> new bundle (archived too)                                    # B1
  if --production:                                                                            # B3
    keep = {drive(row.region, row.bundleSha) for every row}
         ∪ {drive(r, b) for r in 19 regions, b in [new bundle] + 3 newest sandbox_bundles}
    delete agent-tools-* drives not in keep (attached ones are retried next release)
  else: prune skipped

wake (flagged orgs, unchanged otherwise):
  drive = agent-tools-<sandbox.region>-<identity.SUPERSET_BUNDLE_SHA[:12]>                    # B7
create: drive = agent-tools-<environment.region>-<bundle>   (a fork lands in the golden's region, which the row names)

What a customer sees

Nothing changes on any box: the flag still decides who mounts the drive, and every box still has its disk agents. Two visible effects:

  • Workspaces of an archived environment get the current bundle on their next wake, like every other workspace, instead of staying on the bundle the environment had when it was archived.
  • A release can now fail on agents drives, so a box may keep the previous bundle for one more release cycle.

Evidence

  • packages/sandbox bun test: 67 pass. New tests: agentToolsGaps (needed region with missing drive or missing required agent; optional agent and un-needed region ignored), referencedAgentToolsDrives (row drives + recent bundles × regions), the kept-previous warning text with region and version, and available from a publish.
  • packages/trpc bun test: 986 pass. New test: a wake mounts agent-tools-<sandbox.region>-… even when the claim's environment is elsewhere.
  • bun run lint:fix, bun run typecheck (root): clean.
  • Real Vercel: the coordinator reports the real-sandbox check passed on feat(sandbox): agent CLIs only from the agents drive (PR B, draft) #9911's earlier head (52cb11cb); not re-run on this head (no credentials here).
  • Not run: bun run release, real-sandbox, agent-tools (no credentials; creating sandboxes is not allowed here). No drive publish time was measured (see decision 4).
  • B1 count, production (Neon project Superset, default branch, read-only SELECT): 3 live workspaces in 2 archived environments, on 1 frozen (region, bundle) drive.
    SELECT count(*) AS workspaces, count(DISTINCT e.id) AS archived_environments,
           count(DISTINCT (e.region, e.bundle_sha)) AS frozen_drives
    FROM cloud_workspaces w JOIN environments e ON e.id = w.environment_id
    WHERE w.deleted_at IS NULL AND w.status <> 'archived' AND e.archived_at IS NOT NULL;
  • B3, shared project: deploy-production.yml, deploy-paradev-staging.yml, deploy-preview.yml and sandbox.yml all read secrets.VERCEL_SANDBOX_TEAM_ID / VERCEL_SANDBOX_PROJECT_ID. Staging runs in the production GitHub environment with production's DATABASE_URL, so it shares production's rows. Preview runs in the preview environment on a Neon branch of production. The token here cannot list environment secrets (403), so I could not confirm whether preview overrides the project id. The mismatches doc (~812) says everything lives in one sandboxes project. Conclusion: assume one project for every database. Only sandbox.yml's release --production runs in CI; a release from a dev or preview database would see only its own rows, so this PR makes every non-production release skip the prune.

Risks and rollback

  • A release now fails when a needed region's drive is incomplete. That blocks the release, and the image and rows stay on the previous bundle. This is intended, but it means a flaky upstream (claude, codex or pdev download) can hold releases. The earlier drive's version is kept (kept-previous) when the new one fails its check, so this fails only when a region has no earlier drive either.
  • Drives in 19 regions: more builder sandboxes per release (forks of the region's last drive, so only changed agents are fetched).
  • Preview databases (Neon branches) name production's bundles from their branch time; once those bundles leave the newest 3, the production prune deletes their drives. Preview boxes of the flagged org then boot with their disk agents (this PR), or without agents (after PR B).
  • Rollback, step by step:
    1. Revert this PR on main.
    2. Nothing on boxes changes: the next release returns to "regions with an environment" and the old prune.
    3. The old prune deletes the drives this PR kept (all-region and N-1, N-2) on its first run. Before reverting feat(sandbox): agent CLIs only from the agents drive (PR B, draft) #9911 (PR B) as well, re-run its rollback with drives present (see feat(sandbox): agent CLIs only from the agents drive (PR B, draft) #9911).

Needs before merge

  • One real-sandbox dispatch on this head.
  • A release on this head (or a dry run of the publish step) to measure the 19-region publish time, and to confirm every region's manifest has claude, codex and pdev.
  • Owner's answer on decisions 3 and 4.

Decisions for the product owner

  1. No drive → open without agents, or fail the open? Recommend: open without agents. Implemented in feat(sandbox): agent CLIs only from the agents drive (PR B, draft) #9911 (PR B), not here.
  2. Keep disk agents one more release as a fallback? Recommend: no, once N-1 and N-2 drives are kept (this PR). A rollback then has drives to mount.
  3. Should workspaces of archived environments keep working? Recommend: yes. Implemented here by moving archived rows to the new bundle; 3 such workspaces exist in production today.
  4. Cost: drives in all 19 regions per release, vs only regions with environments? Recommend: all 19, because an environment can be created in any region before the next release. Publish time not measured: I have no Vercel credentials here. The release log prints one agent-tools: line per region with a timestamp; the first release on this head gives both numbers (all regions vs the few with environments).

Sequencing

  • feat(sandbox): agent CLIs only from the agents drive (PR B, draft) #9911 (PR B) is stacked on this branch and merges only after a release with this PR has succeeded and the drives are verified in every region.
  • The API deploys on every push to main (deploy-production.yml), independently of the sandbox release; this PR is safe in either order because it changes no box.

Summary by cubic

Publishes the agent-tools drive in all 19 regions on every release so a box in any region can mount its bundle's agents, and fails the release before the image, golden, or rows change when a region with an environment has no drive or a drive missing claude, codex, or pdev. Previously drives were published only for regions with environments and an incomplete one only warned.

A wake now mounts the drive named by the box's own region instead of its environment row's, and archived environments' workspaces move to the current bundle on their next wake instead of staying on the bundle from when they were archived.

  • Only a --production release prunes drives; it keeps every environment row's drive plus the 3 newest released bundles in all regions, so rollback and mid-release environments keep their mounts.
  • The golden, probe, and real-sandbox mount the drive, and the probe verifies the cloud agent presets run from /opt/agents and the wake relinks them.
  • A release can now fail on a missing or incomplete drive, so a box may keep the previous bundle for one more release cycle.

Written for commit fb4ed84. Summary will update on new commits.

View guided diff Turn on auto-fix

The release publishes the bundle's agents drive in every region and fails
before the image, the golden or any row when a region with an environment
has no drive, or a drive without claude, codex or pdev. A kept-previous
agent warns with its region and the version that failed.

Every environment row moves to the new bundle, archived ones too, so their
workspaces' drives are not pruned. Only a --production release prunes, and
it keeps each row's drive and the 3 newest released bundles in every region.

The drive a box mounts is named by the box's own region on a wake. The
golden, the probe and real-sandbox mount the drive, and the probe checks
each cloud preset resolves into /opt/agents and that the wake relinks.
@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: dcd352c5-aae3-4d29-8eb8-1cd11805f687
Commits

Reviewing files that changed from the base of the PR and between 5fab249 and fb4ed84.

Files selected for processing: 11

  • packages/sandbox/README.md
  • packages/sandbox/src/agent-tools.test.ts
  • packages/sandbox/src/agent-tools.ts
  • packages/sandbox/src/environments/probe.ts
  • packages/sandbox/src/environments/release.ts
  • packages/sandbox/src/real-sandbox.ts
  • packages/trpc/src/lib/sandbox/agent-tools.test.ts
  • packages/trpc/src/lib/sandbox/agent-tools.ts
  • packages/trpc/src/lib/sandbox/claim.ts
  • packages/trpc/src/lib/sandbox/vercel.test.ts
  • packages/trpc/src/lib/sandbox/vercel.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@expo

expo Bot commented Nov 18, 2026

Copy link
Copy Markdown

No installed development build can load this pull request as an update: its native fingerprint is 08a5e03ce5fb and no development build has it. Add the mobile-build label to build one.

@github-actions

github-actions Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

🧹 Preview Cleanup Complete

The following preview resources have been cleaned up:

  • ✅ Neon database branch

Thank you for your contribution! 🎉

@saddlepaddle
saddlepaddle marked this pull request as ready for review November 18, 2026 03:55
@saddlepaddle
saddlepaddle merged commit cc593b8 into main Nov 18, 2026
30 of 31 checks passed
saddlepaddle added a commit that referenced this pull request Nov 18, 2026
Main's #9929 (agent-tools drives every box can rely on) touched files this
branch also changed. The merge result kept this branch's side for all of
#9929's files, so they were reset to main's versions: packages/sandbox
agent-tools, environments/probe and release, real-sandbox, trpc vercel and
vercel.test, and the sandbox README (plus this branch's one README line).

claim.ts is main's version plus this branch's box log lines. main reduced
trpc lib/sandbox/agent-tools.ts to one PostHog call (agentToolsDriveEnabled),
so the claim now asks that flag through organizationFlagEnabled in
feature-flags.ts and agent-tools.ts and its test are removed.

This branch was previously deployed

1 inactive deployment
preview — fb4ed849 Deployed Nov 18, 2026 by saddlepaddle via Cleanup Preview Resources #5760
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant