Skip to content

api: report worker runtimes per sandbox class (WorkerSandboxClass, WorkerSandboxRuntime, ActorSandboxRuntime) #3008

Closed

Description

@shrutiyam-glitch

Context & Related Work


Problem Statement

Under the initial schema introduced in #2348, a Worker reports its runtime capabilities through two flat fields on WorkerStatus and RegisterWorkerRequest (default_runtime and restorable_runtimes), and RegisterWorker rejects any restorable_runtimes entry whose sandbox_class differs from default_runtime.sandbox_class. This causes two structural limitations that would require breaking API and wire changes to fix after GA:

  1. Flat worker fields can only represent multiple versions of a single sandbox class:
    While the WorkerPool CRD already models sandbox classes as a keyed list (WorkerPool.spec.sandboxClasses and WorkerPool.status.sandboxClasses, with MaxItems=1 today) so a worker pod can support more than one sandbox class in the future, a singular WorkerStatus.default_runtime can only ever describe one class. Replacing a singular default_runtime with a per-class list after v1.0.0 would be a breaking change.
  2. One SandboxRuntime message conflates worker-installed runtimes with actor/snapshot stamps:
    previously, SandboxRuntime was shared between:
    • What a worker installs for a class, which is already scoped to a class (making sandbox_class redundant inside a per-class entry) and will gain post-GA fields such as repeated VersionedSandboxCompat compatible_versions and an informational SandboxConfig version name.
    • What an actor's assignment (WorkerAssignment.sandbox_runtime), snapshot (Snapshot.sandbox_runtime), and atelet execution request (RunRequest / RestoreRequest) are stamped with, which always records one concrete sandbox_class and one primary VersionedSandboxCompat version, and never carries compatible_versions.

Doing this refactor now—while pre-v1.0.0 stored-data breaks are still permitted alongside #2888 and #2922—aligns the control-plane API with WorkerPool.spec.sandboxClasses and cleanly separates the worker and actor runtime types.


Proposed Protobuf Schema

Public Control-Plane API (pkg/proto/ateapipb/ateapi.proto)

message WorkerStatus {
  // ... fields 1..4 unchanged ...

  // [Removed] SandboxRuntime default_runtime = 5;
  // [Removed] repeated SandboxRuntime restorable_runtimes = 6;

  // The sandbox classes this Worker runs and the runtimes it runs them
  // under, as reported by the Worker at registration; empty until then, and
  // nothing is placed on the Worker until then. The first registration fixes
  // the set of classes; a later report naming other classes is refused. One
  // entry per class the pool declares: one today.
  //
  // +k8s:optional
  // +k8s:maxItems=1 # keep in sync with WorkerPoolSpec.sandboxClasses
  // +k8s:listType=map
  // +k8s:listMapKey=sandbox_class
  repeated WorkerSandboxClass sandbox_classes = 5;
}

// WorkerSandboxClass is one sandbox class a Worker runs, with the runtimes it
// runs it under. A Worker has one entry per WorkerPool sandboxClasses entry,
// so exactly one today.
message WorkerSandboxClass {
  // e.g. "gvisor", "microvm", as on WorkerPool.spec.sandboxClasses[].name.
  //
  // +k8s:required
  // +k8s:maxLength=63
  // +k8s:customValidation # gvisor or microvm
  string sandbox_class = 1;

  // Runtime that new and cold-booted actors of this class start under.
  // Snapshots matching its version can also warm-boot on this worker.
  //
  // +k8s:required
  WorkerSandboxRuntime default_runtime = 2;

  // Additional runtimes of this class (excluding default_runtime) that this
  // worker can restore snapshots from right now.
  //
  // +k8s:optional
  // +k8s:maxItems=32
  // +k8s:listType=atomic
  repeated WorkerSandboxRuntime restorable_runtimes = 3;
}

// One runtime installed on the worker for that class.
message WorkerSandboxRuntime {
  // Primary compatibility version stamped onto an actor's snapshot when a
  // checkpoint is taken on this runtime. Snapshots matching `version`
  // can warm-boot on this runtime.
  //
  // +k8s:required
  VersionedSandboxCompat version = 1;

  // TODO: Add an informational SandboxConfig version name (e.g. "gvisor-2")
  // once workers report one.
  //
  // [Post-GA] Additional compatibility versions (excluding `version`) that can
  // warm-boot directly onto this runtime. When the actor suspends again, its
  // new snapshot is stamped with the primary `version`.
  // repeated VersionedSandboxCompat compatible_versions = 3;
}

// Concrete runtime and compatibility identity recorded on an actor's
// assignment or snapshot.
message ActorSandboxRuntime {
  // e.g. "gvisor", "microvm"
  //
  // +k8s:required
  // +k8s:maxLength=63
  // +k8s:customValidation # gvisor or microvm
  string sandbox_class = 1;

  // Compatibility version stamped onto an actor's assignment or snapshot.
  //
  // +k8s:required
  VersionedSandboxCompat version = 2;
}

message RegisterWorkerRequest {
  ObjectRef worker = 1;
  WorkerResources capacity = 2;

  // [Removed] SandboxRuntime default_runtime = 3;
  // [Removed] repeated SandboxRuntime restorable_runtimes = 4;

  // +k8s:required
  // +k8s:minItems=1
  // +k8s:maxItems=1 # keep in sync with WorkerPoolSpec.sandboxClasses
  // +k8s:listType=map
  // +k8s:listMapKey=sandbox_class
  repeated WorkerSandboxClass sandbox_classes = 3;
}

Activity

  1. shrutiyam-glitch commented on Oct 20, 2026

    @shrutiyam-glitch
    CollaboratorAuthor
  2. added a commit that references this issue on Oct 21, 2026
    3b779a8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions