Skip to content

Snapshot hardware compatibility: stamp snapshots and workers with a HardwareIdentity and place resumes on matching workers #2246

Closed
3 of 3 issues completed

Description

@shrutiyam-glitch

Part of actor lifecycle v2 (#2069).

Motivation

A memory snapshot taken on one CPU does not restore on a different one: the guest state carries ISA, CPUID FeatureSet, and MSR assumptions of the CPU model that produced it. Today the scheduler only matches sandbox class and labels (cmd/ateapi/internal/scheduling), and nothing records
which hardware a snapshot was taken on. A fleet with more than one machine type or CPU generation therefore has no way to prefer a compatible worker on resume, and a mismatch surfaces only as a sandbox load failure that crashes the actor (#1657).

Golden snapshots have the same problem: one golden per template cannot serve a fleet with two CPU models. Per-hardware on-demand goldens (tracked separately) need this work first.

Design: https://docs.google.com/document/d/1CeN65qu64tGYvARYUUrwmESR4S0tbVXmRKubpsAQIdY/edit?tab=t.17rkgpfv7ql7

Proposal

  1. Add a Hardware message to ateapipb advertised by workers and stamped into snapshots (sibling to SandboxRuntime / SandboxClassConfig from How to patch CVEs and upgrade gVisor / microVM version used by SandboxConfig? #587):

    // Advertised by every worker at registration; stamped into every snapshot.
    message HardwareIdentity {
      // Key-value pairs describing host hardware properties (e.g. "architecture", "cpu_vendor", "cpu_model")
      map<string, string> attributes = 1; 
    }
  2. Every Worker reports its Hardware to the control plane (WorkerStatus.hardware) when registering capacity via WorkerService (Register / SetWorkerCapacity). atelet collects the host's Hardware and attaches it when relaying the worker's capacity report so capacity and hardware are recorded atomically.

  3. Every ExternalSnapshot and LocalSnapshot records the Hardware of the worker it was captured on. Tags and CreateActor's borrowed golden snapshot carry it along.

  4. Resume placement prefers a worker whose Hardware matches the actor's latest snapshot for memory restores. When no compatible worker exists, fall back to a cold boot at the template's minimum scope (ActorTemplate: replace on_pause/on_commit/on_resume with capture_scope and minimum_scope #2102) on any worker matching"architecture" instead of failing with ResourceExhausted or crashing.

Out of scope here: CPU feature masking, GPU and device compatibility, per-hardware golden snapshots.

Activity

  1. added this to the M3 milestone on Oct 6, 2026
  2. shrutiyam-glitch commented on Oct 6, 2026

    @shrutiyam-glitch
    CollaboratorAuthor
  3. zlammerts-svg commented on Oct 13, 2026

    @zlammerts-svg
    Collaborator

    Pending PR: #2536

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions