Shared volumes: actors mount an atespace-scoped SharedVolume and hand it off in turn

[hajiler]

Part of #1637 (sequential hand-off is the first milestone; concurrent READ_WRITE_MANY holders follow). Post-GA follow-up.

Depends on #2653, which carries the breaking change (Volume.external with the ExternalVolumeSource union, and atelet's handle renamed to ExternalVolume). This branch includes that commit until #2653 merges; review commits 2 to 5 here. The per-actor record on ActorStatus.external_volumes keeps its name, ExternalVolume.

It's a good idea to open an issue first for discussion.

• Tests pass
• Appropriate changes to documentation are included in the PR

What this adds

A SharedVolume is an atespace-scoped CSI volume whose lifetime is independent of any actor, so several actors can mount the same storage in turn: a producer writes, suspends, and a consumer resumes and reads. Design: go/substrate-shared-volumes. User docs: docs/csi-volumes.md, section "Shared Volumes".

kubectl ate create sharedvolume ctx-42 -a demo --storage-class csi-nfs-sc --capacity 10Gi
kubectl ate create sharedvolume imported -a demo --volume-id <csi-volume-id> --volume-type nfs.csi.k8s.io
kubectl ate create actor producer-42 -a demo --template producer --volume shared=ctx-42

volumes:
- name: shared
  external: {sharedVolume: {name: ctx-42}}   # or {sharedVolume: {}} and bind with --volume

• Resource and RPCs. Create/Get/List/Update/DeleteSharedVolume (annotated with (resource) and (method).resource, list paging through ListOptions), a Postgres table (migration 000004_shared_volumes.sql), store contract tests, authorization rules and an OpenFGA shared_volume type (atespace editors create, update and delete; viewers get and list), and kubectl ate {create,get,update,delete} sharedvolume.
• Provision or register. A volume is provisioned from a StorageClass or registers a volume that already exists (volume_id, volume_type, volume_context).
• Access mode. Fixed at create. Only READ_WRITE_ONCE is accepted today, so at most one resumed actor holds the volume; a resume that would take a held volume fails with FailedPrecondition naming the holder.
• Delete trigger. MANUAL (default when registering) removes the volume only on an explicit delete; LAST_ACTOR (default when provisioning) removes it when the last actor referencing it is deleted. Deleting the record always deletes the storage. A referenced volume cannot be deleted.
• References. status.refs lists every actor referencing the volume with active set while the actor is resumed. Create adds an inactive ref, resume activates it, pause and suspend deactivate it, crash leaves it as is, delete removes it.
• Template binding. Volume.external.sharedVolume either names the volume or leaves the name empty so each actor binds it at creation through Actor.shared_volume_bindings (--volume <slot>=<volume>). Bindings are validated at create and immutable afterwards. The per-actor ExternalVolume record carries shared_volume_name when the volume is borrowed.
• Golden actors never take a ref; they warm up against an empty directory at the same mount path (new atelet empty_dir volume source). Actors created from such a template cold-boot on their first resume and restore their own snapshot afterwards, because runsc derives the disable_file_handle_sharing mount option from the bind source's filesystem and refuses to restore a golden checkpoint taken over a local directory onto an NFS mount. Details and candidate fixes are in the docs; tracked by TODO(shared-volumes).

Commits

1. api: nest the external volume template under Volume.external is Volume templates nest the external volume template under external.template #2653.
2. atelet: add the empty_dir volume source is the golden actor's placeholder mount.
3. ateapi: add the SharedVolume resource adds the resource, store, RPCs and CLI. Nothing references a volume yet.
4. ateapi: let actors reference SharedVolumes adds ExternalVolumeSource.shared_volume as the union's second member (inside the existing ExternalVolumes preview gate), refs, bindings, the RWO holder check, and the e2e hand-off test.
5. ateapi: cold-boot actors whose template references a SharedVolume is the first-resume behaviour described above.

Commits 2 to 5 are additive: no existing field, message or RPC changes shape.

Testing

• Unit, store (Postgres), controlapi and functional suites pass on every commit of the branch; hack/verify/gofmt.sh, hack/verify/boilerplate.sh and the apitool checks pass.
• New e2e: TestSharedVolumeHandoff and TestSharedVolumeMissing in internal/e2e/suites/combinedvolumes.
• Manually validated end to end on a kind cluster with both csi-nfs-sc and csi-hostpath-sc (under the previous names): provisioning and registration, create-time rejections, named and per-actor bindings, the holder check, ref tracking across resume, pause, suspend and crash, delete guards, both delete triggers, and golden actors staying out of refs.

Known follow-ups

• A crashed or failed-restore actor never unmounts its external volume on the node, which can make the next holder's unpublish fail (Actor crash leaves external volumes attached to the node; delete then skips detach #1715 covers the same failure for owned volumes).
• Golden-snapshot restores for templates that reference a SharedVolume (see docs for the two candidate approaches).
• READ_WRITE_MANY claims, a dedicated refs table, and safe unstaging (Milestone 2).

🤖 Generated with Claude Code

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[Salah (spichen)]

Thanks for moving this forward!

I noticed the Golden Snapshot limitation mentioned in this PR, particularly the mount-option mismatch when restoring an actor with an external NFS volume.

We're building a multi-tenant agent execution platform where each session needs actor-specific external mounts (read-only skills/knowledgebase files and writable persistent memory), with a target of sub-second sandbox startup using microVMs.

We'd be interested in contributing Golden Snapshot compatibility for shared volumes, including the necessary runtime changes and end-to-end benchmarks.

I see this is already tracked as TODO(shared-volumes). Would you be comfortable with us taking on that follow-up as a separate PR?

We'd initially focus on Cloud Hypervisor/virtio-fs, while ensuring the design doesn't prevent subsequent gVisor support.

We're also interested in contributing per-actor subPath and readOnly mount support, potentially adapting the implementation from Giant Swarm PR #234 to the new SharedVolume abstraction.

Would you prefer these as two separate follow-up PRs?

[Julian Gutierrez Oschmann (juli4n)]

No need to reserve ids or field names for now.