Skip to content

ateapi: check mounted volumes are attachable and test assigned_node detach on crash - #2611

Open
Vu Le (lekhoavu2909) wants to merge 2 commits into
agent-substrate:mainfrom
lekhoavu2909:volume-assigned-node-detach
Open

Vu Le (lekhoavu2909) wants to merge 2 commits into
agent-substrate:mainfrom
lekhoavu2909:volume-assigned-node-detach

Conversation

@lekhoavu2909

@lekhoavu2909 Vu Le (lekhoavu2909) commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Some storage backends (e.g. Cloud Filestore IP export rules) restrict a volume to the node it is published to via CSI ControllerPublishVolume / ControllerUnpublishVolume, so every publish must be matched by an unpublish to the same node. Since #2222 , detach targets ActorStatus.assigned_node, which survives crashes and worker loss. This PR adds checks before attach and test coverage for the paths where detach used to be skipped.
Follows #2094, which covers the publish side.

  • ateapi: check mounted volumes are attachable before publishing
  • test(ateapi): cover volume detach via assigned_node after crash or worker loss

Changes

  • Worker without a node name: assignWorkerAttempt rejects such a worker with when the actor mounts external volumes. Previously the resume failed in ensureVolumesAttached, after the worker was booked.

  • Volume checks before attach: ensureVolumesAttached checks that every mounted volume is STATUS_CREATED and has a storage_volume_id before any attach call, so one bad volume can't leave earlier volumes published.

  • Unit tests, functional tests

  • E2E: assigned_node equals the worker's node_name while RUNNING and is cleared after suspend and revert. The OSS NFS driver doesn't implement ControllerPublishVolume.

  • Tests pass

@lekhoavu2909
Vu Le (lekhoavu2909) marked this pull request as ready for review October 13, 2026 21:45
@lekhoavu2909 Vu Le (lekhoavu2909) self-assigned this Oct 13, 2026
ref := &ateapipb.ObjectRef{Atespace: actor.GetMetadata().GetAtespace(), Name: actor.GetMetadata().GetName()}
return getMountedActorVolumes(ctx, ref, actor.GetStatus().GetExternalVolumes(), tmpl)
}

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: generally helper functions improve readability, but this is sort of a wrapper for another helper function. I'd recommend keeping it as it was

Comment on lines +679 to +688
func checkVolumesAttachable(volumes []*ateapipb.ExternalVolume) error {
for _, vol := range volumes {
switch {
case vol.GetStatus() != ateapipb.ExternalVolume_STATUS_CREATED:
return apierror.FailedPrecondition("volume %q is %s, want %s before it is attached", vol.GetName(), vol.GetStatus(), ateapipb.ExternalVolume_STATUS_CREATED)
case vol.GetStorageVolumeId() == "":
return apierror.FailedPrecondition("volume %q has no storage volume ID to attach", vol.GetName())
}
}
return nil

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are actually getting rid of volume status in #2456. I don't think you need to rebase on the change, just use the StorageVolumeID != "" check.

@lekhoavu2909
Vu Le (lekhoavu2909) force-pushed the volume-assigned-node-detach branch from a88d3b4 to 47d2b5c Compare October 14, 2026 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants