Repository navigation
Back node pools with CapacityReservations instead of capacityBlock #600
+2,341
−305
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: apiextensions.crossplane.io/v1 | ||
| kind: Composition | ||
| metadata: | ||
| name: capacityreservations.modelplane.ai | ||
| spec: | ||
| compositeTypeRef: | ||
| apiVersion: modelplane.ai/v1alpha1 | ||
| kind: CapacityReservation | ||
| mode: Pipeline | ||
| pipeline: | ||
| - functionRef: | ||
| name: modelplane-modelplanecompose-capacity-reservation | ||
| step: compose-capacity-reservation |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,111 @@ | ||
| apiVersion: apiextensions.crossplane.io/v2 | ||
| kind: CompositeResourceDefinition | ||
| metadata: | ||
| name: capacityreservations.modelplane.ai | ||
| spec: | ||
| group: modelplane.ai | ||
| names: | ||
| categories: [crossplane, modelplane, platform] | ||
| kind: CapacityReservation | ||
| plural: capacityreservations | ||
| scope: Cluster | ||
| versions: | ||
| - name: v1alpha1 | ||
| served: true | ||
| referenceable: true | ||
| additionalPrinterColumns: | ||
| - name: PROVIDER | ||
| type: string | ||
| jsonPath: .spec.provider | ||
| schema: | ||
| openAPIV3Schema: | ||
| type: object | ||
| description: >- | ||
| An existing cloud capacity reservation that InferenceCluster node | ||
| pools can reference by name. Modelplane doesn't create or modify the | ||
| reservation; it only directs node pools to consume it. | ||
| required: [spec] | ||
| properties: | ||
| spec: | ||
| type: object | ||
| required: [provider] | ||
| x-kubernetes-validations: | ||
| - rule: "self.provider != 'GKE' || has(self.gke)" | ||
| message: spec.gke is required when spec.provider is GKE. | ||
| - rule: "self.provider != 'EKS' || has(self.eks)" | ||
| message: spec.eks is required when spec.provider is EKS. | ||
| properties: | ||
| provider: | ||
| type: string | ||
| description: >- | ||
| The cloud the reservation belongs to. A node pool can only | ||
| reference a reservation whose provider matches its cluster's | ||
| source. | ||
| enum: [GKE, EKS] | ||
| gke: | ||
| type: object | ||
| description: >- | ||
| A Compute Engine reservation, consumed as a specific | ||
| reservation. Any reservation type works, including future | ||
| reservations in calendar mode once their reservation period | ||
| starts. Required when provider is GKE. | ||
| required: [zone, name] | ||
| properties: | ||
| zone: | ||
| type: string | ||
| description: >- | ||
| Zone the reservation is in (e.g. us-central1-a). A node | ||
| pool backed by the reservation must be restricted to | ||
| exactly this zone. | ||
| minLength: 1 | ||
| maxLength: 63 | ||
| name: | ||
| type: string | ||
| description: Name of the reservation. | ||
| pattern: '^[a-z]([-a-z0-9]{0,61}[a-z0-9])?$' | ||
| maxLength: 63 | ||
| project: | ||
| type: string | ||
| description: >- | ||
| Project that owns the reservation, for a reservation | ||
| shared from another project. The reservation must be | ||
| shared with the cluster's project, and the cluster | ||
| project's GKE service agent must be allowed to list | ||
| reservations in this project. Omit for a reservation | ||
| in the cluster's own project. | ||
| pattern: '^[a-z][-a-z0-9]{4,28}[a-z0-9]$' | ||
| maxLength: 30 | ||
| eks: | ||
| type: object | ||
| description: >- | ||
| An EC2 Capacity Reservation. Required when provider is EKS. | ||
| required: [region, type, id] | ||
| properties: | ||
| region: | ||
| type: string | ||
| description: AWS region the reservation is in (e.g. us-east-2). | ||
| minLength: 1 | ||
| maxLength: 32 | ||
| type: | ||
| type: string | ||
| description: >- | ||
| The kind of Capacity Reservation. CapacityBlock is a | ||
| Capacity Block for ML. OnDemand is an On-Demand | ||
| Capacity Reservation. | ||
| enum: [CapacityBlock, OnDemand] | ||
| id: | ||
| type: string | ||
| description: >- | ||
| ID of the Capacity Reservation (e.g. | ||
| cr-0123456789abcdef0). Purchasing a Capacity Block | ||
| yields this ID. | ||
| pattern: '^cr-[0-9a-f]+$' | ||
| minLength: 4 | ||
| maxLength: 64 | ||
| status: | ||
| type: object | ||
| properties: | ||
| conditions: | ||
| type: array | ||
| items: | ||
| type: object | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -129,6 +129,32 @@ the rank a gang computes until then. | |
| Composing a full Dynamo graph deployment, for its frontend and request routing, | ||
| is planned. | ||
|
|
||
| ## Capacity reservations | ||
|
|
||
| Large GPU machines are often only available through a capacity reservation. On | ||
| GKE and EKS a node pool can consume one you've already bought. Register the | ||
| reservation as a `CapacityReservation`, then reference it from the pool's | ||
| `reservationRef`: | ||
|
|
||
| {{< manifests path="concepts/capacity-reservation-gke.yaml" apply="false" >}} | ||
|
|
||
| A pool that references a reservation only gets nodes from that reservation, and | ||
| never falls back to on-demand capacity. Keep its `nodeCount` within the | ||
| reserved instance count. On GKE, restrict the pool's `zones` to the | ||
| reservation's zone. On EKS, restrict them to the reservation's Availability | ||
| Zone. | ||
|
|
||
| A `CapacityReservation` describes any GKE reservation, including a future | ||
| reservation in calendar mode once its reservation period starts. On EKS it | ||
| describes a Capacity Block for ML (`type: CapacityBlock`) or an On-Demand | ||
| Capacity Reservation (`type: OnDemand`). Modelplane doesn't create, change, or | ||
| delete the reservation itself. | ||
|
|
||
| When a reservation ends, the cloud removes the nodes that consume it. To move | ||
|
Collaborator
There was a problem hiding this comment. This holds for Capacity Blocks and GKE calendar-mode reservations. It doesn't hold for On-Demand Capacity Reservations: when one ends, its instances keep running as on-demand instances (AWS docs). Compute Engine won't delete a specific reservation that VMs still consume (GCP docs). Maybe "When a Capacity Block or a calendar-mode reservation ends, the cloud removes the nodes that consume it." |
||
| to a new reservation, add a node pool that references it, ideally before the | ||
| old reservation ends, then remove the old pool. Don't change the reservation | ||
|
Collaborator
There was a problem hiding this comment. We can enforce this instead of documenting it. x-kubernetes-validations:
- rule: "!has(oldSelf.reservationRef) || (has(self.reservationRef) && self.reservationRef.name == oldSelf.reservationRef.name)"
message: A node pool's reservationRef can't be changed or removed. Add a new pool instead.The same change can also come in through the |
||
| an existing pool references. | ||
|
|
||
| ## Examples | ||
|
|
||
| {{< tabs >}} | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| # A GKE InferenceCluster whose L4 pool consumes a Compute Engine reservation. | ||
| # The pool is restricted to the reservation's zone. | ||
| apiVersion: modelplane.ai/v1alpha1 | ||
| kind: CapacityReservation | ||
| metadata: | ||
| name: l4-reservation | ||
| spec: | ||
| provider: GKE | ||
| gke: | ||
| zone: us-central1-a | ||
| name: l4-reservation | ||
| # project: shared-reservations # for a reservation shared from another project | ||
| --- | ||
| apiVersion: modelplane.ai/v1alpha1 | ||
| kind: InferenceCluster | ||
| metadata: | ||
| name: gke-reserved | ||
| spec: | ||
| cluster: | ||
| source: GKE | ||
| gke: | ||
| region: us-central1 | ||
| nodePools: | ||
| - name: l4-pool | ||
| className: gke-l4-1x-g2 | ||
| nodeCount: 2 | ||
| maxNodeCount: 2 | ||
| zones: [us-central1-a] | ||
| reservationRef: | ||
| name: l4-reservation |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider making spec immutable (
self == oldSelf). The docs say “don’t change the reservation an existing pool references”, but nothing stops someone from editing the CapacityReservation itself. That would change the pools of every cluster that references it at once....