Repository navigation
Support both the old and new recipes.yaml schema - #5
Merged
Merged
Conversation
…#556 Signed-off-by: Christopher Haar <christopher.haar@upbound.io>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
4 tasks done
tr0njavolta
approved these changes
Oct 29, 2026
negz
pushed a commit
to haarchri/modelplane
that referenced
this pull request
Oct 30, 2026
The recipes covered only NVIDIA GPUs. These add Qwen2.5-0.5B-Instruct on one AMD Instinct MI325X on a bring-your-own cluster, and on CPUs alone on a Civo pool with vLLM's CPU image. The recipe data's gpus list becomes accelerators so it can carry AMD and CPU entries; the docs site reads both (modelplaneai/docs-site#5). Signed-off-by: Christopher Haar <christopher.haar@upbound.io> Signed-off-by: Nic Cope <nicc@rk0n.org>
haarchri
added a commit
to modelplaneai/modelplane
that referenced
this pull request
Oct 30, 2026
…U pools and AMD serving (#556) * Install each device stack only where its driver is claimed Every cluster got NVIDIA's device stack and no other vendor's. Each cloud's component list pinned the NVIDIA GPU operator or DRA driver unconditionally, nothing checked that the devices a class declares match the stack its cluster gets, and a pool without a GPU couldn't be provisioned at all. compose-inference-cluster now collects the DRA drivers its classes claim onto ServingStack spec.draDrivers, and compose-serving-stack installs a component tagged with drivers only when one of them is listed. A class claiming a driver its cluster's cloud can't serve sets ClusterReady=False with reason UnsupportedDevices. The gate is a fatal result rather than an empty desired state, which Crossplane would answer by garbage collecting a running cluster. This also adds CPU inference and AMD GPUs. A class without an accelerator gets a CPU pool served by dra-driver-cpu, on every cloud. On Existing, a class claiming gpu.amd.com gets the AMD GPU operator in DRA mode, and the collector maps its device-metrics-exporter onto the same modelplane_* names as DCGM. Designed in #432. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> Signed-off-by: Nic Cope <nicc@rk0n.org> * Regenerate schemas and aicr stacks Signed-off-by: Christopher Haar <christopher.haar@upbound.io> Signed-off-by: Nic Cope <nicc@rk0n.org> * Fake a pool per device stack in the cloud e2e legs The cloud legs faked one NVIDIA GPU pool, so nothing checked that the serving stack's driver filter puts each device stack on its own pool and keeps it off the others. Each leg now fakes an NVIDIA and a CPU pool, plus an AMD pool on Existing, and stamps spec.draDrivers the way compose-inference-cluster would. Those legs claim every stack their cloud carries, so a join that ignored the filter would pass them all. A new existing-filtered leg keeps the AMD pool but leaves gpu.amd.com unclaimed, and checks the AMD operands don't exist. Signed-off-by: Christopher Haar <christopher.haar@upbound.io> Signed-off-by: Nic Cope <nicc@rk0n.org> * Add Qwen2.5-0.5B recipes for an AMD GPU and for CPUs The recipes covered only NVIDIA GPUs. These add Qwen2.5-0.5B-Instruct on one AMD Instinct MI325X on a bring-your-own cluster, and on CPUs alone on a Civo pool with vLLM's CPU image. The recipe data's gpus list becomes accelerators so it can carry AMD and CPU entries; the docs site reads both (modelplaneai/docs-site#5). Signed-off-by: Christopher Haar <christopher.haar@upbound.io> Signed-off-by: Nic Cope <nicc@rk0n.org> * Observe only NVIDIA GPU pools with the cluster agent The cluster agent counts a node only if every NoSchedule taint on it is nvidia.com/gpu. CPU pools carry modelplane.ai/cpu and BYO AMD pools amd.com/gpu, so the agent reported their nodes as observed but left them out of its free-device histogram. The scheduler then saw those pools as full and never placed a replica on them. This composes a PoolObservation only for a pool whose class claims gpu.nvidia.com. Other pools go unobserved, and the scheduler falls back to whole-node accounting for them, as it did for every pool before the agent. Teaching the agent the other taints would let them use free-device accounting too, but needs an agent release. Signed-off-by: Nic Cope <nicc@rk0n.org> * Replace MetricMapping's cumulative boolean with a fromType enum MetricMapping gained a cumulative boolean for a component that reports a running total as a gauge. A boolean leaves no room for the next type mismatch, and reads apart from fromUnit, which already describes the same kind of correction for units. This replaces it with fromType, an enum whose only value is Gauge, described the way fromUnit is: what the component reports, when that isn't what the name claims. Because Gauge converts the series to a counter, a rule requires the mapping's to name to end in _total. Signed-off-by: Nic Cope <nicc@rk0n.org> * Note that CPU pools need Kubernetes 1.36 on GKE, AKS and Nebius A CPU pool's cores are claimed as consumable capacity, which Kubernetes enables by default only from 1.36. GKE defaults to 1.35, and AKS and Nebius to 1.34, so a CPU pool on their defaults fails when its engine pod's claim is prepared. Only Civo's version field said so. Signed-off-by: Nic Cope <nicc@rk0n.org> * Regenerate schemas Signed-off-by: Nic Cope <nicc@rk0n.org> * Rename the AMD Qwen2.5-0.5B recipe to qwen2.5-0.5b-amd The AMD recipe took the generic qwen2.5-0.5b slug while its CPU sibling is qwen2.5-0.5b-cpu, so an NVIDIA recipe for the same model would have had nowhere to go. Suffixing the AMD recipe matches the CPU one. Signed-off-by: Nic Cope <nicc@rk0n.org> * Check compose-inference-cluster's supported drivers against the stack's compose-inference-cluster rejects a class claiming a DRA driver its cloud can't serve, using a hand-kept copy of the serving stack's derived supported-drivers table. Nothing compared the two. A copy that admitted a driver the stack doesn't serve would skip the UnsupportedDevices condition, and the ServingStack's join would raise on every reconcile instead. This adds a supported-drivers-mirror flake check. It imports the stacks package from compose-serving-stack's venv and reads the copy from compose-inference-cluster's source, since neither function can import the other, and fails naming each cloud whose drivers differ. Signed-off-by: Nic Cope <nicc@rk0n.org> * Name every unsupported class, and gate only Existing out check_devices_supported skipped any cloud missing from _SUPPORTED_DRA_DRIVERS, which is how Existing opted out, so a cloud added without an entry would never be gated. Its UnsupportedDevices message also stopped at the first offending class, and said the serving stack installs every supported driver, which on EKS includes dra.net, installed by compose-eks-cluster instead. This skips Existing explicitly and indexes the table for every other cloud, names every offending class in one message, and says which drivers the cloud supports rather than what its stack installs. It also replaces the "DRA" claim literals with a _CLAIM_DRA constant, as compose-model-deployment does. Signed-off-by: Nic Cope <nicc@rk0n.org> * Declare external DRA drivers in every cloud's module The generated cloud modules declare the drivers that arrive outside the serving stack as EXTERNAL_DRIVERS, while the stacks package hard-coded an empty set for each hand-written cloud. This declares EXTERNAL_DRIVERS in the hand-written modules too, so every cloud states its external drivers in one place. Signed-off-by: Nic Cope <nicc@rk0n.org> * Pin the AMD GPU operator's metrics exporter image The AMD GPU operator chart defaults the DeviceConfig's metrics exporter image to a floating latest tag. The DRA driver, node labeller and utility images were already pinned, but not the exporter, whose field names the modelplane-gpu-amd MetricMapping reads. This pins it to v1.5.1, the release that shipped with the v1.5.1 chart. Signed-off-by: Nic Cope <nicc@rk0n.org> * Reject provisioned classes that claim dra.cpu beside another driver dra-driver-cpu runs only on nodes labelled modelplane.ai/cpu, which the cluster compositions put only on CPU pools. A provisioned class claiming dra.cpu alongside a GPU driver gets a GPU pool, so its dra.cpu claim could never allocate, and nothing said why. This rejects that class at admission. A class without provisioning is unaffected, since its nodes are the user's to label. The descriptions of each cloud's machine shape still called the accelerator block informational, though whether it's set now decides whether a pool is provisioned as a GPU or a CPU pool. This corrects them, and the InferenceCluster nodePools description that called every pool a GPU pool. Signed-off-by: Nic Cope <nicc@rk0n.org> * Regenerate schemas Signed-off-by: Nic Cope <nicc@rk0n.org> * Reject a dra.cpu claim that spans more than one device compose-model-replica sizes a dra.cpu claim by stamping the engine container's cpu request on the claim as device capacity. It stamped it on every device in the request, so a member claiming two CPU devices took the full request from each, pinning twice the cores Kubernetes accounts the pod for. This fails the replica on a member whose dra.cpu requests total more than one device, alongside the existing failure for a dra.cpu claim with no cpu request, which both now report through invalid_cpu_claim. Signed-off-by: Nic Cope <nicc@rk0n.org> * Declare CPU pool taints like GPU pool taints The cluster compositions declare their GPU taint's key, value and effect as constants, but tainted CPU pools by reusing the CPU label constant as the key, an inline "true", and the GPU taint's effect. GKE inlined the effect too. This gives each composition _CPU_TAINT_KEY, _CPU_TAINT_VALUE and _CPU_TAINT_EFFECT, matching AKS's _CPU_TAINT. The e2e legs test now checks the CPU taint's value as well as its key, and the CPU label against each leg's fake nodes. Signed-off-by: Nic Cope <nicc@rk0n.org> * Correct serving stack comments the device stacks made stale Several comments still described the NVIDIA-only stack. The GPU collector job's comment said Modelplane installs no GPU exporter, which is no longer true on Existing. generate.py's toleration comment left out the NODE class. On Existing, the NFD comment said the cluster compositions taint GPU nodes, and the AMD operator comment called DRA mode the chart's default when the values set it against the defaults. This corrects them and rewraps the comments left ragged by earlier edits. Signed-off-by: Nic Cope <nicc@rk0n.org> * Update the docs for per-driver device stacks, CPU pools and AMD Several docs still described an NVIDIA-only stack. The existing-cluster requirements said Modelplane always installs NVIDIA's DRA driver and tolerates only the nvidia.com/gpu taint, and said nothing of the AMD and CPU stacks' needs. The telemetry page said an existing cluster brings every GPU exporter and that dashboards need no change across vendors, though per-card labels differ, and didn't document fromType. The design doc and the CPU recipe had drifted from the implementation. This brings them in line with the code, and rewraps a few paragraphs left ragged by earlier edits. Signed-off-by: Nic Cope <nicc@rk0n.org> --------- Signed-off-by: Christopher Haar <christopher.haar@upbound.io> Signed-off-by: Nic Cope <nicc@rk0n.org> Co-authored-by: Nic Cope <nicc@rk0n.org>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
modelplaneai/modelplane#556 updates
docs/data/recipes.yaml:gpus:becomesaccelerators:Since one branch of this repo builds every docs version, the templates must keep rendering the release branches that still ship the old schema.Changes
acceleratorswith a fallback togpus, guard the now-optionalmem, and render the accelerator icon when the entry carries one. The spec-row label becomes "Accelerator" under the new schema (the set includes CPU), and stays "GPU" under the old one.brand-amd,brand-civo(simple-icons, CC0) andcpu(Bootstrap Icons).Vendors, clouds, servingModes, and engines needed no template changes: the new entries are shape-compatible
Verified
Built both schemas locally via
CONTENT_REF=main ./build.shandCONTENT_REF=refs/pull/556/head ./build.sh: