[haarchri]

Description of your changes

Design: #432

The serving stack installed NVIDIA's device stack on every cluster and no other vendor's anywhere: every component list pinned the NVIDIA operator or DRA driver unconditionally, nothing checked that the devices a class declares match the stack its cluster gets, and a pool without a GPU couldn't be provisioned at all.

This implements the multi-accelerator design. The cluster composition collects the DRA drivers claimed across a cluster's referenced classes (claim: DRA only; Synthetic devices describe placement, not a stack) onto ServingStack spec.draDrivers; components carry the drivers they serve as dra_drivers tags - hand-written in place, fail-closed through ALLOW for the generated clouds and the join installs a tagged component only when one of its drivers is claimed. The supported-drivers table is derived from the tags (plus EKS's declared-external dra.net), mirrored by hand in
compose-inference-cluster, which gates ClusterReady=False / UnsupportedDevices before anything is provisioned.

On top of the mechanism:

• CPU inference (Docs: Add design for CPU inference class support #509, rebased onto the mechanism): dra-driver-cpu is one chart tagged dra.cpu, pinned once in common.py for every cloud including Existing, with scoped tolerations, accelerator becomes optional in every provisioning block; a class without one composes a CPU-role pool with the modelplane.ai/cpu label and taint on all six clouds status.gpuPools becomes status.schedulablePools: nothing in it was GPU-specific.

• AMD on BYO clusters: the AMD GPU operator (DRA mode, driver management off) is pinned on the Existing list tagged gpu.amd.com - the pin validated on 8x MI325X bare metal in Provision Vultr bare metal servers into k3s inference clusters, composing multi-vendor GPU serving stacks #434, which rebases onto this mechanism. NFD carries both GPU drivers' tags there; engine pods tolerate the per-driver taint list (the pool nodeSelector is what places them).

• The cloud e2e legs shape one KWOK pool per claimed device stack, an NVIDIA and a CPU pool on every leg, an AMD pool on Existing and stamp spec.draDrivers, so the placement and toleration audits now check the filter itself: each stack must reach its own pool and stay off the others

• AMD GPU metrics: the collector's GPU job also discovers the AMD operator's device-metrics-exporter (label metrics-exporter, port exporter-port), and a modelplane-gpu-amd builtin maps its series onto the same modelplane_gpu_* names - VRAM MiB to bytes, activity percent to ratio, and energy through a new Microjoules fromUnit. modelplane_gpu_tensor_active_ratio stays NVIDIA-only.

• Two recipes: Qwen2.5-0.5B on a single MI325X on a bring-your-own cluster and Qwen2.5-0.5B on CPUs alone on a Civo g4p.kube.medium pool with vLLM's CPU image.

The recipe site needs two follow-ups in its repo: the gpus data key the ayouts read is now accelerators, and partials/icons.html needs brand-amd, brand-civo and cpu symbols. https://github.com/modelplaneai/docs-site/blob/main/themes/geekboot/layouts/partials/icons.html -> modelplaneai/docs-site#5

I have:

• Read and followed Modelplane's contribution process.
• Run nix flake check (or ./nix.sh flake check) and made sure it passes.
• Added or updated tests covering any composition function changes.
• Signed off every commit with git commit -s.

Tested

AMD

curl -v "http://207.148.15.31/v1/chat/completions" 
-H "Content-Type: application/json" 
-d "{"model":"ml-team/qwen","messages":[{"role":"user","content":"Say Hello from VULTR BYO with AMD GPU"}],"max_tokens":100}"

{"id":"chatcmpl-4cbd10fe-697f-4712-b24b-4edf39dfa069","object":"chat.completion","created":1790793716,"model":"ml-team/qwen-demo","choices":[{"index":0,"message":{"role":"assistant","content":"Hello! It looks like you're looking to set up a machine learning model using a bare metal K3s instance on the VULTR server. Here's how you can do it:\n\n1. Install VULTR: First, ensure that your VULTR account is active and connected.\n\n2. Create an Instance: Use the AWS Management Console or the AWS CLI to create a new EC2 instance. Choose the appropriate AMI for your needs (e.g., Ubuntu Server 2","refusal","annotations","audio","function_call","tool_calls":[],"reasoning"},"logprobs","finish_reason":"length","stop_reason","token_ids","routed_experts"}],"service_tier","system_fingerprint":"vllm-0.23.1.dev1+g9ddef7117.d20260901-ebed204f","usage":{"prompt_tokens":46,"total_tokens":146,"completion_tokens":100,"prompt_tokens_details"},"prompt_logprobs","prompt_token_ids","prompt_text","kv_transfer_params"}pod "curl-test" deleted from default namespace

kubectl get pods -A                                                                    
NAMESPACE                 NAME                                                              READY   STATUS    RESTARTS      AGE
[...]
kube-amd-gpu              default-dra-driver-wsxv2                                          1/1     Running   0             9m56s
kube-amd-gpu              default-metrics-exporter-42kht                                    1/1     Running   0             9m56s
kube-amd-gpu              default-node-labeller-bs7cx                                       1/1     Running   0             12m
kube-amd-gpu              mp-gpu-operator-charts-controller-manager-58c4f8969f-lztzb        1/1     Running   0             22m
kube-amd-gpu              mp-gpu-operator-charts-kmm-controller-889d84758-4m5wd             1/1     Running   0             22m
kube-amd-gpu              mp-gpu-operator-charts-kmm-webhook-server-78b4c5944f-pmkk5        1/1     Running   0             22m
[...]
mp-ml-team-51733          qwen-demo-8f4f6-epp-6fb7699648-mjxfw                              1/1     Running   0             18m
mp-ml-team-51733          qwen-demo-8f4f6-qwen-1d9a7-544587cb94-cv6t7                       1/1     Running   0             

GPU

kubectl get pods -A                                                                      
NAMESPACE                 NAME                                                              READY   STATUS    RESTARTS      AGE
[...]
mp-ml-team-51733          qwen-cpu-5205d-epp-554d95bdf8-xqs87                               1/1     Running   0             4m28s
mp-ml-team-51733          qwen-cpu-5205d-qwen-becae-7d897dd7cc-qfv88                        1/1     Running   0             4m28s

kubectl get resourceslice -A
NAME                                                              NODE                                              DRIVER    POOL                                              AGE
00000-dra.cpu-k3s-civo-chris-2439-383cf1-node-pool-636a-gjrnc4r   k3s-civo-chris-2439-383cf1-node-pool-636a-gjgqv   dra.cpu   k3s-civo-chris-2439-383cf1-node-pool-636a-gjgqv   11m
(⎈|upbound:default)➜  Downloads KUBECONFIG=/Users/haarchri/Downloads/civo-civo-chris-kubeconfig kubectl get resourceslice -A -o yaml
apiVersion: v1
items:
- apiVersion: resource.k8s.io/v1
  kind: ResourceSlice
  metadata:
    creationTimestamp: "2026-10-27T16:50:06Z"
    generateName: 00000-dra.cpu-k3s-civo-chris-2439-383cf1-node-pool-636a-gjgqv-
    generation: 1
    name: 00000-dra.cpu-k3s-civo-chris-2439-383cf1-node-pool-636a-gjrnc4r
    ownerReferences:
    - apiVersion: v1
      controller: true
      kind: Node
      name: k3s-civo-chris-2439-383cf1-node-pool-636a-gjgqv
      uid: 266dde53-7f07-4da1-9f57-b85117f500f6
    resourceVersion: "7469"
    uid: 0c642aed-846a-4e38-8b24-4b9bf8a959b4
  spec:
    devices:
    - allowMultipleAllocations: true
      attributes:
        dra.cpu/numCPUs:
          int: 6
        dra.cpu/numaNodeID:
          int: 0
        dra.cpu/smtEnabled:
          bool: false
        dra.cpu/socketID:
          int: 0
        dra.net/numaNode:
          int: 0
        resource.kubernetes.io/numaNode:
          int: 0
      capacity:
        dra.cpu/cpu:
          value: "6"
      name: cpudevnuma000
    driver: dra.cpu
    nodeName: k3s-civo-chris-2439-383cf1-node-pool-636a-gjgqv
    pool:
      generation: 1
      name: k3s-civo-chris-2439-383cf1-node-pool-636a-gjgqv
      resourceSliceCount: 1
kind: List
metadata:
  resourceVersion: ""

 kubectl run curl --rm -it --image=curlimages/curl --restart=Never --   curl -s http://212.2.244.212/v1/completions -H 'Content-Type: application/json'   -d '{"model": "ml-team/qwen-cpu", "prompt": "Hello", "max_tokens": 16}'

{"id":"cmpl-a992b1ac-241a-4772-8810-790b0ffc6b45","object":"text_completion","created":1793121273,"model":"ml-team/qwen-cpu","choices":[{"index":0,"text":"\n\nI'm currently working on a project that involves creating an interface for a game","logprobs":null,"finish_reason":"length","stop_reason":null,"token_ids":null,"prompt_logprobs":null,"prompt_token_ids":null,"routed_experts":null}],"service_tier":null,"system_fingerprint":"vllm-0.32.0-006d02cd","usage":{"prompt_tokens":1,"total_tokens":17,"completion_tokens":16,"prompt_tokens_details":null,"completion_tokens_details":null},"kv_transfer_params":null,"ec_transfer_params":null,"metrics":null}

[github-actions]

Docs preview: https://modelplane-docs-pr-556.vercel.app (ready once the site's Content workflow finishes)

[dennis-upbound]

Thanks, the mechanism reads well and I checked a lot of it: the tags on every cloud (neutral components untagged, every NVIDIA-only one tagged), the dependency check, the mirror against supported_drivers() for all six clouds, stacks-current regenerating the aicr files byte for byte, the dra-driver-cpu 0.3.0 chart and its dra.cpu capacity key, and the AMD exporter's names, units and discovery labels against device-metrics-exporter v1.5.1. A pre-#556 ServingStack also keeps its full stack until the cluster composition stamps draDrivers, which is the right upgrade behavior.

One problem deletes running things, so I'd fix it before merge: the UnsupportedDevices gate deletes a live cluster (inline on compose-inference-cluster). I've withdrawn the gpuPools rename comment: it only matters on upgrade, which is fine before v1.

The rest are inline: the CPU role coming from a missing accelerator, the AMD operator values, the energy metric's type, the docs-site ordering, and what the two recipes and e2e legs actually validated. Also, the four commits use typed prefixes and have no bodies, and update for vale is a fixup; CONTRIBUTING asks for problem-first messages without prefixes.

[negz]

Approving, but a fair chunk of this is mine now, so please review d36e26c0..4693b4b9 before merging. They're separate commits so the delta is easy to read; fold them into yours once you're happy.

The main changes:

• The supported-drivers gate no longer fails open. It names every offending class, and a new flake check catches its table drifting from the serving stack's.
• A provisioned class can't claim dra.cpu alongside a GPU driver, because that claim could never allocate on a GPU pool.
• A dra.cpu claim across more than one device is rejected, because each device got the full CPU request. Please check I read that right.
• The AMD metrics exporter image is pinned to v1.5.1 instead of latest.
• CPU taint constants are consistent across clouds, the docs and stale comments are updated, and the AMD recipe is now qwen2.5-0.5b-amd.

Two follow-ups for issues: enforce the 1.36 minimum for CPU pools, and decide whether hard-coding dra.cpu in the InferenceClass validation is OK given the design doc.

I'll test the CPU path end to end, but I don't have access to AMD GPUs, so I can't test that side.

[negz]

Tested the CPU path end to end on EKS, using the PR head plus my commits (4693b4b9), with an m6i.2xlarge CPU pool and the CPU recipe's ModelDeployment.

• Both new InferenceClass validation rules reject bad classes at admission.
• A class claiming gpu.amd.com gives UnsupportedDevices ("class eks-amd claims gpu.amd.com; EKS clusters support dra.cpu, dra.net, gpu.nvidia.com"), and nothing is provisioned.
• The ServingStack gets draDrivers: [dra.cpu]. Only dra-driver-cpu installs, with no NVIDIA components.
• The CPU node group gets the standard AL2023 image, Kubernetes 1.36.5, and the modelplane.ai/cpu label and taint.
• The ResourceSlice publishes 6 claimable CPUs with SMT on. The engine's claim consumed dra.cpu/cpu: "5".
• A chat completion through the gateway returned a sensible answer.