Skip to content

One node pool that can't provision blocks the serving stack for the whole InferenceCluster #580

Open
Open
One node pool that can't provision blocks the serving stack for the whole InferenceCluster#580
Labels
bugSomething isn't working

Description

@negz
negz
opened on Oct 26, 2026

What happened?

One EKS GPU pool that couldn't get instances (InsufficientInstanceCapacity) blocked the whole InferenceCluster until I removed the pool. The serving stack, including the cluster agent and gateway, didn't install on any pool, so no ModelDeployment could schedule there.

The InferenceCluster composes the serving stack only once its EKSCluster is Ready, and the EKSCluster is only Ready once every NodeGroup is:

modelplane/functions/compose-inference-cluster/function/fn.py

Lines 305 to 326 in b6b28f0

self.compose_eks_cluster(eks)
eks_ready = resource.get_condition(self.req.observed.resources.get("eks-cluster"), "Ready").status == "True"
kubeconfig = self.observed_eks_secret(_SECRET_TYPE_KUBECONFIG)
backend_exists = BACKEND_RESOURCE_KEY in self.req.observed.resources
if eks_ready and kubeconfig:
self.compose_cluster_provider_config(kubeconfig.name, kubeconfig.key)
backend_secrets = self.resolve_eks_backend_secrets(eks_ready=eks_ready, backend_exists=backend_exists)
if backend_secrets or backend_exists:
if backend_secrets:
self.compose_serving_stack(backend_secrets)
self.compose_eks_usage()
if eks_ready:
self.rsp.desired.resources["eks-cluster"].ready = fnv1.READY_TRUE
if not backend_exists:
response.normal(self.rsp, "EKS cluster ready, composing backend")
self.write_status(self.gpu_pools())
self.derive_conditions(cluster_ready=eks_ready)

I only saw this on EKS. The other clouds' paths in the same file look similar.

How can we reproduce it?

  1. Create an EKS InferenceCluster with two GPU pools, one of an instance type that's out of capacity in its zone.
  2. The InferenceCluster stays ClusterReady=False Provisioning with no serving stack, though the other pool's nodes are Ready.

What environment did it happen in?

Modelplane version: main plus #542 (e574dcf)

  • Crossplane v2.4.0
  • provider-aws-eks v2.8.1
  • EKS 1.36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions