Skip to content

Reject a model member without template.spec at admission - #577

Draft
dennis-upbound wants to merge 2 commits into
modelplaneai:mainfrom
dennis-upbound:dennis/reject-missing-member-spec
Draft

dennis-upbound wants to merge 2 commits into
modelplaneai:mainfrom
dennis-upbound:dennis/reject-missing-member-spec

Conversation

@dennis-upbound

@dennis-upbound dennis-upbound commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Description of your changes

Fixes #416.

A ModelDeployment whose member has template: {} passed admission, because neither the ModelDeployment nor the ModelReplica XRD required template.spec. compose-model-replica then failed an assert in base.engine_container while looking for the engine container, so the user got a function stack trace with nothing pointing at their manifest.

This adds required: [spec] to the member template in both XRDs. kubectl now rejects such a manifest with spec.template.spec.engines[0].members[0].template.spec: Required value. The regenerated models type spec as required, so the asserts and None checks on it in compose-model-replica and compose-model-deployment are gone.

A deployment or replica stored before this change can still lack the field, and the stricter models fail to parse it. Each function now catches that and returns a fatal result with the field's path, such as The ModelReplica doesn't match its schema: spec.engines[0].members[0].template.spec: Field required. compose-model-deployment reads every ModelReplica on the control plane to work out free capacity, so it skips a replica that doesn't parse with a warning instead of failing every deployment. The warning gives the replica's name only when it's in the deployment's own namespace.

This is a breaking change for a manifest without template.spec, which is now rejected. On main it only got as far as the crash.

I checked the other fields the backends read without a guard on the same path. The member template, a non-empty containers, exactly one container named engine (on Workers too), nodeSelector.devices, and the Leader/Worker member shape are already rejected at admission. A Worker without worker is admitted, but both gang backends default it to one node.

Decisions:

  • compose-model-deployment skips any ModelReplica that doesn't parse, as compose-serving-stack does for a MetricMapping. Failing would stop every deployment on the control plane over one bad replica. The replica's own fatal result reports the problem. The scheduler can't count nodes for a replica it can't read, so a skipped replica that's still running goes uncharged. A replica without template.spec never composed a workload, so this case doesn't use any nodes.
  • The skip warning shows on every deployment, so it gives the replica's name only when the replica is in the deployment's own namespace. A deployment's events don't name another tenant's objects.
  • The !has(self.template.spec) guards stay in the existing port CEL rules. With spec required they never hold, but Let engines serve on another port and report health on another path #573 rewrites those rules. Whichever of Let engines serve on another port and report health on another path #573 and this PR merges second removes them.

I tested the XRDs against a real kube-apiserver (envtest 1.34.1, each XRD's schema wrapped in a CRD) with kubectl apply --dry-run=server --validate=strict. All 29 ModelDeployment and ModelReplica manifests under docs/manifests and e2e/manifests pass. A member with template: {} (Standalone, Leader, and a Worker with only metadata) is rejected with Required value on both kinds. The same invalid manifests are admitted against main's XRDs.

I have:

  • Read and followed Modelplane's contribution process.
  • Run nix flake check (or ./nix.sh flake check) and made sure it passes.
  • Added or updated tests covering any composition function changes.
  • Signed off every commit with git commit -s.

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown

Docs preview: https://modelplane-docs-pr-577.vercel.app (ready once the site's Content workflow finishes)

Neither the ModelDeployment nor the ModelReplica XRD required a member's
template.spec, so a member written as template: {} was admitted.
compose-model-deployment passed it through to the ModelReplica, and
compose-model-replica then failed an assert when it looked for the
engine container. The user saw a function stack trace rather than an
error naming the field.

Both XRDs now require template.spec, so kubectl rejects such a member
and reports the field as a required value. The native, llm-d and Grove
backends all read the engine container from the spec, so a member
without one has no pod to compose. The regenerated models type spec as
required, which makes the asserts and None checks on it in
compose-model-replica and compose-model-deployment redundant, so they
go.

This tightens the API. A manifest without template.spec that main
admits is now rejected, though on main it never got further than the
crash.

Towards modelplaneai#416.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Dennis Ramdass <dennis@upbound.io>
@dennis-upbound
dennis-upbound force-pushed the dennis/reject-missing-member-spec branch from 0de207c to 5c654f6 Compare October 26, 2026 20:04
The API server checks an object against its XRD only when it's
written, so a ModelDeployment or ModelReplica stored before the XRD
required template.spec can still lack one. Parsing it with the
regenerated models raises a ValidationError, which the function didn't
handle.

Each function now returns a fatal result with the path of the missing
field, such as spec.engines[0].members[0].template.spec, and composes
nothing. Crossplane keeps the XR's existing resources and shows the
message in its Synced condition.

compose-model-deployment also reads every ModelReplica to account for
used capacity, so one replica that doesn't parse would have failed
every deployment's reconcile. It now skips that replica with a warning,
the same way compose-serving-stack skips a MetricMapping that doesn't
parse. The replica's own fatal result reports the problem. The
scheduler can't count nodes for a replica it can't read, so a skipped
replica that is still running goes uncharged. A replica without
template.spec never composed a workload, so it doesn't use any nodes.

The warning gives the replica's name only when it's in the deployment's
own namespace. Every deployment sees every replica, and a deployment's
events shouldn't name another tenant's objects.

Fixes modelplaneai#416.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Dennis Ramdass <dennis@upbound.io>
@dennis-upbound
dennis-upbound force-pushed the dennis/reject-missing-member-spec branch from 5c654f6 to 3960129 Compare October 26, 2026 20:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Omitting a member's template.spec crashes compose-model-replica instead of failing admission

1 participant