[dennis-upbound]

Description of your changes

Fixes #497.

#562 made an engine's workload Object wait for a ready pod or group per copy, and watched it, which covers the maxSurge: 0 case in the issue and the stale status in its comments. A Deployment could still read ready partway through a rolling update. readyReplicas also counts pods from the previous ReplicaSet, so with the default maxSurge a one-copy engine kept its old pod ready while the new pod started, and the replica, endpoint and deployment all reported Ready though no copy ran the current spec. The query could also pass on the status provider-kubernetes reads back when it applies a spec change, before the Deployment controller has processed that change.

The Deployment's readiness query now also requires status.observedGeneration to match metadata.generation, updatedReplicas to reach spec.replicas, and status.replicas to equal updatedReplicas. These are the checks kubectl rollout status makes. During a rollout the replica keeps serving from its old copies and reports ModelReady=False with reason RollingOut, so ModelServing doesn't drop. Its ModelDeployment reports ReplicasServing=True and Ready=False until the rollout finishes, with reason Degraded, as it already did for a LeaderWorkerSet rollout. The status docs now say a serving deployment that isn't Ready may be rolling out.

The LeaderWorkerSet query doesn't change. lws v0.8.0 sets Available only once every group is ready and runs the current template, and sets it False while it replaces a group (controller). lws doesn't publish an observedGeneration. Grove's queries don't change either: RollingRecreate deletes a copy before it recreates it, so the scaling group's availableReplicas drops during the update.

The shared Available query now checks that status exists, as the other functions' queries do. Without that check cel-go fails with no such key: status on a workload that has no status yet, where the query should return false.

I evaluated the queries with cel-go v0.30.0 in the environment provider-kubernetes v1.4.1 uses, against Deployment and LeaderWorkerSet status for steady state, a surge rollout, a maxSurge: 0 rollout, a fresh create, N-1 of N copies ready, a stale observedGeneration, a stalled surge, and a strategy-only change. The old query passed on the surge rollout, on a two-copy maxSurge: 0 rollout between copies, and on the stale generation. The new one passes only in steady state and while scaling down.

On upgrade the readiness query changes on every engine's and endpoint picker's Object, and no pod restarts. A workload in steady state passes the old and new queries. A Ready replica doesn't change. A replica whose rollout has stalled with its old pod serving, such as a surge pod that can't get a device on a full pool, turns Ready=False with reason RollingOut and keeps serving.

#572 changes the same Deployment's strategy. This PR and #572 merge cleanly, and the merged tests pass. A strategy-only change bumps the Deployment's generation without starting a new ReplicaSet, and this query reads not Ready until the Deployment controller observes it. #572 changes the strategy when copies go down, while the replica is already not Ready, and again when they come back. That second change can turn a replica that just became Ready back to RollingOut for one status update from the Deployment controller. ModelServing doesn't drop.

Decisions:

• The ModelDeployment keeps reason Degraded during a rollout. Its Ready reports full capacity across replicas, and a replica mid-rollout isn't at full capacity. Each replica's own ModelReady says RollingOut. A deployment-level RollingOut reason can follow once the deployment aggregates its replicas' reasons, as ModelDeployment status doesn't say why a model isn't serving #498 designs.
• A stalled replica that still serves turns Ready=False on upgrade by design. ModelDeployment and ModelEndpoint report Ready while the engine has no ready pods #497 asks that Ready mean every copy runs the current spec. ModelServing doesn't change, and neither does routing.
• A LeaderWorkerSet's status read back before the lws controller processes a template change can still pass the query for a moment. This is a known limitation. Closing it needs lws to publish an observedGeneration.

I have:

• Read and followed Modelplane's contribution process.
• Run nix flake check (or ./nix.sh flake check) and made sure it passes.
• Added or updated tests covering any composition function changes.
• Signed off every commit with git commit -s.

🤖 Generated with Claude Code

[github-actions]

Docs preview: https://modelplane-docs-pr-570.vercel.app (ready once the site's Content workflow finishes)