Repository navigation
Move WorkerPoolSyncer out of api-server #729
Description
WorkerPoolSyncer runs an informer to synchronize K8s pods to Substrate workers. This currently runs inside the ate-api-server, which is mixing responsibilities. We should move the WorkerPoolsyncer out of the ate-api-server and likely into the ate-controller, which is a better spot for an informer like this to run.
Activity
sfunkenhauser commentedon Aug 5, 2026
As part of this change, let's switch to using controller runtime to consolidate the reconciliation mechanisms that we are using, as ate-controller already uses controller runtime.
troychiu commentedon Aug 5, 2026
For reference #674 (comment), we are considering moving away from controller-runtime to use client-go as our reconciliation mechanism.
AiWatson343 commentedon Aug 6, 2026
This separation makes sense. The WorkerPoolSyncer is effectively a Kubernetes reconciliation component, so keeping it inside ate-api-server creates unnecessary coupling between the API surface and cluster state synchronization responsibilities.
Moving it into ate-controller aligns better with the Kubernetes controller pattern: informers watch desired/current state changes, controllers reconcile resources, and the API server remains focused on serving requests and maintaining API behavior.
Benefits of the move:
Clearer ownership of Kubernetes-facing logic and lifecycle management.
Better isolation of informer failures or API-server scaling concerns.
Easier future expansion of worker reconciliation logic (status updates, health checks, drift correction).
More consistent architecture with other controller-style components.
The migration should preserve the current synchronization contract and consider leader election/HA behavior if multiple controller replicas will run the informer. It may also be worth reviewing event handling and backpressure so worker state changes do not impact controller responsiveness.
HavenXia commentedon Aug 6, 2026
I think moved syncer should keep calling ate-api over gRPC instead of directly writing to the store, we should keep the
ate-apias the single storage accessor. Having two accessor to our data will become more complicated for the system's compatibility.Scott Funkenhauser (@sfunkenhauser) WDYT?
troychiu commentedon Aug 7, 2026
Yes and I think that's why #730 is a blocker for this.
HavenXia commentedon Aug 7, 2026
Great, thanks!
Move WorkerPoolSyncer to ate-controller (#1203)