Decouple substrate-plane resources from Kubernetes CRDs

Summary

Move user-facing management of substrate-plane resources out of Kubernetes CRDs and into the ateapi gRPC/API surface.

The goal is to make the public API substrate-native instead of Kubernetes-native, while still allowing internal projectors/controllers to
materialize the backing Kubernetes objects needed to run workloads.

Resources in scope

ActorTemplate
WorkerPool
SandboxConfig
WorkerPoolGrant

Motivation

Today, users interact with several substrate concepts as Kubernetes custom resources. That leaks Kubernetes API shape, identity, naming,
lifecycle, and authorization concerns into the substrate API.

We want these resources to be managed through ateapi so that:

the API is consistent across CLI, SaaS, and direct gRPC clients
substrate resources can evolve independently from Kubernetes CRD conventions
Kubernetes becomes an implementation detail behind projectors
future non-Kubernetes substrate planes are easier to support

Proposed direction

Add API-native resource methods to ateapi for the substrate-plane resources.

Use the style-guide shape from the newer API design:

resource messages include identity and metadata fields
create/update/get/list/delete methods are resource-oriented
get/delete use lightweight refs
list methods support pagination
update requests include google.protobuf.FieldMask
deletes return google.protobuf.Empty
Kubernetes-specific identity should not appear in the user-facing resource API

WorkerPool should not expose Kubernetes namespace identity. The user-facing WorkerPool is globally named, and Kubernetes placement is
represented explicitly through fields such as spec.deployment_atespace.

Non-goals for the initial POC

Do not normalize or redesign Atespace APIs in this change.
Do not fully implement true field-mask patch semantics yet; leave this as a TODO until there is an upstream example to follow.
Do not solve the broader SaaS visibility/availability policy model.
Do not keep CRDs as the user-facing API for the transitioned resources.

Acceptance criteria

Users can create, get, list, update, and delete the in-scope resources through ateapi.
CLI flows use ateapi calls instead of applying Kubernetes CRDs.
E2E/demo setup seeds substrate resources through the API-native path.
API projectors consume the new ateapi resource data and create any required backing Kubernetes objects internally.
Existing Kubernetes CRDs for transitioned resources are no longer part of the user-facing flow.
Tests cover the new API CRUD behavior and the CLI/demo paths.
Atespace API shape remains unchanged in this POC.

Julian Gutierrez Oschmann (@juli4n)

Julian Gutierrez Oschmann (@juli4n) and Tim Hockin (@thockin), here's the issue for the API decoupling we discussed

This looks good. I think a good first step (before implementing it) would be to align on the proto definition. Maybe a draft PR with the expected final version of ateapi.proto?

design: expected final ateapi.proto for decoupled substrate resources

Here is the draft PR #378

ActorTemplate seems very appropriate to me, but it's not clear whether it should be atespaced or not. I'm not clear on whether there are real use-cases for restricting who is allowed to create actors from a template or not. If we put them in atespaces then we need a way to describe who is allowed to reference them from which peer namespaces.

WorkerPool seems a lot less appropriate to me. At best it's a way to de-dup some metadata -- is that useful enough to justify a resource? It seems like it's going to leak.

SandboxConfig is less obvious to me. It sort of feels like it could be arguments to the Ateom rather than a whole API?

I have no looked at the PR - your thoughts on the above?

As long as we are in a timeline where Workers are the entity being scheduled against I am in favor of the syncer copying relevant pod/pool labels onto the Worker record at creation. This would allow for more advanced scheduling intersections as workers inevitably start to acquire more metadata: Machine Type, Tenancy, etc.

is that useful enough to justify a resource? It seems like it's going to leak.

If WorkerPool scheduling role shrinks toward pure capacity management it can drift from ever needing to be exposed.

update requests include google.protobuf.FieldMask

nice, glad we will have some scoping power. Maybe this is useful for list as well?

ActorTemplate seems very appropriate to me, but it's not clear whether it should be atespaced or not. I'm not clear on whether there are real use-cases for restricting who is allowed to create actors from a template or not. If we put them in atespaces then we need a way to describe who is allowed to reference them from which peer namespaces.

WorkerPool seems a lot less appropriate to me. At best it's a way to de-dup some metadata -- is that useful enough to justify a resource? It seems like it's going to leak.

SandboxConfig is less obvious to me. It sort of feels like it could be arguments to the Ateom rather than a whole API?

I have no looked at the PR - your thoughts on the above?

Let me revisit the draft in light of these comments, I meant it mostly to get the conversation going and I think it's worth met investigating this before diving deeper on the current iteration.

I think it's OK to start with the parts we are more confident in and expand.

I reworked the draft direction based on the discussion here and the follow-up conversation around
WorkerPool/SandboxConfig.

Current proposed direction:

Keep ActorTemplate as a core ateapi resource.
- ActorTemplate is global-scoped for the initial API.
- Reasoning: starting global is less restrictive. If we later discover we need atespace-local
  templates, sharing rules, or authorization boundaries around templates, we can add scoping then.
  Going from atespace-scoped to global later is harder because clients may already depend on scoped
  identity and access semantics.
Make Worker the core substrate scheduling primitive.
- Substrate schedules Actors onto Workers directly.
- Worker is a managed ateapi Admin resource with CRUD/watch.
- Scheduling matches ActorTemplate / Actor worker selectors against Worker.meta.labels.
- sandbox_class is currently modeled as a scheduling compatibility string on templates and
  workers.
- Open option: sandbox_class could instead become a substrate-owned/special Worker label if we
  want all scheduling compatibility to flow through label matching.
Remove WorkerPool from the core ateapi surface.
- WorkerPool can still exist as a provider-level abstraction, especially for Kubernetes.
- The analogy is Kubernetes Node vs cloud-provider NodePool: substrate core understands
  Workers; a provider may manage pools of capacity however it wants.
- A Kubernetes provider could keep a WorkerPool CRD/config/autoscaler internally, but it would
  reconcile that into Pods/Workers rather than expose WorkerPool as a substrate API resource.
Remove SandboxConfig from the core API draft.
- Runtime/sandbox setup looks more like provider/runtime configuration or ateom arguments than a
  portable substrate resource.
- The API keeps only the compatibility signal needed for scheduling: sandbox_class.
Add a Kubernetes worker discovery path.
- Worker Pods opt in with a substrate-owned label, initially:
  - pod.ate.dev/is-worker: "true"
- A syncer watches matching Pods and creates/updates/deletes ateapi Worker records.
- That syncer can start in ateapi as a standalone goroutine, but should be factored so it can
  later move into a separate provider process.

The main implementation shape becomes:

Add Worker CRUD/watch to ateapi.
Store Workers by provider-neutral identity.
Convert the existing worker cache to consume managed Workers instead of Kubernetes pod/pool state.
Add a Kubernetes pod syncer for pod.ate.dev/is-worker=true.
Change scheduling to use Workers directly.
Remove Actor/ActorTemplate dependencies on WorkerPool.
Keep WorkerPool, if needed, as Kubernetes-provider implementation detail rather than substrate
API.

I pushed an updated draft branch with the docs/proto changes reflecting this direction to #378

How do we propose to limit auth on who can create/destroy an ActorTemplate? By global scope, does this mean that anyone with the permission to manipulate templates can manipulate all templates?

Have we considered that atespaces are used for a smaller granularity than k8s namespaces? My mental model is that an atespace should be used to contain a set of cooperating agents. There could be a lot of singleton atespaces.

so it could be k8s ns <-- one to many --> atespace

Delete ActorTemplate CRD (agent-substrate#1376)

docs: remove stale references to ActorTemplate as a Kubernetes CRD (#…

Navigation Menu

Repository navigation

Decouple substrate-plane resources from Kubernetes CRDs #368

Summary

Resources in scope

Motivation

Proposed direction

Non-goals for the initial POC

Acceptance criteria

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Decouple substrate-plane resources from Kubernetes CRDs #368

Description

Summary

Resources in scope

Motivation

Proposed direction

Non-goals for the initial POC

Acceptance criteria

Activity

EItanya commented on Jul 1, 2026

juli4n commented on Jul 1, 2026

EItanya commented on Jul 2, 2026

thockin commented on Jul 6, 2026

MushuEE commented on Jul 6, 2026

EItanya commented on Jul 7, 2026

thockin commented on Jul 7, 2026

EItanya commented on Jul 7, 2026

bowei commented on Jul 7, 2026

2 remaining items

bowei commented on Jul 9, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions