Skip to content

Actor -> ActorTemplate relationship #1051

Open

Description

@zoez7

Actor -> ActorTemplate relationship

Today we read some data from ActorTemplate and Actor objects while running. Should we:

a). Keep doing that, and prevent deletion of any AT that has an actor (foreign key like atespace)
b). Keep doing that, but not prevent AT deletion (which would break actors), and just tell users "don't do that" (this is where we are today, IIUC)
c). Copy all data from AT -> Actor, so it is self-contained?

CUJ a) Actor references AT, with deletion restriction b) Actor references AT, without deletion restriction c) Copy all data from AT -> Actor, so it is self-contained
CreateActor (same as UpdateActor's tmpl) Set actor.template=templ.
Must ensure tmpl exists in database.
Set actor.template=templ.
Must ensure tmpl exists in database.
Copy all data from templ, including golden snapshot.
Must ensure tmpl exists in database.
ResumeActor
(winner: c > a > b)
Look up the template by foreign key. Look up the template by foreign key. May fail if the AT is deleted. No need to look up the actor template.
Data at rest
(winner: a = b > c, but for use cases with 1 actor per template (see #553 ), c is as good as a & b)
#templates × template size #templates × template size #actors × template size.
Estimate: 1 billion actors × 1 KB per template =~ 1 TB
DeleteActorTemplate
(winner: c > a > b)
Blocks until all actors are deleted. (Least flexible) Does not block. Accidental deletion can cause outages.
The user needs to either
1. ListActors() until pageToken=="",
2. Or "drain" a template by calling UpdateActor(replacementTempl) + ResumeActor(), then delete the template. After deleting the template, the above logic will still take effect, so ResumeActor will still succeed.
Does not block. Accidental deletion does not cause outages. Actors can still resume with their self-contained spec.
When can users delete an ActorTemplate? When no actors reference it. When the draining mechanism above is in place and has soaked. When users will never call UpdateActor or CreateActor with the target template.
May need the same draining logic to be confident.
Vulnerability scanning of container images
(winner: a = b > c)
Iterate through all templates, and mark them as disabled. Subsequent ResumeActor calls using this template will fail. Customers must do UpdateActor(newTempl) + Resume. Iterate through all templates, and mark them as disabled. If the template is already deleted, existing actors referencing it cannot resume without updating their template first. Iterate through all Actors (and label them as "upgrade required").
List Actors by AT
(winner: a = b > c)
Look up by foreign key. Look up by foreign key while the AT still exists. Table scan. Look up all Actors that reference the AT. ActorTemplates are immutable.

Conclusion

  • Option c paves the road best for RL use cases that require creating an Actor without a template. See issue #553.
  • Option b is error prone because "Accidental deletion of templates will cause outages."

Activity

  1. thockin commented on Aug 19, 2026

    @thockin
    Collaborator

    Option A comes at a potentially signficant cost. Actor is the center of a spiderweb of cross-object dependencies. We currently inhibit deletion of Atespaces when some other resource is "in" them, but not much else. Foreign keys and RI are not free. If we open this door it will be hard to close, and will set precedent.

    Option B is simple and is sort of "your break it, you buy it", but has sharp edges. If we do this, we establish a precendent, too.

    Option C is expensive in a different way but does solve another problem.

    Julian Gutierrez Oschmann (@juli4n) Jet Chiang (@supreme-gg-gg) Joe Betz (@jpbetz) Luiz Oliveira (@laoj2)

  2. jpbetz commented on Aug 19, 2026

    @jpbetz
    Contributor

    For Option B, the ability to bypass immutability by deleting and then creating a template with the same name feels like the sort of thing we want to disallow but that the design would encourage. So I'm only seriously considering A and C.

    For Option C, if the template is deleted, what happens to the golden snapshot? If it get's deleted, any actors needing it are broken if they haven't created their own snapshot yet? Is this somehow addressed?

  3. zoez7 commented on Aug 20, 2026

    @zoez7
    CollaboratorAuthor

    For Option C, if the template is deleted, what happens to the golden snapshot

    Good question. Once a template is deleted:

    • Actors that don't need the golden snapshots are:
      • Any actors that has resumed at least once, AND are configured to use "full memory snapshot" or "DurDir only" during resume.
    • Actors that will need the golden snapshots are:
      • Actors that are created, but hasn't resumed yet.
      • Actors that are configured to always use "Golden + DurDir" during resume at the template level.

    If we go with Option C, deleting the template has to leak the golden snapshots, but GC still needs to know "when can golden snapshot be deleted". Then we need to keep track of the golden snapshot's refcount, and only delete it when the refcount drops to 0.

  4. added this to the M3 milestone on Sep 15, 2026
  5. zoez7 commented on Sep 24, 2026

    @zoez7
    CollaboratorAuthor

    Coming back to this.

    For Option C, if the template is deleted, what happens to the golden snapshot? If it get's deleted, any actors needing it are broken if they haven't created their own snapshot yet? Is this somehow addressed?

    Yes, each golden snapshot is a Tagged snapshot. #1554 will address this by preventing deletion of the Tag when any Actors reference it as the source Tag.

    For Option B, the ability to bypass immutability by deleting and then creating a template with the same name feels like the sort of thing we want to disallow but that the design would encourage. So I'm only seriously considering A and C.

    Since the Actors keep track of the ID of its template, if the template is deleted and recreated, the next resume will be treated as a "Upgrade", where we cold boot the actor, so the behavior is different compared to never deleting the template

    I think the problems with Option B are 1. accidentally deleting the template is detrimental , and 2. pre-check by listing actors before deleting the template can not guarantee that new actors are not created concurrently.

    What about a Option D:?

    Option D) Today's option B, but with soft deletion:

    • calling DeleteActorTemplate will put the AT in aDELETING state, no new actors can be created from it, but existing actors can resume as usual.
    • Users can then call ListActors(template=x) to ensure that no actors are using it.
    • DeleteActorTemplate(force=true) which force deletes the AT, and may leave actors still referencing it.
  6. zoez7 commented on Sep 24, 2026

    @zoez7
    CollaboratorAuthor

    I was leaning towards Option C because it solves another problem (where O(templates) ~= O(actors), or when actors want to augment its templates), but I think if we allow disabling the golden snapshot flow, the cost of creating a template object before creating an Actor seems ok, given we also get the benefit where

    1. Admins get to govern template creation and limit what can run in substrate.
    2. Immutable templates also means we have a "history of actor's specs", if we allow UpdateActor(with arbitrary changes), the client will not have a way to rollback bad changes (unless we also track the history of individual actor specs).

    Given these trade-offs, I think the case is weaker for Option C, and Option D above is preferred.

  7. removed this from the M3 milestone on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions