Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 1 addition & 1 deletion .agents/skills/post-draft-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ mutation($review: ID!, $path: String!, $line: Int!, $body: String!) {
path: $path, line: $line, side: RIGHT, body: $body
}) { thread { id } }
}' -f review="PRR_kwDO..." \
-f path="cmd/ateom-microvm/restore.go" -F line=214 \
-f path="cmd/ateworker-microvm/restore.go" -F line=214 \
-F body=@c1.md
```

Expand Down
4 changes: 2 additions & 2 deletions .agents/skills/triage-issues/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,8 +52,8 @@ Apply all areas that are touched. Most issues need one or two; a few need more.
| `area/api-machinery` | API server internals: RPC handlers, storage layer, syncer, controllers (`cmd/ateapi/`, `cmd/atecontroller/`) |
| `area/network` | Networking: atenet-router, Envoy, ext_proc, DNS, xDS, ingress (`cmd/atenet/`) |
| `area/node` | Node agent and worker lifecycle: atelet, sandbox launch, OCI, cgroups (`cmd/atelet/`) |
| `area/gvisor` | gVisor sandbox specifics: runsc integration, GPU in gVisor, gVisor OCI (`cmd/ateom-gvisor/`) |
| `area/microVM` | Micro-VM sandbox specifics: cloud-hypervisor, kata, snapshot/restore (`cmd/ateom-microvm/`) |
| `area/gvisor` | gVisor sandbox specifics: runsc integration, GPU in gVisor, gVisor OCI (`cmd/ateworker-gvisor/`) |
| `area/microVM` | Micro-VM sandbox specifics: cloud-hypervisor, kata, snapshot/restore (`cmd/ateworker-microvm/`) |
| `area/storage` | Snapshot storage, image cache, GCS/S3 backends, retention (`internal/imagecache/`, ActorSnapshot) |
| `area/scheduling` | WorkerPool sizing, actor placement, HPA, resource allocation |
| `area/observability` | Metrics, tracing, logging, OTLP, OTel SDK integration |
Expand Down
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ body:
id: logs
attributes:
label: Relevant Logs and Diagnostic Output
description: Please provide relevant logs from atelet, ateom, ate apiserver, or kubectl describe.
description: Please provide relevant logs from atelet, ateworker, ate apiserver, or kubectl describe.
render: shell
placeholder: |
# Paste relevant logs or command output here
Expand Down
4 changes: 2 additions & 2 deletions .github/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ See the [threat model](../docs/threat-model.md) for the trust boundaries and
assumptions this section is based on.

In scope: the Agent Substrate control plane (`ateapi`), node supervisor
(`atelet`, `ateom`), networking stack (`atenet`), and CLI (`kubectl-ate`).
(`atelet`, `ateworker`), networking stack (`atenet`), and CLI (`kubectl-ate`).

Agent Substrate multiplexes many actors onto a smaller pool of shared
workers, so its most important security boundaries are between actors and
Expand All @@ -70,7 +70,7 @@ valuable, listed roughly from most to least severe:
`ateapi` RPCs without valid credentials (mTLS or JWT auth modes),
misusing workerpool client certificates, or acting beyond what valid
credentials are authorized to do.
- **Actor-to-platform escalation**: an actor compromising `atelet`, `ateom`,
- **Actor-to-platform escalation**: an actor compromising `atelet`, `ateworker`,
or the control plane, whether while the actor is still actively running
or beyond its own lifecycle.
- **Node lateral movement**: using a compromised worker node to reach other
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr-workflow.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ jobs:
done
# Every worker pod in any namespace: the demo pools plus the e2e suites'
# randomly-named per-test namespaces, which the suites keep on failure.
# The failing actor runs in one of these, so this is where its ateom logs
# The failing actor runs in one of these, so this is where its worker logs
# (and, for a micro-VM worker, the guest console tail) live.
kubectl --context kind-kind get pods -A -l ate.dev/worker-pool \
-o 'custom-columns=:.metadata.namespace,:.metadata.name' --no-headers 2>/dev/null \
Expand Down
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ tools/ # Standalone Go tools (go run ./tools/<name>) for Dev/CI
| Shared across binaries, not for external import | `internal/<pkg>` |
| Public API for external consumers | `pkg/<pkg>` |
| Public proto (control-plane gRPC API) | `pkg/proto/<name>` |
| Internal proto (atelet / ateom) | `internal/proto/<name>` |
| Internal proto (atelet / ateworker) | `internal/proto/<name>` |
| Dev/CI scripts | `hack/` |
| Standalone Go dev/CI tools | `tools/<name>` with its own `go.mod` |
| Self-contained plugin that nothing in the repo depends on | `internal/plugins/<name>` with its own `go.mod`, importing only `pkg/`; see `internal/plugins/README.md` |
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
* [Architecture](docs/architecture.md): How the control plane, node supervisor, and networking stack fit together.
* [API Configuration Guide](docs/api-guide.md): Detailed reference for configuring WorkerPools, ActorTemplates, Secrets, and Volumes.
* [Full CLI Documentation](cmd/kubectl-ate/README.md): Installation and usage for `kubectl-ate`.
* [Glossary](docs/glossary.md): Core terms (Actor, Atespace, ActorTemplate, WorkerPool, Worker, ate-api-server, atenet, atelet, ateom) and how they relate.
* [Glossary](docs/glossary.md): Core terms (Actor, Atespace, ActorTemplate, WorkerPool, Worker, ate-api-server, atenet, atelet, ateworker) and how they relate.
* [Integration Repositories](docs/integration-repos.md): Where integrations live, how their repositories are named, and how fixes flow back to core.
* [Observability Guide](docs/observability.md): Guide to actor logging, metrics, and distributed tracing.
* [Authentication Guide](docs/authentication.md): Configure trusted JWT providers and human credentials.
Expand All @@ -263,8 +263,8 @@ We provide several sample applications demonstrating Agent Substrate's capabilit
* `cmd/atelet`: A node-level DaemonSet that supervises physical worker pods, coordinates snapshotting, and manages state transfers.
* `cmd/atecontroller`: A Kubernetes controller that reconciles WorkerPool custom resources.
* `cmd/atenet`: A combined networking controller providing Envoy routing and proxy sidecars.
* `cmd/ateom-gvisor`: An interior-pod helper running inside sandboxed worker pods to execute `runsc` checkpoint and restore commands.
* `cmd/ateom-microvm`: The micro-VM peer of `ateom-gvisor`, running actors as cloud-hypervisor VMs.
* `cmd/ateworker-gvisor`: An interior-pod helper running inside sandboxed worker pods to execute `runsc` checkpoint and restore commands.
* `cmd/ateworker-microvm`: The micro-VM peer of `ateworker-gvisor`, running actors as cloud-hypervisor VMs.
* `cmd/podcertcontroller`: A "polyfill" that provides Pod Certificate signers that
will eventually ship in upstream Kubernetes (with different names).
* `cmd/kubectl-ate`: A CLI tool for managing Agent Substrate resources. See its [README](cmd/kubectl-ate/README.md).
Expand Down
8 changes: 4 additions & 4 deletions benchmarking/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,7 @@ fakes from `cmd/benchmarking/isolate/` instead of worker pods:
up and down with the pool, draining a Worker before deleting it. Each Worker
reports what a real worker would: cpu and memory from the pool's
`template.resources.limits`, or the node's allocatable without one, and 1000
actors, ateom's default, unless the pool's `ate.dev/fake-max-actors`
actors, ateworker's default, unless the pool's `ate.dev/fake-max-actors`
annotation says otherwise. It writes the pool's status, so
`kubectl get workerpool` and `kubectl scale workerpool` work. A node keeps
its Workers for `--node-grace`, five minutes by default, after its
Expand All @@ -472,11 +472,11 @@ fakes from `cmd/benchmarking/isolate/` instead of worker pods:
would run on, and answers every
lifecycle call with success after its `--delay`, without running or
saving any workload. It writes one placeholder object per snapshot, because
ate-api-server refuses to tag an empty one. Like ateom, it mints the actor's
ate-api-server refuses to tag an empty one. Like ateworker, it mints the actor's
certificate from ate-api-server on every run and restore that names an
egress gateway, which the default install does; the mint is ate-api-server's
time, so it adds to `--delay`, the data plane's share of the call.
It also relays fake-workersync's capacity reports, as atelet relays ateom's:
It also relays fake-workersync's capacity reports, as atelet relays ateworker's:
ate-api-server accepts a report only from the atelet on the Worker's node.
The relay admits only fake-workersync's pod identity.

Expand Down Expand Up @@ -647,7 +647,7 @@ still succeeds. A `null` means the value was not measured. It never means zero.

`SuspendActor` and `ResumeActor` latency can be attributed to the phases
inside them (object-storage transfer, hypervisor snapshot/restore, rootfs
assembly, ...) from the structured log records atelet and ateom-microvm
assembly, ...) from the structured log records atelet and ateworker-microvm
emit. `analysis/collect_logs.sh` dumps the node logs of a run and
`analysis/phase_report.py` aggregates them into per-phase percentiles and
waterfalls of the slowest operations. See
Expand Down
30 changes: 15 additions & 15 deletions benchmarking/analysis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
This directory turns the node side's developer-facing timing logs into the
percentiles a benchmark run needs, without making the phases metric API. The
`ate.actor.{restore,checkpoint}.duration` histograms stop at
`ateom_restore` / `ateom_checkpoint` / `persist` / `download` on purpose:
`worker_restore` / `worker_checkpoint` / `persist` / `download` on purpose:
phases inside those are implementation details of a runtime, and metrics are
a contract. The finer breakdown is emitted as structured log records instead,
and this is the consumer that aggregates them.
Expand All @@ -12,8 +12,8 @@ Two joinable JSON log records feed it, each written by both layers:

| record (`msg`) | emitter | keys |
|---|---|---|
| `Restore timing breakdown` | atelet, ateom-microvm and ateom-gvisor | `ate.actor.restore.duration.<phase>` / `ateom.actor.restore.duration.<phase>` |
| `Checkpoint timing breakdown` | atelet, ateom-microvm and ateom-gvisor | `ate.actor.checkpoint.duration.<phase>` / `ateom.actor.checkpoint.duration.<phase>` |
| `Restore timing breakdown` | atelet, ateworker-microvm and ateworker-gvisor | `ate.actor.restore.duration.<phase>` / `ateworker.actor.restore.duration.<phase>` |
| `Checkpoint timing breakdown` | atelet, ateworker-microvm and ateworker-gvisor | `ate.actor.checkpoint.duration.<phase>` / `ateworker.actor.checkpoint.duration.<phase>` |

Every record carries the full actor identity (`ate.actor.uid`, name,
atespace, template) and the snapshot scope, which the histograms are barred
Expand All @@ -40,7 +40,7 @@ A failed atelet operation still writes its record, marked with `error.type`
scripts. `--since-time` (or a relative `--since 30m`) should cover the run
and nothing before it. `--namespace` (default `ate-system`) selects the
atelet pods and `--worker-namespace` (default `benchmark-workloads`) the
worker pods; the ateom records land in the worker pod's stdout. `kubectl logs`
worker pods; the ateworker records land in the worker pod's stdout. `kubectl logs`
returns only a container's current log file, so on a long or busy run
kubelet's rotation can drop earlier records — collect soon after the run.
A container that restarted mid-run is dumped twice (`<pod>.previous.log`).
Expand Down Expand Up @@ -75,33 +75,33 @@ A failed atelet operation still writes its record, marked with `error.type`
scope and phase: count, p50, p90, p95 and max. A `golden` restore downloads
the golden image and a `latest` one the actor's own, so they are separate
rows, as are gVisor and micro-VM operations. The atelet rows split a
checkpoint between `sandbox_assets`, `ateom_checkpoint` and `persist`, and a
checkpoint between `sandbox_assets`, `worker_checkpoint` and `persist`, and a
restore between `volume_mount`, `manifest_fetch`, `sandbox_assets`,
`download`, `oci_unpack` and `ateom_restore`. The ateom rows split the
`ateom_*` phase further with the runtime's own phases: for micro-VM
`download`, `oci_unpack` and `worker_restore`. The ateworker rows split the
`worker_*` phase further with the runtime's own phases: for micro-VM
`prep` / `pause` / `snapshot` / `durable_dir` / `rootfs_upper` / `teardown`
on a checkpoint and `prep` / … / `vm_restore` / `wakeup_probe` on a restore;
for gVisor `pause` / `checkpoint` / `durable_dir` / `resume` / `teardown` and
`prep` / `net_setup` / … / `pause_restore` / `app_restore` / `wakeup_probe` /
`activate` (the full lists are in `cmd/ateom-*/phaselog.go`). The ateom
`activate` (the full lists are in `cmd/ateworker-*/phaselog.go`). The ateworker
records carry no sandbox class or kind of their own; a paired one takes both
from its atelet record, so both layers split into the same rows.

Concurrency matters when reading them: the atelet restore phases overlap
(the download runs alongside the asset fetch and OCI unpack), the three
ateom checkpoint captures run concurrently on the paused guest (the paused
window costs their max), and the ateom restore phases are sequential. For
ateworker checkpoint captures run concurrently on the paused guest (the paused
window costs their max), and the ateworker restore phases are sequential. For
the two checkpoint layers the report derives an `unattributed` row: the total
minus what the logged phases account for, counting the concurrent captures
once. It is the time the instrumentation does not yet name. The ateom
once. It is the time the instrumentation does not yet name. The ateworker
records carry no snapshot kind of their own; a paired one takes its kind from
the atelet record, so both layers split into the same rows.

**Waterfalls.** The slowest operations, with the ateom record of the same
actor nested under the atelet `ateom_*` phase and that phase's gap to the
ateom total (RPC and queueing between the layers). The ateom record is the
**Waterfalls.** The slowest operations, with the ateworker record of the same
actor nested under the atelet `worker_*` phase and that phase's gap to the
ateworker total (RPC and queueing between the layers). The ateworker record is the
one written inside that phase's window (for a checkpoint, before `persist`
began), and each ateom record pairs with at most one operation, so an
began), and each ateworker record pairs with at most one operation, so an
operation whose own record is missing prints without one rather than with
another cycle's. Failed operations (`error.type` set) are left out of both
the percentiles and the waterfalls. Tail outliers that blow up in one phase
Expand Down
2 changes: 1 addition & 1 deletion benchmarking/analysis/collect_logs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

# Dump the node-side logs a benchmark run needs for phase_report.py: every
# atelet pod (the atelet-side timing breakdowns) and every worker pod
# (ateom-microvm writes its records to the worker pod's stdout). Point
# (ateworker-microvm writes its records to the worker pod's stdout). Point
# --since-time (RFC 3339) or --since at the run's start so the report covers
# exactly one run.
#
Expand Down
Loading
Loading