Sync upstream authorization and object-store updates

.github/workflows/helm-e2e.yaml

@@ -63,7 +63,7 @@ jobs:
         # the chart composes {registry}/{repository}/{component}, so the local
         # registry serves each image where the default repository expects it --
         # the same path-preserving rule a production mirror follows.
-        for component in ateapi atecontroller atelet podcertcontroller atenet snapshot-plugin credential-provider/kubernetes-secrets; do
+        for component in ateapi atecontroller atelet podcertcontroller atenet objectstore-plugin credential-provider/kubernetes-secrets; do
           KO_DOCKER_REPO="localhost:5001/kagent-dev/substrate/${component##*/}" \
             ./hack/run-tool.sh ko build --bare --tags helm-e2e \
               --platform linux/amd64 "./cmd/${component}"

.github/workflows/release.yaml

@@ -88,7 +88,7 @@ jobs:
         run: |
           set -o errexit -o nounset -o pipefail
 
-          for component in ateapi atecontroller atelet ateom-gvisor ateom-microvm podcertcontroller atenet snapshot-plugin credential-provider/kubernetes-secrets; do
+          for component in ateapi atecontroller atelet ateom-gvisor ateom-microvm podcertcontroller atenet objectstore-plugin credential-provider/kubernetes-secrets; do
             KO_DOCKER_REPO="${IMAGE_REPOSITORY}/${component##*/}" \
             ./hack/run-tool.sh ko build \
               --tags "${IMAGE_TAGS}" \

AGENTS.md

@@ -102,6 +102,7 @@ The following is what Substrate currently offers.
 Keep this up to date when updating AGENTS.md.
 
 - **Workload Isolation**: The project uses `gVisor` (`runsc`) for sandboxing and security isolation of workloads on pods.
-- **JWTs and JWKs**: Signing (`internal/localjwtauthority`), verification (`cmd/ateapi/internal/oidcjwt`), and JWK set publishing (`internal/oidcdiscovery`) are built on the standard library's crypto packages. Extend those packages rather than adding go-jose or another JOSE library, so the code that decides whether a token is valid stays small and supports only the algorithms Substrate uses.
+- **JWTs and JWKs**: Signing (`internal/localjwtauthority`), verification (`internal/oidcjwt`), and JWK set publishing (`internal/oidcdiscovery`) are built on the standard library's crypto packages. Extend those packages rather than adding go-jose or another JOSE library, so the code that decides whether a token is valid stays small and supports only the algorithms Substrate uses.
+- **Ingress Authentication**: `atenet-router --ingress-auth-mode=static-mtls` requires clients to present a SPIFFE certificate signed by `--ingress-client-ca-file` whose ID is in `--ingress-allowed-spiffe-ids`; that list is the whole authorization policy, so any listed client may reach any Actor. The default, `deprecated-insecure`, checks a certificate only when a client presents one, and also serves plaintext listeners that accept any client. Install with it via `ate-setup --ingress-auth-mode=static-mtls` (`manifests/ate-install/components/router-static-mtls`); the allowlist lives in `manifests/ate-install/atenet-router.yaml`.
 
 For future plans for security, reference `docs/roadmap.md`.

benchmarking/README.md

@@ -13,6 +13,12 @@ scenario ladder, read [observability.md](observability.md).
 > [!IMPORTANT]
 > Source the environment configuration file (e.g., `source .ate-dev-env.sh`)
 > first so `PROJECT_ID`, `BUCKET_NAME`, etc. are set.
+>
+> **For Local Kind Clusters**: Set `KO_DOCKER_REPO` to the local registry so images are pulled without remote cloud credentials:
+> ```bash
+> export KO_DOCKER_REPO="localhost:5001"
+> export BUCKET_NAME="ate-snapshots"
+> ```
 
 Note that deploying the benchmarks does not run them. You must visit Locust's
 web UI to start a test.
@@ -113,6 +119,11 @@ and state restoration latency when a durable directory is attached to the actor.
 * `--durdir-template`: ActorTemplate name:
   * `glutton-durdir-data` (default): Attaches a durable data directory without memory snapshot restore.
   * `glutton-durdir-full`: Attaches a durable data directory and performs a full memory snapshot restore.
+  * `glutton-storage`: Attaches an external CSI volume instead of a durable
+    directory. `deploy_locust.sh` and `workloads/deploy.sh` deploy it only
+    when `--storage-class-name` names an existing StorageClass. To run it on
+    NFS, install with `hack/install-ate.sh --setup-csi=nfs` and pass
+    `--storage-class-name csi-nfs-sc`, the class the e2e tests also use.
 
 #### DurDir Reported Metrics
 

benchmarking/deploy_locust.sh

@@ -33,6 +33,7 @@ OTLP_ENDPOINT=""
 ACTOR_MEMORY=""
 # Empty leaves the WorkerPool pods unsized.
 WORKER_MEMORY=""
+STORAGE_CLASS_NAME=""
 WAIT_TIMEOUT_SECS=""
 
 usage() {
@@ -50,6 +51,8 @@ usage() {
   echo "                          benchmark ActorTemplates (default: 256Mi, the microvm minimum)."
   echo "  --worker-memory SIZE    Forwarded to workloads/deploy.sh. Memory request and limit"
   echo "                          for each WorkerPool pod (default: unset, the pod is unsized)."
+  echo "  --storage-class-name S  Forwarded to workloads/deploy.sh. StorageClass for the glutton-storage"
+  echo "                          external volume; setting it deploys glutton-storage."
   echo "  --wait-timeout SECONDS  Forwarded to workloads/deploy.sh. The timeout in seconds for"
   echo "                          waiting for the ateom workers to be ready (default: 300)"
   echo "  --skip-build            Skip locust image build/push (use the existing :latest image)"
@@ -80,6 +83,8 @@ while [[ "$#" -gt 0 ]]; do
     --actor-memory=*) ACTOR_MEMORY="${1#*=}" ;;
     --worker-memory) shift; WORKER_MEMORY="$1" ;;
     --worker-memory=*) WORKER_MEMORY="${1#*=}" ;;
+    --storage-class-name) shift; STORAGE_CLASS_NAME="$1" ;;
+    --storage-class-name=*) STORAGE_CLASS_NAME="${1#*=}" ;;
     --wait-timeout) shift; WAIT_TIMEOUT_SECS="$1" ;;
     --wait-timeout=*) WAIT_TIMEOUT_SECS="${1#*=}" ;;
     --skip-build) SKIP_BUILD=1 ;;
@@ -121,6 +126,9 @@ if [[ "${action}" == "deploy" ]]; then
   if [[ -n "${WORKER_MEMORY}" ]]; then
     workload_args+=(--worker-memory "${WORKER_MEMORY}")
   fi
+  if [[ -n "${STORAGE_CLASS_NAME}" ]]; then
+    workload_args+=(--storage-class-name "${STORAGE_CLASS_NAME}")
+  fi
   if [[ -n "${WAIT_TIMEOUT_SECS}" ]]; then
     workload_args+=(--wait-timeout "${WAIT_TIMEOUT_SECS}")
   fi

benchmarking/locust/build_and_push.sh

@@ -24,12 +24,18 @@ if [[ -f .ate-dev-env.sh ]]; then
   source .ate-dev-env.sh
 fi
 
-if [ -z "${PROJECT_ID:-}" ]; then
-  echo "Error: PROJECT_ID environment variable must be set." >&2
+if [[ -z "${PROJECT_ID:-}" && -z "${KO_DOCKER_REPO:-}" && -z "${LOCUST_IMAGE:-}" ]]; then
+  echo "Error: PROJECT_ID or KO_DOCKER_REPO environment variable must be set." >&2
   exit 1
 fi
 
-IMAGE="us-docker.pkg.dev/${PROJECT_ID}/gcr.io/ate-images/locust-test:latest"
+if [[ -n "${LOCUST_IMAGE:-}" ]]; then
+  IMAGE="${LOCUST_IMAGE}"
+elif [[ -n "${KO_DOCKER_REPO:-}" ]]; then
+  IMAGE="${KO_DOCKER_REPO}/locust-test:latest"
+else
+  IMAGE="us-docker.pkg.dev/${PROJECT_ID}/gcr.io/ate-images/locust-test:latest"
+fi
 
 # Target platform must match the cluster's nodes, not the build host.
 PLATFORM="${LOCUST_IMAGE_PLATFORM:-linux/amd64}"

benchmarking/locust/deploy.sh

@@ -24,11 +24,23 @@ if [[ -f .ate-dev-env.sh ]]; then
   source .ate-dev-env.sh
 fi
 
-if [ -z "${PROJECT_ID:-}" ]; then
-  echo "Error: PROJECT_ID environment variable must be set." >&2
+if [[ -z "${PROJECT_ID:-}" && -z "${KO_DOCKER_REPO:-}" && -z "${LOCUST_IMAGE:-}" ]]; then
+  echo "Error: PROJECT_ID or KO_DOCKER_REPO environment variable must be set." >&2
   exit 1
 fi
 
+if [[ -z "${LOCUST_IMAGE:-}" ]]; then
+  if [[ -n "${KO_DOCKER_REPO:-}" ]]; then
+    LOCUST_IMAGE="${KO_DOCKER_REPO}/locust-test:latest"
+  else
+    LOCUST_IMAGE="us-docker.pkg.dev/${PROJECT_ID}/gcr.io/ate-images/locust-test:latest"
+  fi
+fi
+
+LOCUST_IMAGE_PULL_POLICY="${LOCUST_IMAGE_PULL_POLICY:-Always}"
+
+export LOCUST_IMAGE LOCUST_IMAGE_PULL_POLICY
+
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 MANIFEST="${SCRIPT_DIR}/manifests/locust.yaml"
 
@@ -67,7 +79,7 @@ deploy() {
     kubectl -n benchmarking create configmap agentsession-script \
       --from-file="script.yaml=${AGENTSESSION_SCRIPT}" --dry-run=client -o yaml | kubectl apply -f -
   fi
-  echo "Deploying Locust load (PROJECT_ID=${PROJECT_ID}, user_class=${BENCHMARK_USER_CLASS}, agentsession_script=${AGENTSESSION_SCRIPT:-<built-in>})..."
+  echo "Deploying Locust load (image=${LOCUST_IMAGE}, pull_policy=${LOCUST_IMAGE_PULL_POLICY}, user_class=${BENCHMARK_USER_CLASS}, agentsession_script=${AGENTSESSION_SCRIPT:-<built-in>})..."
   envsubst < "${MANIFEST}" | kubectl apply -f -
 }
 

benchmarking/locust/manifests/locust.yaml

@@ -46,8 +46,8 @@ spec:
     spec:
       containers:
       - name: locust-master
-        image: us-docker.pkg.dev/${PROJECT_ID}/gcr.io/ate-images/locust-test:latest
-        imagePullPolicy: Always
+        image: ${LOCUST_IMAGE}
+        imagePullPolicy: ${LOCUST_IMAGE_PULL_POLICY}
         args:
         - "-m"
         - "locust"
@@ -86,8 +86,8 @@ spec:
             memory: "512Mi"
       # Python and boomer workers do not play nicely together, leaving this in for reference.
       # - name: locust-python-worker
-      #   image: us-docker.pkg.dev/${PROJECT_ID}/gcr.io/ate-images/locust-test:latest
-      #   imagePullPolicy: Always
+      #   image: ${LOCUST_IMAGE}
+      #   imagePullPolicy: ${LOCUST_IMAGE_PULL_POLICY}
       #   args:
       #   - "-m"
       #   - "locust"
@@ -122,8 +122,8 @@ spec:
       - name: boomer-worker
         # Same image as the master/worker — the boomer-worker Go binary is
         # baked in at /app/boomer-worker by benchmarking/locust/Dockerfile.
-        image: us-docker.pkg.dev/${PROJECT_ID}/gcr.io/ate-images/locust-test:latest
-        imagePullPolicy: Always
+        image: ${LOCUST_IMAGE}
+        imagePullPolicy: ${LOCUST_IMAGE_PULL_POLICY}
         command: ["/app/boomer-worker"]
         args:
         - "--prometheus-addr=:8001"

benchmarking/nighthawk-egress/README.md

@@ -0,0 +1,107 @@
+# Nighthawk Egress Benchmark
+
+Benchmarks the Agent Substrate egress dataplane (`atunnel` -> `atenet-egress`) using [Nighthawk](https://github.com/envoyproxy/nighthawk).
+
+## Architecture
+
+```mermaid
+flowchart LR
+    client["nighthawk_client<br/>(host)"]
+    invoker["traffic-driver-invoker<br/>(:19980, adds ate-target-actor)"]
+    router["atenet-router<br/>(:80)"]
+
+    subgraph worker["WorkerPod (ateom-gvisor)"]
+        atunnel-ingress["ingress atunnel"]
+        actor["Actor: nighthawk/tester<br/>(nighthawk_service :80)"]
+        atunnel-egress["egress atunnel"]
+    end
+
+    egress["atenet-egress<br/>(HTTP / HTTPS MITM / TLS passthrough)"]
+    server["nighthawk-server<br/>(:80, :443, :8443)"]
+
+    client -->|"start test"| invoker
+    invoker -->|"gRPC + ate-target-actor: nighthawk/tester"| router
+    router --> atunnel-ingress
+    atunnel-ingress --> actor
+    actor ==>|"test traffic"| atunnel-egress
+    atunnel-egress ==>|"mTLS"| egress
+    egress ==> server
+```
+
+## Prerequisites
+
+To measure single CPU performance characteristics of Envoy dataplane set E2E_ATE_ENVOY_CONCURRENCY=1 environment
+variable when deploying ATE.
+
+1. **Cluster with Agent Substrate deployed**:
+   - **Existing cluster** (with `.ate-dev-env.sh` configured):
+     ```bash
+     hack/install-ate.sh --deploy-ate-system --credential-provider='{"name":"k8s.io"}'
+     ```
+   - **Local `kind` cluster**:
+     ```bash
+     hack/install-ate-kind.sh --deploy-ate-system --credential-provider='{"enabled":false}'
+     ```
+   - To set the `atenet-egress` Envoy worker thread count (`--concurrency`), set `E2E_ATE_ENVOY_CONCURRENCY` when deploying:
+     ```bash
+     E2E_ATE_ENVOY_CONCURRENCY=1 hack/install-ate-kind.sh --deploy-ate-system --credential-provider='{"enabled":false}'
+     ```
+2. **`kubectl-ate` installed in `PATH`**:
+   ```bash
+   go install ./cmd/kubectl-ate
+   ```
+3. **Docker** running locally (used to run `nighthawk_client`).
+
+## Setup
+
+### 1. Deploy `nighthawk-server` and `traffic-driver-invoker`
+
+```bash
+./benchmarking/nighthawk-egress/deploy.sh
+```
+
+### 2. Deploy the worker pool, actor template, and `nighthawk/tester` actor
+
+```bash
+./benchmarking/nighthawk-egress/deploy-traffic-driver.sh
+```
+
+### 3. Port-forward `traffic-driver-invoker`
+
+In a separate terminal, forward local port `19980` to the `traffic-driver-invoker` service:
+
+```bash
+kubectl -n ate-system port-forward svc/traffic-driver-invoker 19980:19980
+```
+
+## Running the Benchmark
+
+Run `nighthawk_client` against the port-forwarded `traffic-driver-invoker` endpoint (`grpc://127.0.0.1:19980`), which routes the gRPC request to `nighthawk_service` inside the `nighthawk/tester` actor:
+
+```bash
+docker run --rm --network=host --entrypoint /usr/local/bin/nighthawk_client envoyproxy/nighthawk-dev:latest --nighthawk-service grpc://127.0.0.1:19980 --rps 1000 --concurrency 1 --protocol http1 --duration 60  http://nighthawk-server.ate-system.svc.cluster.local:80
+```
+
+See [Nighthawk CLI help](https://github.com/envoyproxy/nighthawk#using-the-nighthawk-client-cli) for all command line options.
+
+### Target URLs and Default Egress Policy
+
+| Egress Mode | Target URL | Port |
+|---|---|---|
+| Plaintext HTTP | `http://nighthawk-server.ate-system.svc.cluster.local:80` | `80` |
+| HTTPS (TLS MITM) | `https://nighthawk-server.ate-system.svc.cluster.local:443` | `443` |
+| TLS Passthrough | `https://nighthawk-server.ate-system.svc.cluster.local:8443` | `8443` |
+
+## Cleanup
+
+Delete the `nighthawk/tester` actor, actor template, atespace, and worker pool:
+
+```bash
+./benchmarking/nighthawk-egress/deploy-traffic-driver.sh --delete
+```
+
+Delete `nighthawk-server` and `traffic-driver-invoker` from `ate-system`:
+
+```bash
+./benchmarking/nighthawk-egress/deploy.sh --delete
+```