Reach daemons inside an environment: ate-env-api port proxy, NeMo Gym endpoint()

[Tomer Glottmann (tomergee)]

Lets code outside the cluster talk to a daemon running inside an environment, and wires it into the NeMo Gym provider as nemo-gym 0.7's endpoint(handle, port).

Why

NeMo Gym recipes can have daemon attached to unmodified task images at runtime, the way #70 attaches the ate-env guest. Half of that exists: a runtime layer (--layer) mounts the daemon's image read-only, the guest starts it (--sidecar) and gates readiness on it (--sidecar-readyz), and every template derived per task image inherits it (RUNTIMES.md). The missing half, listed as future work in DESIGN.md, was reaching the daemon from outside: ate-env-api proxied only the guest. Gym's provider contract models this as SandboxSpec.ports plus the optional SupportsSandboxEndpoint.endpoint(handle, port) -> SandboxEndpoint(endpoint, headers) (nemo-gym 0.7, sandbox/providers/base.py). The same gaps, binary injection into users' images and sidecar services, came up from the OpenKruise Agents team on the RL sandboxes discussion.

What

ate-env-api port proxy (internal/portproxy). A plain HTTP request to the API's listener carrying X-Env-Id, X-Env-Port and optionally X-Env-Atespace goes to that port in that environment, whatever its path. The atenet router already supports this shape of traffic: an HTTP CONNECT to its CONNECT listener (port 8081 on a stock install, enabled and admitted by the default network policy) naming the actor in ate-target-actor and the port in the authority; the router resumes the actor first. The proxy:

• validates the target (DNS-label id and atespace, port 1-65535) before anything reaches a CONNECT line;
• strips the three routing headers, sets Host: 127.0.0.1:<port> (what an in-sandbox client sends), flushes streamed responses at once;
• keeps idle tunnels in one pool per environment and port (the transport's pool key encodes the target, so tunnels are never shared across environments; there is a test for that);
• passes a router 404 or 503 through, answers other failures with 502 and a handshake timeout with 504.

Path form for clients that cannot set headers (WebSocket clients, browsers): /v1alpha/envs/{id}/ports/{port}/..., or /v1alpha/atespaces/{atespace}/envs/{id}/ports/{port}/...; the prefix is stripped, so it works as a base URL (the OpenHands SDK builds its event WebSocket from it). HTTP upgrades pass through either form.

It shares ate-env-api's existing listener: requests with X-Env-Port go to the proxy before the gRPC and MCP handlers. No new port, no manifest change. --atenet-connect names the router's CONNECT listener (default atenet-router.ate-system.svc.cluster.local:8081); empty disables it. Trust model unchanged: anyone who can reach ate-env-api can already run commands in any environment through it.

NeMo Gym provider. create() keeps SandboxSpec.ports on the handle. endpoint(handle, port) raises ValueError for an undeclared port (as Gym's Docker provider does), wakes the sandbox with one guest probe, and returns SandboxEndpoint(endpoint=<ate-env-api URL>, headers={X-Env-Id, X-Env-Atespace, X-Env-Port}). connection.endpoint_url overrides the URL. _compat gains a SandboxEndpoint mirror for nemo-gym 0.5 and the hermetic tests. The README documents the full setup: daemon layer and sidecar on the base template, template: in the config, ports on the spec, endpoint() at call time.

Fake router answers CONNECT like the real one, for the proxy tests.

Verification

• go test -race ./...: 13 packages ok (10 new proxy tests incl. path form and an upgrade through the tunnel, run 3 times with -race; a mutation of the pool key fails the isolation test). make verify-gofmt verify-boilerplate pass, staticcheck clean.
• nemo-gym: 41 passed (5 new endpoint tests), installed as CI does. Python client: 119 passed. Ruff on the integration: the 12 pre-existing findings, none new.
• Live on a GKE cluster (Substrate from 2026-10-12, router with --port-connect=8081), ate-env-api built from this branch, a base template with busybox:1.37-musl as the runtime layer and busybox httpd -p 44772 as the sidecar, environments created from unmodified python:3.12-slim:
  • from outside, curl -H 'X-Env-Id: …' -H 'X-Env-Port: 44772' <api>/etc/hostname returned the file from the daemon (200), query strings and the daemon's own 404 passed through;
  • unknown environment: 404 from the router; bad port header: 400; requests without the header still reach the API (/healthz);
  • proxied requests cost the same as the API's own /healthz over the same port-forward (about 0.3 s from a laptop), so the in-cluster overhead is not visible at that resolution;
  • environment suspended, then one proxied request: the router resumed it and the request answered 200 in 3.3 s; state RUNNING afterwards;
  • NeMo provider: create(SandboxSpec(image=python@sha256:…, ports=(44772,))) in 3.5 s, endpoint() in 0.22 s, a GET with the endpoint's headers reached the daemon, exec still worked, an undeclared port raised ValueError, close() left the sandbox stopped.
  • Templates, environments and the API image were reverted after the run.

Limits and follow-ups

• HTTP/1.1 only. WebSocket upgrades pass through (tests in both forms; live with the OpenHands agent-server's event stream in examples: OpenHands agent-server in any image, with OpenHands' CodeAct agent; ate-env manifest template --env #93). gRPC to the daemon needs an h2c upstream, raw TCP needs a raw tunnel.
• A port with nothing listening answers an empty 404 from the router, indistinguishable from a daemon 404 without the body.
• The daemon must be self-contained, like any layer: NeMo Skills' Flask sandbox server needs a relocatable Python in its layer; Gym's ECS exec server needs python3 in the task image.
• Each distinct task image still yields one derived ActorTemplate per base (created on first use, reused after). The daemon adds none, but the per-image template count and its garbage collection are open (DESIGN.md future work).

Overlap

cmd/ate-env-api/main.go conflicts with #86 (both add a flag to the same block). Whichever lands second rebases; the change is mechanical. Merges clean with #88 and #91.