examples: OpenHands agent-server in any image, with OpenHands' CodeAct agent; ate-env manifest template --env

[Tomer Glottmann (tomergee)]

Stacked on #92 (base branch runtime-port-endpoints); retarget to main once #92 lands. Draft until then.

A working example of a real daemon behind #92: the OpenHands agent-server in any unmodified image, driven from outside the cluster: commands and files, OpenHands' own CodeAct agent, and CodeAct trajectories executed step by step from a client. Details in examples/openhands-agent-server/README.md.

What

• ate-env manifest template --env NAME=VALUE (repeatable). Daemons in layers are configured through the environment (the agent-server reads its API key from OH_SESSION_API_KEYS_0), and the guest passes its environment to sidecars, but the template command could not set a variable. POSIX names only, PORT reserved for the guest, duplicates rejected; derived per-image templates keep them. Tests included.
• examples/openhands-agent-server/:
  • Dockerfile: the agent-server as a runtime layer, amd64 and arm64 (see the finding below).
  • demo.sh: registers a base template (layer at /opt/oh, sidecar python -m openhands.agent_server, readiness on /health, key via --env) and drives the agent-server in one environment per image over HTTP. PORT, AGENT_SERVER_ENV, COMMAND, WORKDIR, KEEP make it general.
  • ate_env_workspace.py: workspace_for(env), an OpenHands RemoteWorkspace whose host is Reach daemons inside an environment: ate-env-api port proxy, NeMo Gym endpoint() #92's path form, /v1alpha/envs/<env>/ports/<port>. The SDK appends its /api/... paths and builds its event WebSocket from the same host, so RemoteWorkspace and RemoteConversation work unchanged.
  • codeact_agent.py: runs OpenHands' default agent (terminal, file editor, task tracker, finish; the successor of CodeActAgent) on the agent-server inside an environment, with any litellm model, or --scripted.
  • scripted_llm.py: a stand-in model (standard library, OpenAI-compatible) that replays a fixed CodeAct trajectory through the real agent loop and tools, for runs where no model is reachable from the actor.
  • run_trajectory.py + oh_tool.py: execute a CodeAct trajectory (terminal / file_editor / finish calls) in an environment from outside. Terminal commands go through the agent-server's command endpoint; file edits through OpenHands' own file_editor, served inside the environment by oh_tool.py on the layer's Python and reached through the path form. Observations are recorded next to their calls (--out). No model inside the environment: with a model call before each step it is a client-side agent, which needs no actor egress.
  • trajectories/: hello.json for any image; django-11099.json (a fix for SWE-bench django__django-11099) and django-11099-grade.json (applies the instance's hidden test change and runs its target tests with and without the fix).
• docs/rootfs-images/RUNTIMES.md links the example and lists --env in the layer model.

Finding: the published agent-server binary does not run on older distributions

ghcr.io/openhands/agent-server:1.55.0-* ships a PyInstaller binary built on Debian bookworm (glibc 2.36). On Ubuntu 22.04 (glibc 2.35), the base of many coding-task images:

ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.36' not found (required by /tmp/_MEI…/libstdc++.so.6)

Ubuntu 20.04 fails the same way on GLIBC_2.33. Starting it through the image's own bookworm loader fails too (PyInstaller re-executes itself: Floating point exception on 22.04, an ld.so assertion on 20.04). The layer is python-build-standalone CPython 3.12.15 (built for glibc 2.17, relocatable, checksum-pinned per architecture) with openhands-agent-server==1.55.0 installed by pip on Debian bullseye: every wheel runs on glibc 2.31 and newer, and nothing from the image is used but the kernel and /bin/sh. About 210 MB compressed.

Verification

• go test ./cmd/ate-env ok (new --env tests); make verify-gofmt verify-boilerplate pass; ruff and shellcheck clean on the example.

• Live on GKE (gVisor workers, Substrate from 2026-10-12, ate-env-api from Reach daemons inside an environment: ate-env-api port proxy, NeMo Gym endpoint() #92 at 593f7e6), agent-server answering after create:

  image	base	answering
  ubuntu:20.04	glibc 2.31	16 s
  swebench/sweb.eval.x86_64.django_1776_django-11099	Ubuntu 22.04	16 to 23 s
  namanjain12/pandas_final (R2E-Gym)	Ubuntu 22.04	20 s
  node:22-slim	Debian 12	16 s
  python:3.12-slim	Debian 13	16 s

  demo.sh on each: commands ran with the image's own shell and tools, file download worked, calls without the key got 401; also with PORT=3000 and an extra --env setting.

• CodeAct through ate-env: codeact_agent.py --scripted in the SWE-bench environment ran the full OpenHands agent loop on the in-sandbox agent-server (terminal, file editor create, terminal check, finish) in 2.0 s, with its event WebSocket going through ate-env-api's path form and the Substrate router. A direct WebSocket handshake through the same path returned 101 and the agent-server's own close frame.

• Trajectory from outside: run_trajectory.py with django-11099.json in the SWE-bench environment fixed the issue in 9 steps and 5.2 s (reproduction flips from "accepts" to "rejects a trailing newline", 22 validator tests OK, two-line diff); file edits 0.3 s each with the long-lived tool runner (9 s per edit when each started a fresh process). django-11099-grade.json: target tests OK with the fix, FAILED (failures=2) with it stashed. hello.json ran on the same image.

• ate_env_workspace.py on node:22-slim saw the image's own node v22; on the SWE-bench image the image's Python 3.11.5, not the layer's. After a suspend, the next SDK call woke the environment within 9 s.

• The arm64 layer answered /health in an arm64 Ubuntu 22.04 container (local); the CodeAct flow also ran locally against the layer before the cluster run.

• Actors on the test cluster had no internet egress (connection reset), hence --scripted for the live CodeAct run. Environments, templates and the API image were reverted after the runs.

Limits (also in the README)

• codeact_agent.py calls the model from inside the environment: it must be allowed by Substrate's egress policy or served in-cluster. run_trajectory.py (and a client-side agent built on it) does not.
• A daemon reached through ate-env-api must listen on all interfaces: a loopback-only listener answers 502 bad gateway from the router (found with oh_tool.py; now in RUNTIMES.md).
• One API key per template, visible to readers of templates and shared by its derived templates.
• The agent-server runs as the image's user; the first environment on a node pulls the layer.