feat(dev): run agentex locally without Docker - #353
Conversation
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Stand up the full backend as host processes with embedded datastores — no Docker daemon required — as a lighter alternative to the container stack. `./dev.sh local` (also `make dev-local` / `python -m scripts.dev_local`) provisions: - Postgres via bundled pgserver (unix socket) and Redis via bundled redislite - a Temporal dev server + UI and the agentex worker (--no-temporal to skip) - a local mongod, required for the full stack (--no-mongo / --lean to skip) - an optional OpenTelemetry collector (--no-otel to skip) then runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on SIGINT/SIGTERM. --lean is a minimal Postgres+Redis+API stack; --ephemeral uses a throwaway data dir. The runner is a small scripts/dev_local package (config / services / supervise / runner) so the pure config/env layer stays testable. App-side changes to make the no-Docker path robust (all no-ops when Mongo is configured, as it always is in Docker/prod): - The Temporal worker no longer crashes when MongoDB is unavailable — the Mongo CRUD adapter tolerates an unset database and errors only on real use, so the worker degrades like the API instead of taking down the stack. - Skip the Mongo connection entirely when MONGODB_URI is unset, removing a ~20s startup hang against the implicit localhost:27017 default. - In local mode the backend rewrites agents' host.docker.internal ACP host to loopback (AGENTEX_ACP_HOST_OVERRIDE), so default-scaffolded agents work without manifest edits. Also fix a frontend dev-server process leak in dev.sh (kill the whole make→npm→next tree and sweep orphans; report status by listening port), correct the MongoDB and OpenTelemetry install commands, and document local mode in README and CLAUDE.md.
356b3eb to
aa8bb61
Compare
| raw = acp_url_override | ||
|
|
||
| # Prefer the production deployment's URL when there's no explicit override. | ||
| if raw is None and agent.production_deployment_id: |
There was a problem hiding this comment.
@smoreinis can you take a look here just make sure it doesn't conflict with the preview workflow?
| """ | ||
| # In docker-free local mode, rewrite host.docker.internal -> the host-reachable | ||
| # override so the healthcheck matches how the request path dials the agent. | ||
| acp_url = resolve_acp_url(acp_url) |
There was a problem hiding this comment.
just making sure, no other places we need to do conversion right?
| if acp_url_override: | ||
| return acp_url_override | ||
| """Resolve the ACP URL for an agent, optionally overriding with a specific URL. | ||
|
|
There was a problem hiding this comment.
did we lose the override?
There was a problem hiding this comment.
hi, so ./dev.sh no-docker will pass _ACP_HOST_OVERRIDE_ENV variable, so the function resolve_acp_url will resolve this. Besides this, the logic of this function is kept the same
| Returns the URL unchanged when the override env var is unset or the URL does | ||
| not use the Docker sentinel host, so it is safe to call on every ACP dial. | ||
| """ | ||
| override = os.environ.get(_ACP_HOST_OVERRIDE_ENV) |
There was a problem hiding this comment.
is this where the override moved?
There was a problem hiding this comment.
yes, here we override the environment variable _ACP_HOST_OVERRIDE_ENV. This environment is set and passed from config.py file
|
|
||
| > **MongoDB is required for the full local stack** and is always started — the Temporal | ||
| > worker builds Mongo-backed repositories at startup, so a missing/unreachable Mongo | ||
| > makes the runner fail fast (with an install message) rather than crash the worker. |
danielmillerp
left a comment
danielmillerp
left a comment
There was a problem hiding this comment.
lots of small questions! also as far as testing on PC, not super sure. A lot of our clients do use PCs and I know there are Scaliens who have PCs. What problems do you anticipate?
There was a problem hiding this comment.
Overall this is rad.
One UX suggestion: in addition dev.sh, I think this would be really nice to have in the agentex CLI. I.E. you can run a single agent inline. therefore to start running a single agent you can start quick rather than having both the agentex backend and cli running. then this setup would be suitable for someone testing a multiagent flow.
Not sure if that is possible given the abstraction and integration, but wanted to throw it out there as a north star.
+1 to that! |
@danielmillerp So Im testing this on Windows (via AWS Workspaces) but the embedded Redis uses (the redislite package has no native Windows build (Redis ships no supported Windows server) so the dependency required to run locally without Docker won't even install on native Windows. |
Some findings from testing local dev on Windows:
@NiteshDhanpal had a clear view on how to scope this. His take: don't treat Windows-native local mode as a hard requirement for now (given the redislite/Redis limitation), and use a support matrix instead:
His main concern is a reliable fallback so customers aren't blocked when local setup is painful or impossible, and he thinks a cloud/dev-environment fallback is the most reliable option for that. |
I'm aligned @levilentz does that make sense for customers? Also CC: @lucyakoroleva for context per today's standup discussion |
@danielmillerp @aringuyen3 I think broadly this is a good first step. I think assuming that the customer will not have WSL or docker is the way to test this as that it normally the problem. can we just disable redis in this mode? |
I like that! not necessary to go from 0 to 1 to stream |
Default the no-docker embedded Redis to 6379 (was 6390) and the Temporal UI to 8080 (was 8233) so both modes expose the same URLs and quick start needs no port flags. Users can still pass --redis-port / --ui-port if a Docker service already holds a port. Also drop the now-unnecessary per-mode Temporal UI branching in dev.sh and add a README note pointing users to wait for the readiness message before starting an agent.
@danielmillerp @levilentz Redis is required. Agentex uses Redis for streaming and messaging. Referred to this PR #343 |
Add a "Choosing a Setup" section to WINDOWS.md: run docker-free local mode (./dev.sh no-docker) inside WSL2 when it's available, and fall back to the Docker-based PowerShell flow when WSL2 isn't (locked-down or managed environments). Includes a WSL2 quick-start subsection. Expand the README's Windows pointer to surface the same decision from the main entry point.
A previous dev stack whose supervisor died without reaping its children leaves processes bound to the API/datastore ports. The next launch then fails to bind a managed process and tears the whole stack back down, with the real EADDRINUSE buried in a background log. Add a preflight port-conflict check to both start paths that lists any process already listening on a needed port (with PID + command) and offers to kill it before launching: - TTY: prompts [y/N] to kill and continue, else aborts with a clear hint. - DEV_KILL_PORTS=1: kills without prompting (restart/CI). - Non-interactive without the opt-in: warns and continues (unchanged). Kills gracefully via kill_tree (TERM, then force-KILL survivors) so a supervisor and its datastore children go down together. The no-docker check honors the runner's port-override flags; Postgres is skipped there since it uses a Unix socket.
|
Want your agent to iterate on Greptile's feedback? Start a greploop in Cursor and it will work through the open comments and keep going until this PR reviews clean. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Warning Review the following alerts detected in dependencies. According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.
|
3 participants
What
Adds a docker-free mode that runs the full agentex backend as host processes with embedded datastores — a lighter alternative to the Docker Compose stack, closer to a one-command
langgraph dev-style workflow.The bare
./dev.sh(Docker) is unchanged and now also accepts an explicit./dev.sh dockeralias. The docker-free mode is also available asmake dev-no-dockerandpython -m scripts.dev_nodocker.Why
Standing up a local environment previously required the full Docker stack. This lets a developer run the backend with a single command and no Docker daemon:
pgserver(unix socket) and Redis via bundledredislitemongod— always started; the stack requires it (the Temporal worker builds Mongo-backed repositories at boot)It runs migrations, supervises uvicorn + the worker, and tears everything down cleanly on Ctrl-C / SIGTERM.
--ephemeraluses a throwaway data dir;--mongo-uripoints at an existing MongoDB instead of launching a localmongod.App-side change
Safe no-op wherever the backend runs in Docker / staging / prod (the env var it keys on is unset there):
ACP host rewrite for docker-free mode. Agents register their ACP URL at
host.docker.internal(the SDK default, so a Docker backend can reach an agent on the host), which a host-process backend can't resolve. The runner setsAGENTEX_ACP_HOST_OVERRIDE=127.0.0.1and the backend rewrites only that sentinel host to the override when dialing agents — in the ACP request path, the agent-API-key proxy path, and the Temporal healthcheck. When the env var is unset, the stored URL is used verbatim. Default-scaffolded agents work without manifest edits.Also
dev.sh(kill the wholemake → npm → nexttree and sweep orphans; report status by listening port).brew trustfirst) and OpenTelemetry (release binary; not in Homebrew) install commands.dev-no-dockeruv dependency group (pgserver,redislite,greenlet) pulled only for docker-free mode.README.mdandCLAUDE.md.Contract note
The runner is a package (
scripts/dev_nodocker/), so the direct invocation ispython -m scripts.dev_nodocker(notpython scripts/dev_nodocker.py)../dev.sh no-dockerandmake dev-no-dockerare unchanged for callers.Testing
Manually exercised on macOS (Docker stopped):
--lean— the API passes/healthzwithin ~1s and/readyzreports Postgres, Redis, and MongoDB all healthy; the Temporal worker stays up; teardown frees all ports with no orphaned processes.mongodabsent aborts with an actionable install message (or point at an external instance with--mongo-uri).python -m scripts.dev_nodocker; agents scaffolded byagentex initconnect without manifest edits (ACP host rewrite).Platform support: macOS and Linux only. Not supported on native Windows — the embedded Redis (
redislite) ships no Windows server build; run it under WSL2, whereredislite/pgserver/mongodbehave as on Linux (WSL2 path not yet verified).Greptile Summary
This PR adds a docker-free local development mode (
./dev.sh no-docker/make dev-no-docker/python -m scripts.dev_nodocker) that boots the full agentex backend using embedded datastores (pgserver, redislite) and spawns Temporal, MongoDB, and an optional OTel collector as host processes. It also introducessrc/utils/acp_url.pyto rewritehost.docker.internalACP URLs to127.0.0.1in no-docker mode, fixes a frontend process-tree leak indev.sh, and corrects a pre-existing empty-stringacp_urlfallback bug inagents_acp_use_case.py.scripts/dev_nodockerpackage (config,services,supervise,runner): provisions embedded Postgres/Redis/Temporal/MongoDB/OTel, runs Alembic migrations, supervises uvicorn + the Temporal worker, and tears everything down cleanly on Ctrl-C/SIGTERM. The crash path correctly exits with code 1.src/utils/acp_url.py): resolveshost.docker.internal→ the override address only whenAGENTEX_ACP_HOST_OVERRIDEis set (no-docker mode); the env var is never set in Docker/staging/prod, making this a safe no-op elsewhere.dev.shextensions: newno-dockersubcommand withstart_backend_nodocker, mode-awarestop_services/show_status, process-tree kill (kill_tree), stray-frontend sweep, and port preflight checks for both Docker and no-docker stacks.Confidence Score: 5/5
Important Files Changed
Sequence Diagram
sequenceDiagram participant U as User participant S as dev.sh participant R as runner.py (no-docker) participant SV as services.py participant API as uvicorn API participant W as Temporal Worker U->>S: ./dev.sh no-docker [flags] S->>S: validate_nodocker_args S->>S: ensure_prerequisites S->>S: ensure_nodocker_service_binaries (install mongod/otel) S->>S: preflight_ports_nodocker S->>R: uv run python -m scripts.dev_nodocker R->>SV: provision_postgres → pgserver (embedded) R->>SV: provision_redis → redislite (embedded) R->>SV: provision_mongo → local mongod / external URI R->>SV: provision_temporal → WorkflowEnvironment.start_local R->>SV: provision_otel → otelcol-contrib (optional) R->>R: run_migrations (alembic upgrade head) R->>API: spawn uvicorn src.api.app:app R->>W: spawn src/temporal/run_worker.py R->>R: wait_for_health /healthz R->>U: print banner (URLs) Note over R: waits for SIGINT/SIGTERM or process death U->>S: ./dev.sh stop (or Ctrl-C) S->>R: kill -TERM backend_pid R->>W: terminate worker R->>API: terminate uvicorn R->>SV: temporal_env.shutdown() R->>SV: terminate otel / mongo R->>SV: teardown_redis / pg_server.cleanup()Reviews (10): Last reviewed commit: "Merge branch 'main' into aringuyen/run-l..." | Re-trigger Greptile