Agent instance rpc

.gitignore

@@ -6,9 +6,9 @@
 # What an example under `examples/` is given or writes: its own dependency tree and the
 # lockfile that comes with it (each is a real npm application that installs the published
 # package with `npm install`, and a committed lockfile is what 1.0.0 buys), the model key its
-# README tells a reader to put in `.env`, and the Workspace and agent directory its Gateway
-# writes. The committed `.env.example` and `settings.json` beside them are not these: the
-# `.env` line matches that one name and no other.
+# README tells a reader to put in `.env`, and the Workspace, agent directory and Sessions its
+# Agent Instance writes. The committed `.env.example` and `settings.json` beside them are not
+# these: the `.env` line matches that one name and no other.
 #
 # These four lines also decide what Biome sees, because `biome.json` sets `vcs.useIgnoreFile`.
 # That is the whole of what keeps four dependency trees out of `biome check .`, and no Biome

CLAUDE.md

@@ -165,10 +165,24 @@ Four ways that fails quietly, all of them a deployment's to get right:
   generation to the first render**, so `src/signals/template-handler.ts` calls `precompile` above it
   and throws the result away; dropping that puts the case back on a Signal. What it costs is live
   editing: changing a prompt is a rebuild.
-- **Nothing in `src/agent-container/` knows about an Agent Implementation.** That directory is what
-  `docker run` takes and what to do with the result; `src/pi/` is the other half and imports from
-  it. Nothing imports back, no lint rule enforces that, and an import of `../pi/` from there is the
-  thing to refuse in review.
+- **`switch_session` is create-or-resume, that behaviour is undocumented, and the `get_state` that
+  follows it in `src/pi/runtime.ts` is the only thing holding it to account.** A path the Agent
+  Instance has never seen becomes a fresh Session kept there; a path it has seen is loaded; and the
+  same `success: true` comes back either way, so the second step reads `sessionFile` out of the
+  instance's own state and compares it to what was asked for. Deleting that round trip costs a Run
+  nothing any test of the happy path would notice and buys a Prompt delivered into whichever Session
+  the previous connection happened to leave open. `success: true` carrying `cancelled: true`, which
+  is an extension of the Operator's refusing the switch, is the same failure by a documented route
+  and is refused two lines above it.
+- **`sessionNames` in `src/pi/runtime.ts` is a transcription of `pi`'s own `assertValidSessionId`,
+  and nothing anywhere checks that it still matches.** The framework never had to carry one before:
+  `pi` was handed `--session-id` and refused a bad one itself, with its own message. A Session is
+  addressed **by path** over RPC now and `pi` opens any path it is handed, so the grammar has to
+  live on this side and a copy goes stale in silence. It is also the whole of the traversal
+  argument, there being no `/` in it and both ends required to be alphanumeric, so nothing resolves
+  a path and compares it against a prefix. Widening that character class is therefore a Signal
+  Handler's Session name climbing out of `sessionsDir` and not a cosmetic edit; re-read
+  `core/session-manager` against it whenever the pinned `pi` version moves.
 - **`src/http-client-tui/` is the one shipped directory that is not a subpath.** It is a `bin`, so
   the export map is untouched. **It has zero dependencies and must keep them**: a dependency added
   here lands in every consumer's install, and the answer to needing one is a second package.

README.md

@@ -30,7 +30,7 @@ The framework gives you the Gateway. You state what your deployment is made of,
 A Gateway is a set of components. Four are core, and every deployment gets them:
 
 - A **Db**, the PostgreSQL connection every other component stores its state through.
-- A **Signal Worker** that runs the agent, one thing at a time and never two at once.
+- A **Signal Worker** that drives the agent, one thing at a time and never two at once.
 - An **Agent server**, the HTTP API the running agent calls back into.
 - A **Public server**, the HTTP API the parties' own clients talk to.
 
@@ -50,10 +50,12 @@ stopped with it, and can serve its own routes on either server.
 
 ![The parts of a shared agent. A dashed boundary encloses the Gateway, holding the Db, the Signal
 Worker, the Agent server and the Public server, together with the Messenger and its two Channels
-and Users with its two Auths. Outside it are the Agent Implementation, a person's client, and a
-Nostr Relay.](./site/public/architecture.svg)
+and Users with its two Auths. Outside it are the Agent Instance, a person's client, and a Nostr
+Relay.](./site/public/architecture.svg)
 
-Everything inside the dashed boundary is the Gateway. The
+Everything inside the dashed boundary is the Gateway. The agent is drawn outside it because it is
+outside it: you run the agent, and the Gateway connects to it over a TCP address that is the whole
+of what it knows about it. The
 [architecture page](https://shutter-network.github.io/concorde/architecture) reads the picture
 part by part.
 
@@ -82,16 +84,9 @@ import { createUsers } from "@shutter-network/concorde/users";
 const tokenTtl = 30 * 24 * 60 * 60 * 1000;
 
 const runtime = createPiRuntime({
-  image: process.env.AGENT_IMAGE!,
-  env: { AGENT_SERVER_URL: process.env.AGENT_SERVER_URL! },
-  networks: [process.env.AGENT_NETWORK!],
-  mounts: {
-    runtimeDir: process.env.RUNTIME_DIR_HOST!,
-    entries: [
-      { agentPath: "/workspace", path: "state/workspace" },
-      { agentPath: "/workspace/AGENTS.md", path: "AGENTS.md", readOnly: true },
-    ],
-  },
+  host: process.env.AGENT_INSTANCE_HOST!,
+  port: Number(process.env.AGENT_INSTANCE_PORT),
+  sessionsDir: process.env.AGENT_SESSIONS_DIR!,
 });
 
 const gateway = createGateway({
@@ -102,8 +97,8 @@ const gateway = createGateway({
     port: Number(process.env.PUBLIC_PORT),
   },
   agentListen: {
-    host: process.env.AGENT_HOST!,
-    port: Number(process.env.AGENT_PORT),
+    host: process.env.AGENT_SERVER_HOST!,
+    port: Number(process.env.AGENT_SERVER_PORT),
   },
   extend: ({ db, agentServer, publicServer, worker }) => {
     const users = createUsers({ db, agentServer, publicServer });
@@ -133,10 +128,16 @@ Answer them by sending them a Message. Your final reply here reaches nobody.`,
 await gateway.start();
 ```
 
-That is close to the whole of a working deployment. It waits for a message, and runs the agent in
-a container each time one arrives. What
-[`examples/00_minimal/main.ts`](./examples/00_minimal/main.ts) adds to it is the block that seeds
-the first person and a signal handler for shutdown.
+That is close to the whole of a working deployment. It waits for a message, and each time one
+arrives it opens a connection to the agent, prompts it, and closes the connection when the agent
+has settled. What [`examples/00_minimal/main.ts`](./examples/00_minimal/main.ts) adds to it is the
+block that seeds the first person and a signal handler for shutdown.
+
+**You run the agent; the framework does not.** `createPiRuntime` takes an address and the
+directory the agent keeps its Sessions in, and that is all it takes: no image, no model, no
+credential and no path on your host. The agent is `pi` behind a listener in a container of your
+own, on a private network with the Gateway, and every example's `compose.yml` is one service for
+each of the two.
 
 Each component is its own import subpath, and the package root exports nothing. A component that
 owns tables ships them on a `/schema` subpath, which your own `drizzle.config.ts` applies. Your

examples/00_minimal/.env.example

@@ -1,4 +1,6 @@
 # Copy this file to .env and fill in the one blank at the top.
+# The model credential, and the Agent Instance's alone: it is set on `agent` in `compose.yml`
+# and the Gateway is never given it.
 ANTHROPIC_API_KEY=
 
 # The seeded User's password, which `main.ts` sets from trusted code on the first boot and the

examples/00_minimal/Dockerfile

@@ -1,7 +1,5 @@
 FROM node:24-alpine
 
-RUN apk add --no-cache docker-cli
-
 WORKDIR /app
 
 COPY package.json ./

examples/00_minimal/Dockerfile.agent

@@ -1,10 +1,8 @@
 FROM node:24-alpine
 
-RUN apk add --no-cache curl
+RUN apk add --no-cache curl socat
 
-RUN npm install -g @earendil-works/pi-coding-agent@0.83.0
+RUN npm install -g @earendil-works/pi-coding-agent@0.85.1
 
 WORKDIR /workspace
 ENV PI_CODING_AGENT_DIR=/home/agent/.pi/agent
-
-ENTRYPOINT ["pi"]

examples/00_minimal/README.md

@@ -45,12 +45,85 @@ second, so the agent's answer arrives a moment after it is sent. Ctrl-C leaves.
 docker compose down -v
 ```
 
+## The two services
+
+This deployment is two containers that matter, and the Gateway is one of them. `gateway` serves
+the people and decides what the agent is asked; `agent` is the **Agent Instance**, which the
+Operator runs and the Gateway merely connects to. Nothing here mounts a container runtime
+socket and no host path is written anywhere, so the Gateway cannot start a container and has no
+name for a directory the Docker daemon would resolve. That is the whole arrangement: the
+Gateway holds a TCP address and nothing else about the agent.
+
+What `agent` starts is a listener rather than the agent:
+
+```yaml
+    command:
+      - socat
+      - TCP-LISTEN:4000,reuseaddr,fork
+      - EXEC:pi --mode rpc --no-approve
+```
+
+`fork` means one `pi` process per TCP connection, and the Gateway opens one connection per Run
+and closes it when the agent has settled. So a Run is still a process of its own, started for it
+and gone with it; what carries over from one Run to the next is this container and the
+directories mounted into it, which is the Operator's arrangement and nobody else's. Four things
+in those three lines are load-bearing:
+
+- **No commas may appear in the `EXEC:` argument.** Comma is `socat`'s own option separator, so
+  a comma anywhere in it is read as an option rather than as part of the command, and what runs
+  is quietly not what was written. The command is a list and not a string for the neighbouring
+  reason: no shell is involved, and `socat` splits `EXEC:` on spaces itself.
+- **`--no-approve` is the Operator's to pass**, and this line is the only place it now appears.
+  It is the flag that stops a Run arranging for the next one to load configuration out of the
+  writable Workspace, and the framework cannot fasten it to a command line it does not write.
+- **`PI_OFFLINE` was a framework default** back when the framework started the container. It is
+  an environment variable in this file now, and there is nowhere else left for it to be.
+- **`socat`'s `stderr` option must never be added.** `EXEC` wires stdin and stdout to the
+  socket, and `pi` writes its diagnostics to `socat`'s stderr; merging the two puts non-JSON
+  into the record stream and fails every Run.
+
+`Dockerfile.agent` is that container's image: `pi` at a pinned version, `socat` to listen with,
+and `curl`, because `pi` ships no HTTP client and reaching the Agent server is the agent's own
+shell tool plus `curl`. It declares no `ENTRYPOINT`, since the process this container starts is
+the listener and the agent is what the listener starts.
+
+**There is no healthcheck on `agent`, deliberately.** With `fork`, every connection starts a
+`pi`, so a probe on an interval would boot and discard one for ever. A plain `depends_on` is
+enough: an instance that is not listening is an ordinary failed Run carrying the address, not a
+boot failure, and a Gateway that refused to start over it would take every other Party's access
+down with the agent's.
+
+**The agent's environment is the Operator's, whole.** The image, the model credential, the files
+`pi` reads, the flags it is started with: every one of them is written on `agent` in
+`compose.yml`, and the Gateway is given none of it. `ANTHROPIC_API_KEY` is set there and in no
+other service. `AGENTS.md` and `settings.json` are mounted read-only out of this directory,
+which is how the agent learns that the Agent server exists and which model to talk to, and
+mounting them read-only is what keeps a Run from rewriting the instructions the next Run reads.
+
+**The Gateway is told three things about the instance and nothing else:**
+
+```yaml
+      AGENT_INSTANCE_HOST: agent
+      AGENT_INSTANCE_PORT: "4000"
+      AGENT_SESSIONS_DIR: /sessions
+```
+
+`AGENT_SERVER_HOST` and `AGENT_SERVER_PORT` beside them point the other way entirely: they are
+where the Gateway's own Agent server listens for the agent's `curl`. Two addresses cross on the
+same network and the names are long so that neither can be read as the other.
+
+`AGENT_SESSIONS_DIR` is **the path the Agent Instance sees**, and it is the same string as the
+target of the `./state/sessions:/sessions` mount on `agent`. It is written twice because nothing
+can check it once: one end is resolved by `pi` and the other by the Docker daemon, and neither
+can see the other's filesystem. A Session named `user_abc` is `/sessions/user_abc.jsonl` inside that
+container and `state/sessions/user_abc.jsonl` here, which is where to read a transcript.
+
 ## Look around
 
 - `main.ts` is the whole deployment: the Runtime, four components, one Handler, the prompt that
   Handler renders, and the seeding block.
 - The Gateway describes its own HTTP API, and the Public server is published at
   <http://127.0.0.1:8081/docs>. That is 8081 and not 8080, so this stack and the other examples
   can run at the same time.
-- `AGENTS.md` is mounted read-only into the agent's Workspace and is the only thing that tells
-  it the Agent server exists.
+- `AGENTS.md` is mounted read-only into the Agent Instance's Workspace and is the only thing
+  that tells it the Agent server exists.

examples/00_minimal/compose.yml

@@ -8,21 +8,15 @@ services:
       context: .
       dockerfile: Dockerfile
     environment:
-      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:?copy .env.example to .env and put your model key in it}
       DATABASE_URL: *database-url
       USER_PASSWORD: ${USER_PASSWORD:?copy .env.example to .env}
       PUBLIC_HOST: 0.0.0.0
       PUBLIC_PORT: "8081"
-      AGENT_HOST: 0.0.0.0
-      AGENT_PORT: "7411"
-      AGENT_SERVER_URL: http://gateway:7411
-      AGENT_IMAGE: concorde-minimal-agent:0.83.0
-      AGENT_NETWORK: concorde_minimal_agent
-      RUNTIME_DIR_HOST: ${PWD}
-    volumes:
-      - /var/run/docker.sock:/var/run/docker.sock
-      - ./state/workspace:/app/state/workspace
-      - ./state/agent:/app/state/agent
+      AGENT_SERVER_HOST: 0.0.0.0
+      AGENT_SERVER_PORT: "7411"
+      AGENT_INSTANCE_HOST: agent
+      AGENT_INSTANCE_PORT: "4000"
+      AGENT_SESSIONS_DIR: /sessions
     ports:
       - "127.0.0.1:8081:8081"
     networks: [db, agent, public]
@@ -31,10 +25,31 @@ services:
         condition: service_healthy
       migrate:
         condition: service_completed_successfully
-      agent-image:
-        condition: service_completed_successfully
+      agent:
+        condition: service_started
     stop_grace_period: 300s
 
+  agent:
+    build:
+      context: .
+      dockerfile: Dockerfile.agent
+    command:
+      - socat
+      - TCP-LISTEN:4000,reuseaddr,fork
+      - EXEC:pi --mode rpc --no-approve
+    environment:
+      ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:?copy .env.example to .env and put your model key in it}
+      AGENT_SERVER_URL: http://gateway:7411
+      PI_OFFLINE: "1"
+      PI_CODING_AGENT_DIR: /home/agent/.pi/agent
+    volumes:
+      - ./state/workspace:/workspace
+      - ./state/agent:/home/agent/.pi/agent
+      - ./state/sessions:/sessions
+      - ./AGENTS.md:/workspace/AGENTS.md:ro
+      - ./settings.json:/home/agent/.pi/agent/settings.json:ro
+    networks: [agent]
+
   migrate:
     build:
       context: .
@@ -63,15 +78,6 @@ services:
       timeout: 3s
       retries: 20
 
-  agent-image:
-    build:
-      context: .
-      dockerfile: Dockerfile.agent
-    image: concorde-minimal-agent:0.83.0
-    command: ["--version"]
-    restart: "no"
-    networks: [agent]
-
   tui:
     build:
       context: .

examples/00_minimal/main.ts

@@ -15,28 +15,19 @@ const password = process.env.USER_PASSWORD!;
 const tokenTtl = 30 * 24 * 60 * 60 * 1000;
 
 const runtime = createPiRuntime({
-  image: process.env.AGENT_IMAGE!,
-  env: {
-    ANTHROPIC_API_KEY: process.env.ANTHROPIC_API_KEY!,
-    AGENT_SERVER_URL: process.env.AGENT_SERVER_URL!,
-  },
-  networks: [process.env.AGENT_NETWORK!],
-  mounts: {
-    runtimeDir: process.env.RUNTIME_DIR_HOST!,
-    entries: [
-      { agentPath: "/workspace", path: "state/workspace" },
-      { agentPath: "/home/agent/.pi/agent", path: "state/agent" },
-      { agentPath: "/workspace/AGENTS.md", path: "AGENTS.md", readOnly: true },
-      { agentPath: "/home/agent/.pi/agent/settings.json", path: "settings.json", readOnly: true },
-    ],
-  },
+  host: process.env.AGENT_INSTANCE_HOST!,
+  port: Number(process.env.AGENT_INSTANCE_PORT),
+  sessionsDir: process.env.AGENT_SESSIONS_DIR!,
 });
 
 const gateway = createGateway({
   databaseUrl: process.env.DATABASE_URL!,
   runtime,
   publicListen: { host: process.env.PUBLIC_HOST!, port: Number(process.env.PUBLIC_PORT) },
-  agentListen: { host: process.env.AGENT_HOST!, port: Number(process.env.AGENT_PORT) },
+  agentListen: {
+    host: process.env.AGENT_SERVER_HOST!,
+    port: Number(process.env.AGENT_SERVER_PORT),
+  },
   extend: ({ db, agentServer, publicServer, worker }) => {
     const users = createUsers({ db, agentServer, publicServer });
     const passwordAuth = createPasswordAuth({ db, users, publicServer, tokenTtl });

examples/01_scheduler/.env.example

@@ -1,2 +1,4 @@
 # Copy this file to .env and fill in the one blank. Nothing else here needs a secret.
+# The model credential, and the Agent Instance's alone: it is set on `agent` in `compose.yml`
+# and the Gateway is never given it.
 ANTHROPIC_API_KEY=

examples/01_scheduler/AGENTS.md

@@ -21,8 +21,8 @@ fired and carries the `data` whoever created it supplied. There is no Messenger
 Channel in this deployment, so nobody is waiting on a reply and nothing you say leaves the
 Gateway.
 
-**Write what you did into your Workspace instead.** `/workspace` is a directory on the host
-that survives the container. Append a line, do not rewrite a file, and keep it short.
+**Write what you did into your Workspace instead.** `/workspace` is a mounted directory that
+outlives any one Run. Append a line, do not rewrite a file, and keep it short.
 
 ## Schedules of your own
 

examples/01_scheduler/Dockerfile

@@ -1,7 +1,5 @@
 FROM node:24-alpine
 
-RUN apk add --no-cache docker-cli
-
 WORKDIR /app
 
 COPY package.json ./

examples/01_scheduler/Dockerfile.agent

@@ -1,10 +1,8 @@
 FROM node:24-alpine
 
-RUN apk add --no-cache curl
+RUN apk add --no-cache curl socat
 
-RUN npm install -g @earendil-works/pi-coding-agent@0.83.0
+RUN npm install -g @earendil-works/pi-coding-agent@0.85.1
 
 WORKDIR /workspace
 ENV PI_CODING_AGENT_DIR=/home/agent/.pi/agent
-
-ENTRYPOINT ["pi"]