Protocol page: the checker's --probe mode, checking a server, and optional features

docs/protocol/index.md

@@ -141,34 +141,87 @@ Its report has two severities. A **violation** is something the real server
 would reject or mishandle. A **note** is something it accepts that differs
 from what the Python client does — a portability risk, not a breach.
 
-The checker watches one well-behaved connection, so it checks what that
-connection shows: framing, alternation, environment indices, observation
-shape, and the `feedback` payload's keys, dtypes and lengths. It does not
-check the rest of the SPEC.md §8 checklist, and a clause it did not exercise
-leaves no trace in its report. A client that breaks all of these still
-prints "no violations":
-
-- the connection options (compression off, no frame size cap);
-- reading `metadata` before sending anything;
-- the chunk-summed reward, and the terminal observation on a done step;
-- handling the close reasons `plugrl-server-stop` and `plugrl-server-resync`
-  differently, dropping held `feedback` on a reconnect, and treating a text
-  frame as fatal;
-- what the client does with the `action` it receives: the time-major layout
-  and reading `env_ids`.
-
-Two reference clients pass it. `raw_client.py` is 275 lines of Python using
-only `msgpack` and `websockets` — no numpy, nothing from PlugRL.
+By default the checker watches one connection, so it sees the messages and
+not what the client did with them: framing, alternation, environment
+indices, observation shape, and the `feedback` payload's keys, dtypes and
+lengths.
+
+To check the rest, have the client run the **probe environment** of
+[SPEC.md §8.1](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#81-the-probe-environment),
+a few lines in any language and no simulator, and add `--probe`:
+
+```bash
+uv run --extra conformance plugrl-conformance --probe --scenario all \
+    --client "./my_client --probe 127.0.0.1 8000"
+```
+
+The checker's actions then encode their own position, so each `feedback`
+says what the client did with the chunk: whether it summed the reward over
+the steps it ran, sent the terminal observation on a done step, and applied
+the actions time-major and in order. It also drives the connection instead
+of watching it, starting the client once per scenario:
+
+- `basic`: it speaks late, sends a `metadata` frame over 1 MiB with a key the
+  client has never seen, checks that compression was not offered, and ends
+  with `plugrl-server-stop`, after which the client must exit 0 and not
+  reconnect;
+- `resync`: it closes with `plugrl-server-resync` while the client holds a
+  `feedback`; the client must reconnect and open with an `infer`;
+- `text`: it answers with a text frame, which the client must treat as fatal.
+
+Four clauses of the §8 checklist stay unchecked even then, because a correct
+and an incorrect client look the same from the wire: reading `env_ids`,
+tolerating a `feedback` set unlike the `infer` set, requiring no particular
+`metadata` key, and dropping held `feedback` after a close other than a
+resync.
+
+Two reference clients pass every scenario, each with one note: they send
+`text` as a msgpack string array, the §3.4 Gap. `raw_client.py` is Python
+using only `msgpack` and `websockets` — no numpy, nothing from PlugRL.
 `plugrl_client.cpp` is C++17 with **no third-party libraries at all**: SHA-1,
 base64, WebSocket framing and the msgpack subset the protocol needs are all
 in the one file, because that is the situation an embedded controller is
-actually in.
+actually in. Both go through the checker, with `--probe`, in
+`plugrl-protocol`'s CI on every push to `main` and every pull request.
+
+## Checking a server
+
+The other direction is checked too. `plugrl-conformance-server` drives a
+training server the way SPEC.md lets a client behave, and checks what comes
+back: the action layout and `env_ids`, ragged batches, frames over 1 MiB, a
+resync close on each kind of malformed message with the server staying up
+for its other clients, and the stop at the end of the run
+([SPEC.md §8.2](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#82-checking-a-server)):
+
+```bash
+uv run --extra conformance plugrl-conformance-server --port 8000 --state-dim 3 --until-stop
+```
 
-Both go through the checker in `plugrl-protocol`'s CI on every push to `main`
-and every pull request. CI also sends the C++ client float64, time-major
-actions and checks from its printed output that it decoded them. That is a
-check of the C++ client, not something the checker can do for yours. Apart
-from it, nothing in the list above is checked for either client.
+`examples/reference_server.py` in `plugrl-protocol` is a server written
+against the specification that trains nothing, and passes. So does
+`plugrl-server`, whose CI runs the checker against it.
+
+## Optional features
+
+The protocol is version 1. The server sends that as `protocol_version` in
+`metadata`, and a client must not require the key. What changes is
+negotiated as **features**: the server lists the ones it implements in
+`metadata`'s `features`, a client uses one only when it is listed, and
+nothing changes for either side when it is absent. So an old client works
+with a new server, and the other way round
+([SPEC.md §10](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#10-versioning)).
+
+There is one so far, `reuse-feedback-obs`
+([§10.1](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#101-reuse-feedback-obs)).
+Without it every observation crosses the link twice: in the `feedback` that
+ends a chunk, and again in the next `infer`. With it the `infer` marks those
+rows `reuse` and leaves them out, so only the observation after a reset
+crosses twice. `plugrl-server` offers it and `plugrl-env-client` uses it by
+default. Between two machines it cut a step with a 184 KiB observation from
+18.5 to 11.6 ms, and training ended on byte-identical weights
+([E46](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e46-reuse-feedback-obs)).
+`plugrl-conformance --features reuse-feedback-obs` offers it to a client, and
+`plugrl-conformance-server` exercises it when the server lists it.
 
 ## Relationship to openpi
 

docs/protocol/index.zh.md

@@ -118,27 +118,68 @@ uv run --extra conformance plugrl-conformance --port 8000 --steps 20 \
 报告分两个等级。**violation** 是真服务端会拒绝或处理错的问题；**note** 是真服务端
 接受、但与 Python 客户端做法不同的地方 —— 是可移植性风险，不是违约。
 
-检验器看的是一条规规矩矩的连接，所以它只查这条连接看得到的东西：分帧、消息交替、
-环境编号、观测形状，以及 `feedback` 载荷的键、dtype 和长度。SPEC.md §8 清单里的其余
-条款它不查，而且没被触及的条款在报告里不留任何痕迹。下面这些全违反的客户端，照样
-打印 "no violations"：
-
-- 连接选项（关闭压缩、不限帧大小）；
-- 发任何东西之前先读 `metadata`；
-- 按动作块求和的奖励，以及结束那一步的终止观测；
-- 区别对待 `plugrl-server-stop` 和 `plugrl-server-resync` 两种关闭原因、重连时丢弃
-  手上的 `feedback`、把文本帧当致命错误；
-- 客户端拿到 `action` 之后怎么用：时间优先的布局，以及读 `env_ids`。
-
-两个参考客户端都能通过。`raw_client.py` 是 275 行 Python，只用 `msgpack` 和
-`websockets`，不用 numpy，也不用 PlugRL 的任何东西。`plugrl_client.cpp` 是 C++17，
-**完全不依赖第三方库**：SHA-1、base64、WebSocket 分帧，以及协议需要的那部分
-msgpack，全都写在同一个文件里 —— 因为嵌入式控制器面对的就是这种处境。
-
-`plugrl-protocol` 的 CI 在每次推到 `main` 和每个 pull request 上都让两者过一遍检验器。
-CI 还会给 C++ 客户端发 float64、时间优先的动作，再从它打印的输出核对它解对了。
-这是对 C++ 客户端的检查，检验器没法替你的客户端做。除此之外，上面清单里的各条对
-两个客户端都没有被检查。
+默认情况下，检验器旁观一条连接，所以它看得到消息，看不到客户端拿这些消息做了什么：
+它查分帧、消息交替、环境编号、观测形状，以及 `feedback` 载荷的键、dtype 和长度。
+
+其余的要让客户端跑 [SPEC.md §8.1](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#81-the-probe-environment)
+里的**探针环境**（任何语言几行就能写完，不需要模拟器），再加上 `--probe`：
+
+```bash
+uv run --extra conformance plugrl-conformance --probe --scenario all \
+    --client "./my_client --probe 127.0.0.1 8000"
+```
+
+这时检验器发出的动作值编码了它自己在动作块里的位置，所以每条 `feedback` 都说明了客户端
+怎么处理这个动作块：奖励有没有按实际跑的步数求和，结束那一步发的是不是终止观测，动作有没有
+按时间优先、按顺序执行。检验器也不再只是旁观，而是主动驱动连接，每个场景各启动一次客户端：
+
+- `basic`：它晚一点才开口，发一个超过 1 MiB、带着客户端没见过的键的 `metadata`，检查
+  握手没有提供压缩，最后用 `plugrl-server-stop` 结束，此后客户端必须以 0 退出、不再重连；
+- `resync`：在客户端手上还压着一条 `feedback` 时用 `plugrl-server-resync` 关闭连接，
+  客户端必须重连，并且新连接上的第一条消息必须是 `infer`；
+- `text`：它用一个文本帧作答，客户端必须把它当致命错误。
+
+即便如此，§8 清单里仍有四条查不了，因为从线上看，做对和做错的客户端表现一样：读
+`env_ids`、容忍 `feedback` 的环境集合和 `infer` 不同、不要求 `metadata` 里有特定的键，
+以及非 resync 关闭之后丢弃手上的 `feedback`。
+
+两个参考客户端都能通过全部场景，各有一条 note：它们把 `text` 发成 msgpack 字符串数组，
+也就是 §3.4 里的 Gap。`raw_client.py` 是只用 `msgpack` 和 `websockets` 的 Python，
+不用 numpy，也不用 PlugRL 的任何东西。`plugrl_client.cpp` 是 C++17，**完全不依赖第三方库**：
+SHA-1、base64、WebSocket 分帧，以及协议需要的那部分 msgpack，全都写在同一个文件里 ——
+因为嵌入式控制器面对的就是这种处境。`plugrl-protocol` 的 CI 在每次推到 `main` 和每个
+pull request 上，都让两者带着 `--probe` 过一遍检验器。
+
+## 检验训练端
+
+反方向也能检验。`plugrl-conformance-server` 按 SPEC.md 允许客户端的方式去驱动一个训练端，
+检查它的回应：动作布局和 `env_ids`、参差的批次、超过 1 MiB 的帧、遇到每一种格式错误的
+消息都以 resync 关闭且服务端对其他客户端照常工作，以及训练结束时的 stop
+（[SPEC.md §8.2](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#82-checking-a-server)）：
+
+```bash
+uv run --extra conformance plugrl-conformance-server --port 8000 --state-dim 3 --until-stop
+```
+
+`plugrl-protocol` 里的 `examples/reference_server.py` 是照着规范写的一个什么都不训练的
+训练端，能通过。`plugrl-server` 也能通过，它的 CI 每次都跑这个检验器。
+
+## 可选特性
+
+协议是第 1 版。服务端在 `metadata` 里以 `protocol_version` 发出版本号，客户端不得依赖
+这个键。变化通过**特性**协商：服务端在 `metadata` 的 `features` 里列出自己实现了哪些，
+客户端只用列出来的；某个特性不存在时，两边都和原来一样。所以旧客户端能连新服务端，反过来也行
+（[SPEC.md §10](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#10-versioning)）。
+
+目前只有一个特性：`reuse-feedback-obs`
+（[§10.1](https://github.com/PlugRL/plugrl-protocol/blob/main/SPEC.md#101-reuse-feedback-obs)）。
+没有它时，每个观测要过两次链路：一次在结束动作块的 `feedback` 里，一次在下一条 `infer` 里。
+有了它，`infer` 把这些行标成 `reuse` 并略去，只有重置后的那个观测还要过两次。
+`plugrl-server` 提供这个特性，`plugrl-env-client` 默认使用。跨两台机器时，它把带 184 KiB
+观测的一步从 18.5 毫秒降到 11.6 毫秒，训练出的权重逐字节相同
+（[E46](https://github.com/PlugRL/plugrl-server/tree/main/experiments/e46-reuse-feedback-obs)）。
+`plugrl-conformance --features reuse-feedback-obs` 会向客户端提供这个特性，
+`plugrl-conformance-server` 则在服务端列出它时去检验它。
 
 ## 与 openpi 的关系