From 32af99bd0ae9af37aee05cfa1792a6f265ef99eb Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 2 Oct 2026 09:55:59 -0400 Subject: [PATCH 1/2] E53: protocol, after the pilot and before the registered runs - with image observations and a CNN, does SB3 end on the same weights through the bridge --- .../e53-image-observations/PROTOCOL.md | 97 +++++++++++++ .../e53-image-observations/atari_client.py | 128 ++++++++++++++++++ .../e53-image-observations/results/pilot.txt | 23 ++++ experiments/e53-image-observations/run.sh | 22 +++ .../e53-image-observations/summarise.py | 75 ++++++++++ experiments/e53-image-observations/train.py | 124 +++++++++++++++++ 6 files changed, 469 insertions(+) create mode 100644 experiments/e53-image-observations/PROTOCOL.md create mode 100644 experiments/e53-image-observations/atari_client.py create mode 100644 experiments/e53-image-observations/results/pilot.txt create mode 100644 experiments/e53-image-observations/run.sh create mode 100644 experiments/e53-image-observations/summarise.py create mode 100644 experiments/e53-image-observations/train.py diff --git a/experiments/e53-image-observations/PROTOCOL.md b/experiments/e53-image-observations/PROTOCOL.md new file mode 100644 index 0000000..b3753d9 --- /dev/null +++ b/experiments/e53-image-observations/PROTOCOL.md @@ -0,0 +1,97 @@ +# E53 measurement protocol (pre-registered) + +**Written 2026-10-02, after the pilot in `results/pilot.txt`, before the +registered runs.** + +This file must not be edited after the first registered data point. Anything +learned afterwards goes in `AMENDMENT.md`, dated. + +--- + +## The question + +E50 found the boundary transparent on one machine: SB3 ends on byte-identical +weights with its environments in its own process or behind the bridge. But +only state vectors crossed. Image policies are where the boundary matters +most (VLAs; E43's and E46's costs), and a camera frame takes a different +path: uint8 arrays, an image observation space, a CNN, frame stacking. So: + +**With image observations and a CNN policy, does SB3 still end on the same +weights through the bridge? And what does crossing cost per step when every +step carries 16 frames?** + +--- + +## Declared in advance: what was already known + +1. **The code.** + - `train.py` follows E50's design. SB3 2.9.0 PPO runs CnnPolicy at + rl-baselines3-zoo's Atari settings, held constant: n_steps 128, + 4 epochs, minibatch 256, lr 2.5e-4, clip 0.1, entropy 0.01, value 0.5. + - There are 16 `AtariWrapper(gym.make("ALE/Pong-v5"))` environments, + with VecMonitor and VecFrameStack(4) on both arms. + - The run is 40,960 steps on CPU, four torch threads. + - The bridge arm uses plugrl-bridges#3 (`803141d`): + `PlugRLVecEnv(image_key="frame")`. + - `atari_client.py` sends each 84x84x1 uint8 frame as rendered. It + applies the float action that crosses as an integer, and it seeds and + resets as DummyVecEnv does. +2. **The versions.** ale-py 0.12.1 and opencv-python-headless 5.0.0.93 were + installed into bridges-venv for E53. +3. **The pilot, seed 9** (`results/pilot.txt`). + - Both arms ended on `76f991b8...`, with the observation space + `Box(0, 255, (84, 84, 4), uint8)` in both. + - The arms took 68.3 s in process and 74.5 s through the bridge. That is + 2.44 ms more per vector step (2,560 of them). + - An earlier attempt failed for want of OpenCV. Its stuck bridge process + then took a later attempt's client, and both were discarded + (`pilot.txt`). +4. **This is far too short to learn Pong.** E53 compares weights, not + returns. + +--- + +## Design + +On guangzhao, one run at a time, port 8830: seeds 0, 1 and 2, each with arms +inprocess and bridge. That is 6 runs (`run.sh`). The port is checked free +before each run. + +--- + +## Checks + +* **V1 - complete.** Every run has 40,960 steps. Every bridge run's client + exited 0, its log has one `plugrl-bridges: slots 0-15` line, and its + observation space is uint8. + +--- + +## Predictions, and what falsifies each + +**P1 - the boundary leaves the weights alone, with images too.** For each +seed, the two arms' weights are equal: 3 of 3. + +> Grounds: the pilot, and E50. Nothing in the data path differs. The frames +> are the same uint8 arrays, and SB3 applies its own image preprocessing +> (transpose, /255) in both arms alike. + +**P2 - and the episodes.** For each seed, the two `monitor.csv` files list +the same episodes, and the bridge's episode log agrees with the bridge +arm's. + +**P3 - the price of 16 frames a step, on one machine.** The bridge arm takes +1.0 to 5.0 ms more per vector step than its in-process pair, on every seed. + +> Grounds: the pilot's 2.44 ms. E50's state-only steps cost 0.37-0.58 ms. +> Each step now moves 16 x 7 KiB of frames twice, in the feedback and again +> in the next infer, since this client does not reuse observations. + +**Reported, not predicted:** each run's learn time and steps per second, and +the overhead as a fraction of the in-process time. + +--- + +## Reading order + +V1, P1, P2, P3, then the reported figures. diff --git a/experiments/e53-image-observations/atari_client.py b/experiments/e53-image-observations/atari_client.py new file mode 100644 index 0000000..77ecd0a --- /dev/null +++ b/experiments/e53-image-observations/atari_client.py @@ -0,0 +1,128 @@ +"""E53's env client: N Atari environments behind SB3's AtariWrapper, on one connection. + + python atari_client.py --port 8830 --num-envs 16 --seed 0 [--env-id ALE/Pong-v5] + +E50's gym_client.py, with three changes: +- Each env is `AtariWrapper(gym.make(env_id))`, as SB3 preprocesses Atari: + no-op starts, frame skip 4, 84x84 grayscale, a life lost ends the episode, + and rewards clipped to their sign. +- The observation it sends is that frame, a uint8 [n, 84, 84, 1] image + named `frame`, with no states. +- The action arrives as one float per env (the protocol's actions are + float), and is applied as the integer it is. + +Env i is reset with seed + i first and with no seed afterwards, as SB3's +DummyVecEnv does after `VecEnv.seed(seed)`. An env whose episode ended is +reset before the next infer, as DummyVecEnv resets it inside the step. +""" + +from __future__ import annotations + +import argparse +import time + +import ale_py +import gymnasium as gym +import numpy as np +import websockets.exceptions +import websockets.sync.client +from stable_baselines3.common.atari_wrappers import AtariWrapper + +from plugrl_protocol import msgpack_numpy + +gym.register_envs(ale_py) +STOP_REASON = "plugrl-server-stop" + + +def observation(frames: np.ndarray) -> dict: + return {"images": {"frame": frames}, "states": {}, "text": "atari"} + + +def connect(uri: str, deadline: float): + while True: + try: + return websockets.sync.client.connect( + uri, compression=None, max_size=None, open_timeout=10 + ) + except (OSError, websockets.exceptions.InvalidHandshake): + if time.monotonic() > deadline: + raise + time.sleep(0.1) + + +def main() -> int: + p = argparse.ArgumentParser(description=__doc__) + p.add_argument("--host", default="127.0.0.1") + p.add_argument("--port", type=int, default=8830) + p.add_argument("--num-envs", type=int, default=16) + p.add_argument("--seed", type=int, default=0) + p.add_argument("--env-id", default="ALE/Pong-v5") + a = p.parse_args() + + n = a.num_envs + envs = [AtariWrapper(gym.make(a.env_id)) for _ in range(n)] + obs = np.stack([env.reset(seed=a.seed + i)[0] for i, env in enumerate(envs)]) + indices = np.arange(n, dtype=np.int64) + step_ids = np.zeros(n, dtype=np.int64) + packer = msgpack_numpy.Packer() + steps = 0 + + ws = connect(f"ws://{a.host}:{a.port}", time.monotonic() + 120) + with ws: + msgpack_numpy.unpackb(ws.recv()) # the metadata; nothing in it is needed + try: + while True: + ws.send( + packer.pack( + { + "message_type": "infer", + "data": observation(obs), + "env_indices": indices, + "step_ids": step_ids.copy(), + } + ) + ) + reply = msgpack_numpy.unpackb(ws.recv()) + actions = np.asarray(reply["data"]["action"])[0] # [H, n, 1] -> [n, 1] + ended = np.empty_like(obs) + after = np.empty_like(obs) + rewards = np.zeros(n, dtype=np.float32) + terminated = np.zeros(n, dtype=np.bool_) + truncated = np.zeros(n, dtype=np.bool_) + sent_ids = step_ids.copy() + for i, env in enumerate(envs): + o, r, te, tr, _ = env.step(int(round(float(actions[i][0])))) + ended[i] = o + rewards[i], terminated[i], truncated[i] = r, te, tr + if te or tr: + after[i] = env.reset()[0] + step_ids[i] = 0 + else: + after[i] = o + step_ids[i] += 1 + ws.send( + packer.pack( + { + "message_type": "feedback", + "env_indices": indices, + "step_ids": sent_ids, + "data": { + "obs": observation(ended), + "rewards": rewards, + "terminated": terminated, + "truncated": truncated, + "info": {}, + }, + } + ) + ) + obs = after + steps += 1 + except websockets.exceptions.ConnectionClosed as closed: + reason = closed.rcvd.reason if closed.rcvd is not None else "" + print(f"closed after {steps} steps: {reason!r}", flush=True) + return 0 if reason == STOP_REASON else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/e53-image-observations/results/pilot.txt b/experiments/e53-image-observations/results/pilot.txt new file mode 100644 index 0000000..7a59114 --- /dev/null +++ b/experiments/e53-image-observations/results/pilot.txt @@ -0,0 +1,23 @@ +E53 pilot, 2026-10-02, seed 9 (not a registered seed), guangzhao, bridges-venv +(torch 2.14.1+cpu, SB3 2.9.0, gymnasium 1.3.0, ale-py 0.12.1, then +opencv-python-headless 5.0.0.93), plugrl-bridges#3 (803141d). + +First attempt: both arms failed. SB3's AtariWrapper needs OpenCV to resize +frames, and bridges-venv had none; the client died with "OpenCV is not +installed". OpenCV was installed (opencv-python-headless 5.0.0.93). + +Second attempt: + inprocess 76f991b80e896bd88068d6e1507bf05845710580b40a05cb375b5dcf622edddc 68.284 s 599.8 steps/s + bridge failed to bind port 8830: the first attempt's bridge arm was still + running, waiting (timeout 600 s) for the client that had died. That + stuck process then accepted the second attempt's client. Both + processes were killed and their output deleted, unread. + +Third attempt, bridge arm only, port 8830 checked free first: + bridge 76f991b80e896bd88068d6e1507bf05845710580b40a05cb375b5dcf622edddc 74.524 s 549.6 steps/s client rc 0 + observation space in both: Box(0, 255, (84, 84, 4), uint8) + +Same weights. (74.524 - 68.284) / 2560 = 2.44 ms more per vector step. + +Before the registered runs: run.sh checks that port 8830 is free before every +run. Nothing else was changed. diff --git a/experiments/e53-image-observations/run.sh b/experiments/e53-image-observations/run.sh new file mode 100644 index 0000000..efc8214 --- /dev/null +++ b/experiments/e53-image-observations/run.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +# E53's 6 registered runs on guangzhao, one at a time (PROTOCOL.md). +# bash run.sh OUT_DIR +set -uo pipefail +OUT="${1:?OUT_DIR}" +export PATH="$HOME/.local/bin:$PATH" OMP_NUM_THREADS=1 +PY="$HOME/zuogou/plugrl/bridges-venv/bin/python" +cd "$(dirname "$0")" +mkdir -p "$OUT" +uptime | tee -a "$OUT/load.txt" +(cd "$HOME/zuogou/plugrl/plugrl-bridges" && find src -name "*.py" | sort | xargs sha256sum) > "$OUT/bridges-src.sha256" +for seed in 0 1 2; do + for arm in inprocess bridge; do + name="pong-$arm-seed$seed" + if ss -ltn | grep -q ":8830 "; then echo "port 8830 busy before $name; stopping"; exit 1; fi + echo "== $name $(date +%T)" + "$PY" train.py --arm "$arm" --seed "$seed" --out "$OUT/$name" --port 8830 > "$OUT/$name.log" 2>&1 + tail -n 1 "$OUT/$name.log" + done +done +uptime | tee -a "$OUT/load.txt" +echo E53_DONE diff --git a/experiments/e53-image-observations/summarise.py b/experiments/e53-image-observations/summarise.py new file mode 100644 index 0000000..64a0f2b --- /dev/null +++ b/experiments/e53-image-observations/summarise.py @@ -0,0 +1,75 @@ +"""E53: judge PROTOCOL.md's checks and predictions from a results directory. + + python summarise.py [DIR] [LOW_MS HIGH_MS] # default: results/, P3's band + +Expects DIR/pong--seed/ with result.json and monitor.csv (and, for the +bridge arm, episodes.csv), plus DIR/pong-bridge-seed.log. +""" + +from __future__ import annotations + +import csv +import json +import pathlib +import sys + +HERE = pathlib.Path(__file__).resolve().parent +SEEDS = (0, 1, 2) +VECTOR_STEPS = 40_960 // 16 + + +def monitor(run: pathlib.Path) -> list[tuple[str, str]]: + with open(run / "monitor.csv", encoding="utf-8") as f: + next(f) # VecMonitor's JSON header + return [(r["r"], r["l"]) for r in csv.DictReader(f)] + + +def episode_log(run: pathlib.Path) -> list[tuple[float, int]]: + with open(run / "episodes.csv", encoding="utf-8", newline="") as f: + return [(float(r["return"]), int(r["length"])) for r in csv.DictReader(f)] + + +def main() -> int: + args = sys.argv[1:] + root = pathlib.Path(args[0]) if args else HERE / "results" + low, high = (float(args[1]), float(args[2])) if len(args) == 3 else (0.0, 1e9) + v1 = p1 = p2 = p3 = True + for seed in SEEDS: + a, b = (root / f"pong-{arm}-seed{seed}" for arm in ("inprocess", "bridge")) + ra, rb = (json.loads((r / "result.json").read_text()) for r in (a, b)) + log = (root / f"pong-bridge-seed{seed}.log").read_text( + encoding="utf-8", errors="replace" + ) + v1 &= ( + ra["steps"] == rb["steps"] == 40_960 + and rb.get("client_rc") == 0 + and log.count("plugrl-bridges: slots 0-15 <-") == 1 + and "uint8" in rb["observation_space"] + ) + same = ra["weights_sha256"] == rb["weights_sha256"] + ma, mb = monitor(a), monitor(b) + own = episode_log(b) + agrees = len(own) == len(mb) and all( + abs(r - float(mr)) < 1e-4 * max(1.0, abs(r)) and n == int(ml) + for (r, n), (mr, ml) in zip(own, mb) + ) + extra = (rb["learn_s"] - ra["learn_s"]) / VECTOR_STEPS * 1000 + p1 &= same + p2 &= ma == mb and agrees + p3 &= low <= extra <= high + print( + f"seed {seed}: weights {'same' if same else 'DIFFER'} " + f"({ra['weights_sha256'][:12]} / {rb['weights_sha256'][:12]}), " + f"{len(ma)} episodes {'same' if ma == mb else 'DIFFER'}, " + f"episode log {'agrees' if agrees else 'DISAGREES'}; " + f"{ra['learn_s']:.1f} s vs {rb['learn_s']:.1f} s " + f"(+{rb['learn_s'] / ra['learn_s'] - 1:.0%}, {extra:.2f} ms per vector step)" + ) + print() + for name, ok in (("V1", v1), ("P1", p1), ("P2", p2), ("P3", p3)): + print(f"{name}: {'holds' if ok else 'FAILS'}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/experiments/e53-image-observations/train.py b/experiments/e53-image-observations/train.py new file mode 100644 index 0000000..2b79acc --- /dev/null +++ b/experiments/e53-image-observations/train.py @@ -0,0 +1,124 @@ +"""E53: SB3's PPO with a CNN on 16 Atari environments, in its own process or behind the bridge. + + python train.py --arm inprocess --seed 0 --out runs/... + python train.py --arm bridge --seed 0 --out runs/... [--port 8830] + +E50's design with image observations. The arms differ only in where the 16 +`AtariWrapper(gym.make("ALE/Pong-v5"))` environments are stepped: +- inprocess: in SB3's DummyVecEnv, in this process; +- bridge: in atari_client.py, another process, on one connection to + plugrl-bridges' `PlugRLVecEnv(image_key="frame")`, which sends each + 84x84x1 uint8 frame as it was rendered. + +Everything else is one code path: VecMonitor, VecFrameStack(4), then PPO with +CnnPolicy at rl-baselines3-zoo's Atari settings (n_steps 128, 4 epochs, +minibatch 256, lr 2.5e-4, clip 0.1, entropy 0.01, value 0.5), held constant +rather than annealed. 40,960 steps (20 rollouts of 16 x 128), four torch +threads, CPU. This is far too short to learn Pong; E53 compares weights, not +returns. result.json gets the SHA-256 of the final policy and the time +model.learn() took. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import pathlib +import subprocess +import sys +import time + +import ale_py +import gymnasium as gym +import torch +from stable_baselines3 import PPO +from stable_baselines3.common.atari_wrappers import AtariWrapper +from stable_baselines3.common.vec_env import DummyVecEnv, VecFrameStack, VecMonitor + +gym.register_envs(ale_py) +HERE = pathlib.Path(__file__).resolve().parent +ENV_ID = "ALE/Pong-v5" +NUM_ENVS = 16 +TOTAL = 40_960 +THREADS = 4 +SETTINGS = dict( + n_steps=128, + n_epochs=4, + batch_size=256, + learning_rate=2.5e-4, + clip_range=0.1, + ent_coef=0.01, + vf_coef=0.5, +) + + +def weights_sha256(model: PPO) -> str: + h = hashlib.sha256() + for name, tensor in sorted(model.policy.state_dict().items()): + h.update(name.encode()) + h.update(tensor.detach().cpu().contiguous().numpy().tobytes()) + return h.hexdigest() + + +def main() -> int: + p = argparse.ArgumentParser(description=__doc__) + p.add_argument("--arm", choices=["inprocess", "bridge"], required=True) + p.add_argument("--seed", type=int, required=True) + p.add_argument("--out", required=True) + p.add_argument("--port", type=int, default=8830) + a = p.parse_args() + out = pathlib.Path(a.out).resolve() + out.mkdir(parents=True, exist_ok=True) + torch.set_num_threads(THREADS) + + client = None + if a.arm == "inprocess": + env = DummyVecEnv([lambda: AtariWrapper(gym.make(ENV_ID))] * NUM_ENVS) + else: + from plugrl_bridges.sb3 import PlugRLVecEnv + + client_log = open(out / "client.log", "w", encoding="utf-8") + client = subprocess.Popen( + [sys.executable, str(HERE / "atari_client.py"), "--port", str(a.port), + "--num-envs", str(NUM_ENVS), "--seed", str(a.seed), "--env-id", ENV_ID], + stdout=client_log, stderr=subprocess.STDOUT, + ) # fmt: skip + actions = gym.make(ENV_ID).action_space + env = PlugRLVecEnv( + num_envs=NUM_ENVS, + action_space=actions, + port=a.port, + host="127.0.0.1", + image_key="frame", + episode_log=str(out / "episodes.csv"), + ) + env = VecMonitor(env, filename=str(out / "monitor.csv")) + env = VecFrameStack(env, n_stack=4) + # PPO(seed=) seeds torch and numpy, and calls env.seed(seed), which makes + # DummyVecEnv reset env i with seed + i. atari_client.py does the same. + model = PPO("CnnPolicy", env, seed=a.seed, device="cpu", verbose=0, **SETTINGS) + start = time.perf_counter() + model.learn(total_timesteps=TOTAL) + learn_s = time.perf_counter() - start + result = { + "env": ENV_ID, + "arm": a.arm, + "seed": a.seed, + "weights_sha256": weights_sha256(model), + "learn_s": round(learn_s, 3), + "steps": model.num_timesteps, + "steps_per_s": round(model.num_timesteps / learn_s, 1), + "observation_space": str(env.observation_space), + "torch": torch.__version__, + } + env.close() + if client is not None: + result["client_rc"] = client.wait(timeout=60) + (out / "result.json").write_text(json.dumps(result, indent=2) + "\n") + print(json.dumps(result)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 23f12de292bb744dcb697f1d06333ada8bb7e19c Mon Sep 17 00:00:00 2001 From: tactino <18781106300@163.com> Date: Fri, 2 Oct 2026 10:04:42 -0400 Subject: [PATCH 2/2] E53: findings - with camera frames and a CNN, SB3 still ends on byte-identical weights through the bridge; 16 frames a step cost 2.6-3.0 ms SB3 PPO CnnPolicy, 16 AtariWrapper(ALE/Pong-v5) envs, 40,960 steps, three seeds: same weights 3/3, same episodes, the bridge episode log agrees; +2.60-2.99 ms per vector step (10-11%). Every check and prediction holds. --- .../e53-image-observations/FINDINGS.md | 64 ++++++ .../results/bridges-src.sha256 | 6 + .../e53-image-observations/results/load.txt | 2 + .../results/pong-bridge-seed0.log | 4 + .../results/pong-bridge-seed0/client.log | 3 + .../results/pong-bridge-seed0/episodes.csv | 184 +++++++++++++++++ .../results/pong-bridge-seed0/monitor.csv | 185 ++++++++++++++++++ .../results/pong-bridge-seed0/result.json | 12 ++ .../results/pong-bridge-seed1.log | 4 + .../results/pong-bridge-seed1/client.log | 3 + .../results/pong-bridge-seed1/episodes.csv | 177 +++++++++++++++++ .../results/pong-bridge-seed1/monitor.csv | 178 +++++++++++++++++ .../results/pong-bridge-seed1/result.json | 12 ++ .../results/pong-bridge-seed2.log | 4 + .../results/pong-bridge-seed2/client.log | 3 + .../results/pong-bridge-seed2/episodes.csv | 184 +++++++++++++++++ .../results/pong-bridge-seed2/monitor.csv | 185 ++++++++++++++++++ .../results/pong-bridge-seed2/result.json | 12 ++ .../results/pong-inprocess-seed0.log | 3 + .../results/pong-inprocess-seed0/monitor.csv | 185 ++++++++++++++++++ .../results/pong-inprocess-seed0/result.json | 11 ++ .../results/pong-inprocess-seed1.log | 3 + .../results/pong-inprocess-seed1/monitor.csv | 178 +++++++++++++++++ .../results/pong-inprocess-seed1/result.json | 11 ++ .../results/pong-inprocess-seed2.log | 3 + .../results/pong-inprocess-seed2/monitor.csv | 185 ++++++++++++++++++ .../results/pong-inprocess-seed2/result.json | 11 ++ .../results/registered.out | 15 ++ .../results/verdicts.txt | 8 + 29 files changed, 1835 insertions(+) create mode 100644 experiments/e53-image-observations/FINDINGS.md create mode 100644 experiments/e53-image-observations/results/bridges-src.sha256 create mode 100644 experiments/e53-image-observations/results/load.txt create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed0.log create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed0/client.log create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed0/episodes.csv create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed0/monitor.csv create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed0/result.json create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed1.log create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed1/client.log create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed1/episodes.csv create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed1/monitor.csv create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed1/result.json create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed2.log create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed2/client.log create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed2/episodes.csv create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed2/monitor.csv create mode 100644 experiments/e53-image-observations/results/pong-bridge-seed2/result.json create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed0.log create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed0/monitor.csv create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed0/result.json create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed1.log create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed1/monitor.csv create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed1/result.json create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed2.log create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed2/monitor.csv create mode 100644 experiments/e53-image-observations/results/pong-inprocess-seed2/result.json create mode 100644 experiments/e53-image-observations/results/registered.out create mode 100644 experiments/e53-image-observations/results/verdicts.txt diff --git a/experiments/e53-image-observations/FINDINGS.md b/experiments/e53-image-observations/FINDINGS.md new file mode 100644 index 0000000..eec472e --- /dev/null +++ b/experiments/e53-image-observations/FINDINGS.md @@ -0,0 +1,64 @@ +# E53: with camera frames and a CNN, SB3 still ends on byte-identical weights through the bridge; 16 frames a step cost 2.6-3.0 ms + +2026-10-02 · guangzhao (bridges-venv: torch 2.14.1+cpu, SB3 2.9.0, gymnasium +1.3.0, ale-py 0.12.1, opencv-python-headless 5.0.0.93; plugrl-bridges#3, +803141d) · protocol: [`PROTOCOL.md`](PROTOCOL.md) (`32af99b`, after the pilot +in `results/pilot.txt`, before the registered runs) · no amendment + +--- + +## The result + +SB3's PPO with CnnPolicy trained 16 `AtariWrapper(ALE/Pong-v5)` environments +for 40,960 steps, with 84x84 grayscale frames, stacked 4 deep. The two arms: +- **inprocess:** in SB3's DummyVecEnv; +- **bridge:** in another process, behind + `PlugRLVecEnv(image_key="frame")`, which receives each uint8 frame as + rendered. + +| seed | final weights (SHA-256) | episodes | in-process | bridge | per vector step | +| --- | --- | --- | --- | --- | --- | +| 0 | `82f80c02...` both | 183, identical | 67.4 s | 75.0 s (+11%) | +2.99 ms | +| 1 | `60acba42...` both | 176, identical | 69.6 s | 76.2 s (+10%) | +2.60 ms | +| 2 | `6bf04095...` both | 183, identical | 68.1 s | 75.8 s (+11%) | +2.98 ms | + +Every check and prediction holds (`results/verdicts.txt`): +- **V1:** all 6 runs are complete. Each bridge run served one connection, and + its observation space was `Box(0, 255, (84, 84, 4), uint8)`, as in process. +- **P1:** 3 of 3 seeds end on the same weights. +- **P2:** both arms logged the same episodes, and the bridge's own episode + log agrees. (AtariWrapper ends an episode at each lost life.) +- **P3:** the bridge adds 2.60-2.99 ms per vector step, inside 1.0-5.0. + +--- + +## What it means + +- **The boundary stays transparent with images.** E50 showed this for state + vectors. Here the observations are camera frames: a uint8 image space, a + CNN, SB3's own transposition and scaling, and frame stacking on the + trainer's side. Nothing in that path changes when the frames cross a + process boundary. +- **Images cost more to cross, in proportion to their bytes.** Each step + moves 16 frames of 7 KiB twice: in the feedback, and again in the next + infer, since this client does not use `reuse-feedback-obs`. That is about + 2.8 ms on the loopback, against 0.4-0.6 ms for E50's states. It is still + 10-11% of a short CNN run on CPU, and a far smaller share beside a VLA's + forward pass (E10). + +--- + +## What E53 does not show + +* **Learning.** 40,960 steps is far too short to learn Pong. E53 compares + weights. E49 shows learning through the bridge, with state observations. +* **A GPU.** The CNN ran on CPU with four threads, in both arms. On a GPU, + bit-identical comparisons need deterministic kernels. +* **Two machines, or reuse.** E43 and E46 measured what frames cost across a + link, and how `reuse-feedback-obs` halves it. + +Kept here: +- each run's `result.json`, `monitor.csv`, the bridge runs' `episodes.csv`, + and the logs; +- `registered.out`, `load.txt`, `bridges-src.sha256` and `verdicts.txt` + (`summarise.py`). diff --git a/experiments/e53-image-observations/results/bridges-src.sha256 b/experiments/e53-image-observations/results/bridges-src.sha256 new file mode 100644 index 0000000..755002f --- /dev/null +++ b/experiments/e53-image-observations/results/bridges-src.sha256 @@ -0,0 +1,6 @@ +766ad3d83be9696dff26543ce2fb35c3c31cb845fc42cb148d411b263e8bf9b5 src/plugrl_bridges/gym.py +e89ebb2c4ed69ed5788cd945bc4d36ecc023949627433818b98d9ea43d1fa7d5 src/plugrl_bridges/__init__.py +4001b6400d4031504525949fa721ad425fa0cb16a91cf30537dfd6d5859d8336 src/plugrl_bridges/observations.py +1c22fca6e0f01e8ec730ee893568cc301d2d21c3e16cc2102c22959ced00ff08 src/plugrl_bridges/rlinf.py +f02264e551925eed72ba2e4dfb5159f7e174e4ac7231aa122689ef2172484646 src/plugrl_bridges/sb3.py +0e56388fea17a73d4defcb437d3a4b6c8392c609dfca07efe92343e6cac72656 src/plugrl_bridges/server.py diff --git a/experiments/e53-image-observations/results/load.txt b/experiments/e53-image-observations/results/load.txt new file mode 100644 index 0000000..b643bfa --- /dev/null +++ b/experiments/e53-image-observations/results/load.txt @@ -0,0 +1,2 @@ + 09:56:03 up 27 days, 22:57, 1 user, load average: 0.96, 1.31, 0.93 + 10:03:32 up 27 days, 23:05, 1 user, load average: 3.27, 3.00, 1.94 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed0.log b/experiments/e53-image-observations/results/pong-bridge-seed0.log new file mode 100644 index 0000000..3a84839 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed0.log @@ -0,0 +1,4 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +plugrl-bridges: slots 0-15 <- ('127.0.0.1', 34864) +{"env": "ALE/Pong-v5", "arm": "bridge", "seed": 0, "weights_sha256": "82f80c02616bc24c4f2baa984af8162e0f94532e1f4c3a3ff49fb977ae1b489f", "learn_s": 75.04, "steps": 40960, "steps_per_s": 545.8, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu", "client_rc": 0} diff --git a/experiments/e53-image-observations/results/pong-bridge-seed0/client.log b/experiments/e53-image-observations/results/pong-bridge-seed0/client.log new file mode 100644 index 0000000..78f0b06 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed0/client.log @@ -0,0 +1,3 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +closed after 2560 steps: 'plugrl-server-stop' diff --git a/experiments/e53-image-observations/results/pong-bridge-seed0/episodes.csv b/experiments/e53-image-observations/results/pong-bridge-seed0/episodes.csv new file mode 100644 index 0000000..8d240f0 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed0/episodes.csv @@ -0,0 +1,184 @@ +step,slot,episode,return,length +183,15,1,-21.000000,183 +186,12,1,-21.000000,186 +189,0,1,-21.000000,189 +192,5,1,-21.000000,192 +196,4,1,-21.000000,196 +196,6,1,-21.000000,196 +202,11,1,-21.000000,202 +204,2,1,-21.000000,204 +205,9,1,-21.000000,205 +215,13,1,-21.000000,215 +220,1,1,-21.000000,220 +236,3,1,-21.000000,236 +250,10,1,-21.000000,250 +283,14,1,-20.000000,283 +286,8,1,-19.000000,286 +295,7,1,-19.000000,295 +370,12,2,-21.000000,184 +383,15,2,-21.000000,200 +391,2,2,-21.000000,187 +409,0,2,-21.000000,220 +410,13,2,-21.000000,195 +411,5,2,-20.000000,219 +417,4,2,-20.000000,221 +418,9,2,-21.000000,213 +422,6,2,-21.000000,226 +431,1,2,-21.000000,211 +437,3,2,-21.000000,201 +438,11,2,-20.000000,236 +450,10,2,-21.000000,200 +498,8,2,-20.000000,212 +501,14,2,-20.000000,218 +524,7,2,-21.000000,229 +593,12,3,-20.000000,223 +597,5,3,-21.000000,186 +613,0,3,-21.000000,204 +618,1,3,-21.000000,187 +623,15,3,-20.000000,240 +625,2,3,-20.000000,234 +630,4,3,-20.000000,213 +637,10,3,-21.000000,187 +637,11,3,-21.000000,199 +653,6,3,-21.000000,231 +653,9,3,-20.000000,235 +653,13,3,-20.000000,243 +655,3,3,-21.000000,218 +701,8,3,-20.000000,203 +707,14,3,-21.000000,206 +735,7,3,-21.000000,211 +794,5,4,-21.000000,197 +812,0,4,-21.000000,199 +832,12,4,-21.000000,239 +836,11,4,-21.000000,199 +854,2,4,-21.000000,229 +858,9,4,-21.000000,205 +861,15,4,-20.000000,238 +863,4,4,-21.000000,233 +874,10,4,-20.000000,237 +876,13,4,-21.000000,223 +881,1,4,-19.000000,263 +895,3,4,-21.000000,240 +909,6,4,-20.000000,256 +926,14,4,-20.000000,219 +930,7,4,-21.000000,195 +955,8,4,-20.000000,254 +1002,5,5,-21.000000,208 +1043,12,5,-20.000000,211 +1049,0,5,-21.000000,237 +1055,4,5,-21.000000,192 +1056,11,5,-21.000000,220 +1058,9,5,-21.000000,200 +1061,10,5,-21.000000,187 +1077,2,5,-20.000000,223 +1077,13,5,-21.000000,201 +1108,1,5,-21.000000,227 +1110,15,5,-19.000000,249 +1111,3,5,-20.000000,216 +1127,14,5,-20.000000,201 +1132,6,5,-20.000000,223 +1150,8,5,-21.000000,195 +1174,7,5,-21.000000,244 +1204,5,6,-21.000000,202 +1239,0,6,-21.000000,190 +1241,4,6,-21.000000,186 +1243,10,6,-21.000000,182 +1244,12,6,-21.000000,201 +1282,11,6,-21.000000,226 +1283,9,6,-20.000000,225 +1285,13,6,-21.000000,208 +1287,2,6,-21.000000,210 +1325,3,6,-21.000000,214 +1328,1,6,-21.000000,220 +1328,15,6,-21.000000,218 +1334,8,6,-21.000000,184 +1363,14,6,-20.000000,236 +1375,6,6,-19.000000,243 +1384,7,6,-21.000000,210 +1424,5,7,-20.000000,220 +1447,0,7,-21.000000,208 +1447,12,7,-21.000000,203 +1464,4,7,-21.000000,223 +1483,10,7,-20.000000,240 +1488,9,7,-21.000000,205 +1493,11,7,-21.000000,211 +1517,2,7,-20.000000,230 +1525,13,7,-20.000000,240 +1538,3,7,-21.000000,213 +1563,1,7,-20.000000,235 +1571,14,7,-21.000000,208 +1575,15,7,-21.000000,247 +1593,7,7,-21.000000,209 +1602,6,7,-20.000000,227 +1619,5,8,-21.000000,195 +1647,8,7,-19.000000,313 +1649,12,8,-21.000000,202 +1673,0,8,-21.000000,226 +1673,9,8,-21.000000,185 +1680,4,8,-21.000000,216 +1723,10,8,-21.000000,240 +1723,11,8,-21.000000,230 +1734,13,8,-20.000000,209 +1738,2,8,-21.000000,221 +1762,1,8,-21.000000,199 +1774,3,8,-21.000000,236 +1778,7,8,-21.000000,185 +1791,15,8,-21.000000,216 +1805,6,8,-21.000000,203 +1817,5,9,-21.000000,198 +1818,14,8,-21.000000,247 +1855,8,8,-21.000000,208 +1882,12,9,-19.000000,233 +1898,9,9,-20.000000,225 +1899,0,9,-21.000000,226 +1909,10,9,-21.000000,186 +1916,11,9,-21.000000,193 +1922,4,9,-21.000000,242 +1924,2,9,-21.000000,186 +1966,1,9,-21.000000,204 +1998,7,9,-20.000000,220 +1998,13,9,-20.000000,264 +2001,3,9,-20.000000,227 +2011,15,9,-21.000000,220 +2021,5,10,-21.000000,204 +2027,6,9,-21.000000,222 +2039,8,9,-21.000000,184 +2064,14,9,-20.000000,246 +2088,12,10,-20.000000,206 +2096,9,10,-21.000000,198 +2099,11,10,-21.000000,183 +2100,0,10,-21.000000,201 +2108,2,10,-21.000000,184 +2120,10,10,-21.000000,211 +2122,4,10,-21.000000,200 +2189,13,10,-21.000000,191 +2206,7,10,-20.000000,208 +2216,1,10,-19.000000,250 +2230,6,10,-21.000000,203 +2244,8,10,-21.000000,205 +2268,15,10,-19.000000,257 +2275,3,10,-20.000000,274 +2275,5,11,-19.000000,254 +2276,12,11,-21.000000,188 +2282,11,11,-21.000000,183 +2295,14,10,-21.000000,231 +2303,9,11,-21.000000,207 +2317,0,11,-21.000000,217 +2320,2,11,-21.000000,212 +2328,4,11,-21.000000,206 +2341,10,11,-21.000000,221 +2413,1,11,-21.000000,197 +2421,7,11,-21.000000,215 +2424,13,11,-20.000000,235 +2447,8,11,-20.000000,203 +2451,6,11,-21.000000,221 +2472,3,11,-21.000000,197 +2473,12,12,-21.000000,197 +2488,9,12,-21.000000,185 +2489,11,12,-20.000000,207 +2491,15,11,-21.000000,223 +2496,5,12,-21.000000,221 +2503,2,12,-21.000000,183 +2518,0,12,-21.000000,201 +2531,4,12,-21.000000,203 +2558,14,11,-19.000000,263 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed0/monitor.csv b/experiments/e53-image-observations/results/pong-bridge-seed0/monitor.csv new file mode 100644 index 0000000..671eb24 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed0/monitor.csv @@ -0,0 +1,185 @@ +#{"t_start": 1790949435.307914, "env_id": "None"} +r,l,t +-21.0,183,4.800455 +-21.0,186,4.846668 +-21.0,189,4.885978 +-21.0,192,4.928199 +-21.0,196,4.986429 +-21.0,196,4.986445 +-21.0,202,5.082782 +-21.0,204,5.11301 +-21.0,205,5.134537 +-21.0,215,5.305281 +-21.0,220,5.44114 +-21.0,236,5.653283 +-21.0,250,5.826562 +-20.0,283,8.075813 +-19.0,286,8.115384 +-19.0,295,8.22875 +-21.0,184,9.22776 +-21.0,200,9.403684 +-21.0,187,11.372089 +-21.0,220,11.678391 +-21.0,195,11.695651 +-20.0,219,11.713137 +-20.0,221,11.789062 +-21.0,213,11.811343 +-21.0,226,11.864873 +-21.0,211,11.982001 +-21.0,201,12.060903 +-20.0,236,12.108544 +-21.0,200,12.276139 +-20.0,212,12.945184 +-20.0,218,12.986606 +-21.0,229,15.143244 +-20.0,223,16.105901 +-21.0,186,16.160422 +-21.0,204,16.406879 +-21.0,187,16.474528 +-20.0,240,16.538997 +-20.0,234,16.568768 +-20.0,213,16.632461 +-21.0,187,16.729184 +-21.0,199,16.729199 +-21.0,231,18.796417 +-20.0,235,18.796433 +-20.0,243,18.796437 +-21.0,218,18.829633 +-20.0,203,19.460692 +-21.0,206,19.541051 +-21.0,211,19.898233 +-21.0,197,22.676585 +-21.0,199,22.944323 +-21.0,239,23.258986 +-21.0,199,23.343414 +-21.0,229,23.640144 +-21.0,205,23.694362 +-20.0,238,23.733954 +-21.0,233,23.762679 +-20.0,237,23.90803 +-21.0,223,23.973072 +-19.0,263,24.073321 +-21.0,240,24.331773 +-20.0,256,26.369588 +-20.0,219,26.605997 +-21.0,195,26.66094 +-20.0,254,27.076663 +-21.0,208,27.762953 +-20.0,211,30.340707 +-21.0,237,30.449709 +-21.0,192,30.559264 +-21.0,220,30.57833 +-21.0,200,30.611719 +-21.0,187,30.659917 +-20.0,223,30.977605 +-21.0,201,30.977621 +-21.0,227,31.419414 +-19.0,249,31.449269 +-20.0,216,31.467216 +-20.0,201,31.743926 +-20.0,223,31.824444 +-21.0,195,32.121758 +-21.0,244,34.308805 +-21.0,202,34.75464 +-21.0,190,35.371433 +-21.0,186,35.39992 +-21.0,182,35.433838 +-21.0,201,35.450135 +-21.0,226,37.824189 +-20.0,225,37.842639 +-21.0,208,37.87173 +-21.0,210,37.901754 +-21.0,214,38.510992 +-21.0,220,38.55678 +-21.0,218,38.556797 +-21.0,184,38.684714 +-20.0,236,39.124242 +-19.0,243,39.334918 +-21.0,210,39.467105 +-20.0,220,41.857203 +-21.0,208,42.199434 +-21.0,203,42.199451 +-21.0,223,42.491918 +-20.0,240,42.85149 +-21.0,205,42.92443 +-21.0,211,43.012036 +-20.0,230,43.367403 +-20.0,240,43.482609 +-21.0,213,45.537067 +-20.0,235,45.936535 +-21.0,208,46.039111 +-21.0,247,46.091023 +-21.0,209,46.345649 +-20.0,227,46.513197 +-21.0,195,46.790629 +-19.0,313,47.223933 +-21.0,202,47.258852 +-21.0,226,49.42749 +-21.0,185,49.427508 +-21.0,216,49.560548 +-21.0,240,50.190028 +-21.0,230,50.190046 +-20.0,209,50.385314 +-21.0,221,50.439938 +-21.0,199,50.792299 +-21.0,236,50.952373 +-21.0,185,51.042191 +-21.0,216,51.264234 +-21.0,203,53.340539 +-21.0,198,53.488908 +-21.0,247,53.506223 +-21.0,208,54.085169 +-19.0,233,54.525502 +-20.0,225,54.807272 +-21.0,226,54.823493 +-21.0,186,54.980527 +-21.0,193,55.096856 +-21.0,242,57.090203 +-21.0,186,57.130837 +-21.0,204,57.876476 +-20.0,220,58.323493 +-20.0,264,58.323509 +-20.0,227,58.368397 +-21.0,220,58.491329 +-21.0,204,58.611608 +-21.0,222,58.687884 +-21.0,184,58.884619 +-20.0,246,61.061795 +-20.0,206,61.359703 +-21.0,198,61.492153 +-21.0,183,61.534412 +-21.0,201,61.55328 +-21.0,184,61.666908 +-21.0,211,61.866585 +-21.0,200,61.899944 +-21.0,191,64.740115 +-20.0,208,64.98615 +-19.0,250,65.114995 +-21.0,203,65.291978 +-21.0,205,65.489422 +-19.0,257,65.837119 +-20.0,274,65.936267 +-19.0,254,65.936285 +-21.0,188,65.955936 +-21.0,183,66.04449 +-21.0,231,66.230375 +-21.0,207,66.352544 +-21.0,217,68.379617 +-21.0,212,68.428005 +-21.0,206,68.552974 +-21.0,221,68.713155 +-21.0,197,69.661359 +-21.0,215,69.761045 +-20.0,235,69.801488 +-20.0,203,71.93387 +-21.0,221,71.991718 +-21.0,197,72.26262 +-21.0,197,72.279742 +-21.0,185,72.490465 +-20.0,207,72.508543 +-21.0,223,72.579214 +-21.0,221,72.686442 +-21.0,183,72.83382 +-21.0,201,73.056014 +-21.0,203,73.216943 +-19.0,263,73.633538 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed0/result.json b/experiments/e53-image-observations/results/pong-bridge-seed0/result.json new file mode 100644 index 0000000..a127369 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed0/result.json @@ -0,0 +1,12 @@ +{ + "env": "ALE/Pong-v5", + "arm": "bridge", + "seed": 0, + "weights_sha256": "82f80c02616bc24c4f2baa984af8162e0f94532e1f4c3a3ff49fb977ae1b489f", + "learn_s": 75.04, + "steps": 40960, + "steps_per_s": 545.8, + "observation_space": "Box(0, 255, (84, 84, 4), uint8)", + "torch": "2.14.1+cpu", + "client_rc": 0 +} diff --git a/experiments/e53-image-observations/results/pong-bridge-seed1.log b/experiments/e53-image-observations/results/pong-bridge-seed1.log new file mode 100644 index 0000000..8a01e77 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed1.log @@ -0,0 +1,4 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +plugrl-bridges: slots 0-15 <- ('127.0.0.1', 49294) +{"env": "ALE/Pong-v5", "arm": "bridge", "seed": 1, "weights_sha256": "60acba42dad30f0cc497681237ee46bbc236d2374ad6502a6c65381ad2e0ddec", "learn_s": 76.244, "steps": 40960, "steps_per_s": 537.2, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu", "client_rc": 0} diff --git a/experiments/e53-image-observations/results/pong-bridge-seed1/client.log b/experiments/e53-image-observations/results/pong-bridge-seed1/client.log new file mode 100644 index 0000000..78f0b06 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed1/client.log @@ -0,0 +1,3 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +closed after 2560 steps: 'plugrl-server-stop' diff --git a/experiments/e53-image-observations/results/pong-bridge-seed1/episodes.csv b/experiments/e53-image-observations/results/pong-bridge-seed1/episodes.csv new file mode 100644 index 0000000..7b0b951 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed1/episodes.csv @@ -0,0 +1,177 @@ +step,slot,episode,return,length +184,5,1,-21.000000,184 +187,10,1,-21.000000,187 +187,13,1,-21.000000,187 +193,0,1,-21.000000,193 +203,14,1,-21.000000,203 +205,15,1,-20.000000,205 +209,3,1,-21.000000,209 +220,9,1,-21.000000,220 +228,1,1,-20.000000,228 +230,4,1,-20.000000,230 +238,11,1,-20.000000,238 +239,6,1,-21.000000,239 +249,12,1,-21.000000,249 +250,8,1,-19.000000,250 +265,7,1,-19.000000,265 +285,2,1,-20.000000,285 +373,10,2,-21.000000,186 +387,5,2,-21.000000,203 +392,0,2,-21.000000,199 +423,15,2,-21.000000,218 +426,3,2,-21.000000,217 +433,4,2,-21.000000,203 +433,13,2,-20.000000,246 +436,12,2,-21.000000,187 +445,1,2,-21.000000,217 +446,6,2,-21.000000,207 +449,11,2,-21.000000,211 +483,14,2,-21.000000,280 +494,9,2,-19.000000,274 +497,8,2,-21.000000,247 +509,7,2,-20.000000,244 +513,2,2,-21.000000,228 +577,10,3,-21.000000,204 +596,0,3,-21.000000,204 +615,5,3,-19.000000,228 +634,3,3,-21.000000,208 +653,4,3,-21.000000,220 +653,11,3,-21.000000,204 +659,1,3,-20.000000,214 +660,13,3,-21.000000,227 +660,15,3,-20.000000,237 +662,12,3,-20.000000,226 +681,6,3,-21.000000,235 +681,14,3,-21.000000,198 +689,9,3,-21.000000,195 +721,7,3,-21.000000,212 +722,8,3,-21.000000,225 +730,2,3,-21.000000,217 +780,10,4,-21.000000,203 +784,0,4,-21.000000,188 +820,5,4,-21.000000,205 +866,11,4,-21.000000,213 +871,12,4,-21.000000,209 +876,9,4,-21.000000,187 +884,1,4,-20.000000,225 +885,3,4,-18.000000,251 +887,4,4,-20.000000,234 +902,13,4,-20.000000,242 +915,6,4,-21.000000,234 +919,14,4,-19.000000,238 +924,15,4,-21.000000,264 +938,2,4,-21.000000,208 +950,8,4,-20.000000,228 +969,7,4,-19.000000,248 +981,10,5,-21.000000,201 +1006,0,5,-20.000000,222 +1063,9,5,-21.000000,187 +1074,5,5,-19.000000,254 +1088,11,5,-21.000000,222 +1093,4,5,-21.000000,206 +1094,13,5,-21.000000,192 +1097,12,5,-20.000000,226 +1100,3,5,-21.000000,215 +1114,1,5,-21.000000,230 +1116,6,5,-21.000000,201 +1132,14,5,-20.000000,213 +1137,15,5,-21.000000,213 +1145,2,5,-21.000000,207 +1164,7,5,-21.000000,195 +1164,8,5,-21.000000,214 +1233,10,6,-21.000000,252 +1280,11,6,-21.000000,192 +1286,0,6,-20.000000,280 +1287,13,6,-21.000000,193 +1291,9,6,-20.000000,228 +1302,1,6,-21.000000,188 +1311,6,6,-21.000000,195 +1312,5,6,-20.000000,238 +1321,3,6,-21.000000,221 +1323,4,6,-19.000000,230 +1335,14,6,-21.000000,203 +1337,15,6,-21.000000,200 +1356,12,6,-21.000000,259 +1361,2,6,-21.000000,216 +1363,7,6,-21.000000,199 +1378,8,6,-21.000000,214 +1466,10,7,-21.000000,233 +1487,0,7,-21.000000,201 +1502,1,7,-21.000000,200 +1513,6,7,-21.000000,202 +1529,15,7,-21.000000,192 +1533,5,7,-21.000000,221 +1533,9,7,-20.000000,242 +1539,3,7,-21.000000,218 +1542,12,7,-21.000000,186 +1543,4,7,-21.000000,220 +1560,14,7,-21.000000,225 +1573,7,7,-21.000000,210 +1573,13,7,-18.000000,286 +1589,11,7,-21.000000,309 +1600,2,7,-20.000000,239 +1649,8,7,-21.000000,271 +1678,10,8,-21.000000,212 +1724,9,8,-21.000000,191 +1736,0,8,-21.000000,249 +1737,6,8,-20.000000,224 +1740,12,8,-21.000000,198 +1741,5,8,-20.000000,208 +1744,3,8,-21.000000,205 +1749,14,8,-21.000000,189 +1750,15,8,-21.000000,221 +1761,7,8,-21.000000,188 +1778,13,8,-21.000000,205 +1779,4,8,-20.000000,236 +1807,1,8,-20.000000,305 +1808,2,8,-21.000000,208 +1845,8,8,-21.000000,196 +1846,11,8,-21.000000,257 +1908,10,9,-21.000000,230 +1927,12,9,-21.000000,187 +1933,5,9,-21.000000,192 +1937,6,9,-21.000000,200 +1937,9,9,-21.000000,213 +1952,0,9,-20.000000,216 +1960,15,9,-21.000000,210 +1963,3,9,-21.000000,219 +1998,7,9,-21.000000,237 +1999,13,9,-21.000000,221 +2015,1,9,-21.000000,208 +2015,2,9,-21.000000,207 +2018,14,9,-21.000000,269 +2073,8,9,-21.000000,228 +2080,11,9,-21.000000,234 +2093,4,9,-19.000000,314 +2133,10,10,-21.000000,225 +2141,9,10,-21.000000,204 +2154,0,10,-21.000000,202 +2161,15,10,-21.000000,201 +2162,12,10,-21.000000,235 +2173,3,10,-21.000000,210 +2175,6,10,-21.000000,238 +2190,5,10,-20.000000,257 +2196,7,10,-21.000000,198 +2199,13,10,-21.000000,200 +2230,1,10,-21.000000,215 +2237,14,10,-20.000000,219 +2287,2,10,-19.000000,272 +2312,8,10,-21.000000,239 +2317,11,10,-21.000000,237 +2320,4,10,-20.000000,227 +2347,10,11,-20.000000,214 +2378,15,11,-20.000000,217 +2382,6,11,-21.000000,207 +2386,3,11,-20.000000,213 +2393,12,11,-20.000000,231 +2400,13,11,-21.000000,201 +2401,0,11,-20.000000,247 +2412,9,11,-19.000000,271 +2415,7,11,-21.000000,219 +2420,5,11,-20.000000,230 +2447,1,11,-20.000000,217 +2479,14,11,-20.000000,242 +2523,4,11,-21.000000,203 +2538,2,11,-20.000000,251 +2551,11,11,-21.000000,234 +2559,8,11,-21.000000,247 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed1/monitor.csv b/experiments/e53-image-observations/results/pong-bridge-seed1/monitor.csv new file mode 100644 index 0000000..f85f558 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed1/monitor.csv @@ -0,0 +1,178 @@ +#{"t_start": 1790949585.7527168, "env_id": "None"} +r,l,t +-21.0,184,4.957053 +-21.0,187,5.003764 +-21.0,187,5.003784 +-21.0,193,5.082787 +-21.0,203,5.24942 +-20.0,205,5.277145 +-21.0,209,5.329025 +-21.0,220,5.478633 +-20.0,228,5.637216 +-20.0,230,5.667584 +-20.0,238,5.791063 +-21.0,239,5.808208 +-21.0,249,5.956864 +-19.0,250,5.976822 +-19.0,265,8.14901 +-20.0,285,8.443884 +-21.0,186,9.746234 +-21.0,203,11.813787 +-21.0,199,11.884729 +-21.0,218,12.392156 +-21.0,217,12.43573 +-21.0,203,12.538726 +-20.0,246,12.538743 +-21.0,187,12.579923 +-21.0,217,12.730548 +-21.0,207,12.753819 +-21.0,211,12.800421 +-21.0,280,13.304513 +-19.0,274,13.438751 +-21.0,247,13.480569 +-20.0,244,13.641218 +-21.0,228,15.596099 +-21.0,204,16.423138 +-21.0,204,16.773305 +-19.0,228,17.091808 +-21.0,208,17.368885 +-21.0,220,19.545469 +-21.0,204,19.545487 +-20.0,214,19.627211 +-21.0,227,19.653127 +-20.0,237,19.653138 +-20.0,226,19.682241 +-21.0,235,19.989465 +-21.0,198,19.989482 +-21.0,195,20.117112 +-21.0,212,20.566227 +-21.0,225,20.584001 +-21.0,217,20.711675 +-21.0,203,23.300022 +-21.0,188,23.367809 +-21.0,205,23.933977 +-21.0,213,24.567997 +-21.0,209,24.675655 +-21.0,187,24.752212 +-20.0,225,24.902198 +-18.0,251,24.926511 +-20.0,234,24.962651 +-20.0,242,26.982899 +-21.0,234,27.183578 +-19.0,238,27.246585 +-21.0,264,27.316741 +-21.0,208,27.581088 +-20.0,228,27.73532 +-19.0,248,28.043847 +-21.0,201,28.244182 +-20.0,222,28.644721 +-21.0,187,31.332702 +-19.0,254,31.487825 +-21.0,222,31.670749 +-21.0,206,31.771962 +-21.0,192,31.789711 +-20.0,226,31.829715 +-21.0,215,31.872174 +-21.0,230,32.067994 +-21.0,201,32.095824 +-20.0,213,32.295111 +-21.0,213,32.367495 +-21.0,207,32.489045 +-21.0,195,34.716258 +-21.0,214,34.716276 +-21.0,252,35.825659 +-21.0,192,36.463684 +-20.0,280,38.353405 +-21.0,193,38.36877 +-20.0,228,38.426879 +-21.0,188,38.614281 +-21.0,195,38.741048 +-20.0,238,38.758631 +-21.0,221,38.939314 +-19.0,230,38.969445 +-21.0,203,39.172029 +-21.0,200,39.205236 +-21.0,259,39.505646 +-21.0,216,39.622618 +-21.0,199,39.664633 +-21.0,214,39.862534 +-21.0,233,42.983267 +-21.0,201,43.309419 +-21.0,200,43.498736 +-21.0,202,43.678049 +-21.0,192,43.921581 +-21.0,221,43.991313 +-20.0,242,43.991341 +-21.0,218,45.952791 +-21.0,186,45.996941 +-21.0,220,46.014152 +-21.0,225,46.277387 +-21.0,210,46.460016 +-18.0,286,46.460032 +-21.0,309,46.70099 +-20.0,239,46.916684 +-21.0,271,47.631837 +-21.0,212,49.968742 +-21.0,191,50.643366 +-21.0,249,50.860144 +-20.0,224,50.877178 +-21.0,198,50.918337 +-20.0,208,50.936184 +-21.0,205,50.978466 +-21.0,189,51.043162 +-21.0,221,51.059929 +-21.0,188,51.240917 +-21.0,205,51.501522 +-20.0,236,51.523028 +-20.0,305,53.785083 +-21.0,208,53.804792 +-21.0,196,54.379535 +-21.0,257,54.43758 +-21.0,230,55.338288 +-21.0,187,57.5763 +-21.0,192,57.667521 +-21.0,200,57.779826 +-21.0,213,57.779841 +-20.0,216,58.030668 +-21.0,210,58.137293 +-21.0,219,58.181952 +-21.0,237,58.690923 +-21.0,221,58.708267 +-21.0,208,58.91556 +-21.0,207,58.915579 +-21.0,269,59.014764 +-21.0,228,61.720373 +-21.0,234,61.820161 +-19.0,314,62.055445 +-21.0,225,62.672052 +-21.0,204,62.811389 +-21.0,202,62.993285 +-21.0,201,63.082756 +-21.0,235,63.09948 +-21.0,210,63.269932 +-21.0,238,63.299476 +-20.0,257,65.334141 +-21.0,198,65.41403 +-21.0,200,65.461865 +-21.0,215,65.892667 +-20.0,219,65.986426 +-19.0,272,66.725833 +-21.0,239,69.036173 +-21.0,237,69.110889 +-20.0,227,69.152588 +-20.0,214,69.500391 +-20.0,217,70.040399 +-21.0,207,70.092413 +-20.0,213,70.148867 +-20.0,231,70.269965 +-21.0,201,70.367254 +-20.0,247,70.386345 +-19.0,271,70.54601 +-21.0,219,70.587838 +-20.0,230,70.663789 +-20.0,217,73.024422 +-20.0,242,73.497877 +-21.0,203,74.15256 +-20.0,251,74.373603 +-21.0,234,74.634926 +-21.0,247,74.753372 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed1/result.json b/experiments/e53-image-observations/results/pong-bridge-seed1/result.json new file mode 100644 index 0000000..b15ae2d --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed1/result.json @@ -0,0 +1,12 @@ +{ + "env": "ALE/Pong-v5", + "arm": "bridge", + "seed": 1, + "weights_sha256": "60acba42dad30f0cc497681237ee46bbc236d2374ad6502a6c65381ad2e0ddec", + "learn_s": 76.244, + "steps": 40960, + "steps_per_s": 537.2, + "observation_space": "Box(0, 255, (84, 84, 4), uint8)", + "torch": "2.14.1+cpu", + "client_rc": 0 +} diff --git a/experiments/e53-image-observations/results/pong-bridge-seed2.log b/experiments/e53-image-observations/results/pong-bridge-seed2.log new file mode 100644 index 0000000..3b1cde9 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed2.log @@ -0,0 +1,4 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +plugrl-bridges: slots 0-15 <- ('127.0.0.1', 39522) +{"env": "ALE/Pong-v5", "arm": "bridge", "seed": 2, "weights_sha256": "6bf04095eb14a02d5a4831dfa0d29d8e185cf3edfdcfa62c55df123f842ae611", "learn_s": 75.771, "steps": 40960, "steps_per_s": 540.6, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu", "client_rc": 0} diff --git a/experiments/e53-image-observations/results/pong-bridge-seed2/client.log b/experiments/e53-image-observations/results/pong-bridge-seed2/client.log new file mode 100644 index 0000000..78f0b06 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed2/client.log @@ -0,0 +1,3 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +closed after 2560 steps: 'plugrl-server-stop' diff --git a/experiments/e53-image-observations/results/pong-bridge-seed2/episodes.csv b/experiments/e53-image-observations/results/pong-bridge-seed2/episodes.csv new file mode 100644 index 0000000..9a6dc46 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed2/episodes.csv @@ -0,0 +1,184 @@ +step,slot,episode,return,length +183,8,1,-21.000000,183 +183,13,1,-21.000000,183 +186,5,1,-21.000000,186 +190,7,1,-21.000000,190 +197,12,1,-21.000000,197 +200,4,1,-21.000000,200 +203,11,1,-21.000000,203 +205,10,1,-20.000000,205 +210,0,1,-20.000000,210 +212,1,1,-20.000000,212 +213,15,1,-21.000000,213 +220,14,1,-20.000000,220 +224,2,1,-21.000000,224 +226,6,1,-21.000000,226 +237,3,1,-20.000000,237 +261,9,1,-19.000000,261 +383,13,2,-21.000000,200 +389,10,2,-21.000000,184 +391,12,2,-21.000000,194 +392,7,2,-21.000000,202 +398,4,2,-21.000000,198 +398,8,2,-21.000000,215 +420,3,2,-21.000000,183 +420,11,2,-20.000000,217 +424,5,2,-21.000000,238 +431,1,2,-20.000000,219 +447,14,2,-21.000000,227 +447,15,2,-20.000000,234 +466,6,2,-21.000000,240 +474,0,2,-20.000000,264 +502,2,2,-20.000000,278 +506,9,2,-20.000000,245 +573,13,3,-21.000000,190 +583,4,3,-21.000000,185 +600,8,3,-21.000000,202 +602,12,3,-21.000000,211 +627,10,3,-20.000000,238 +628,3,3,-21.000000,208 +631,15,3,-21.000000,184 +633,7,3,-19.000000,241 +637,11,3,-21.000000,217 +655,5,3,-21.000000,231 +665,14,3,-21.000000,218 +673,1,3,-20.000000,242 +679,6,3,-21.000000,213 +695,2,3,-21.000000,193 +707,0,3,-21.000000,233 +714,9,3,-21.000000,208 +760,13,4,-21.000000,187 +789,12,4,-21.000000,187 +813,4,4,-21.000000,230 +828,3,4,-21.000000,200 +829,8,4,-21.000000,229 +838,10,4,-21.000000,211 +839,11,4,-21.000000,202 +849,7,4,-20.000000,216 +851,15,4,-21.000000,220 +861,1,4,-21.000000,188 +868,14,4,-21.000000,203 +878,5,4,-20.000000,223 +889,0,4,-21.000000,182 +890,6,4,-21.000000,211 +897,2,4,-21.000000,202 +917,9,4,-21.000000,203 +971,13,5,-20.000000,211 +1001,4,5,-21.000000,188 +1026,12,5,-20.000000,237 +1041,10,5,-21.000000,203 +1045,15,5,-21.000000,194 +1049,7,5,-21.000000,200 +1054,14,5,-21.000000,186 +1056,3,5,-20.000000,228 +1077,1,5,-21.000000,216 +1079,11,5,-20.000000,240 +1085,5,5,-20.000000,207 +1101,0,5,-21.000000,212 +1108,8,5,-18.000000,279 +1111,2,5,-21.000000,214 +1124,6,5,-19.000000,234 +1138,9,5,-21.000000,221 +1193,13,6,-21.000000,222 +1197,4,6,-21.000000,196 +1231,12,6,-21.000000,205 +1244,7,6,-21.000000,195 +1253,14,6,-21.000000,199 +1263,15,6,-21.000000,218 +1273,5,6,-21.000000,188 +1288,10,6,-21.000000,247 +1292,1,6,-21.000000,215 +1301,11,6,-20.000000,222 +1305,8,6,-21.000000,197 +1324,6,6,-21.000000,200 +1335,3,6,-19.000000,279 +1347,0,6,-19.000000,246 +1347,2,6,-21.000000,236 +1387,9,6,-21.000000,249 +1396,13,7,-21.000000,203 +1418,4,7,-21.000000,221 +1450,12,7,-20.000000,219 +1459,14,7,-21.000000,206 +1465,7,7,-21.000000,221 +1476,15,7,-21.000000,213 +1480,5,7,-21.000000,207 +1506,10,7,-21.000000,218 +1516,11,7,-21.000000,215 +1521,3,7,-21.000000,186 +1524,1,7,-20.000000,232 +1542,8,7,-21.000000,237 +1545,0,7,-21.000000,198 +1551,6,7,-20.000000,227 +1580,2,7,-21.000000,233 +1600,13,8,-21.000000,204 +1606,9,7,-21.000000,219 +1654,4,8,-20.000000,236 +1662,14,8,-21.000000,203 +1669,12,8,-21.000000,219 +1689,7,8,-20.000000,224 +1693,10,8,-21.000000,187 +1695,5,8,-21.000000,215 +1710,15,8,-20.000000,234 +1713,11,8,-21.000000,197 +1723,3,8,-21.000000,202 +1732,8,8,-21.000000,190 +1748,1,8,-21.000000,224 +1794,6,8,-21.000000,243 +1807,2,8,-21.000000,227 +1823,0,8,-18.000000,278 +1831,13,9,-21.000000,231 +1847,9,8,-20.000000,241 +1855,12,9,-21.000000,186 +1861,4,9,-21.000000,207 +1862,14,9,-21.000000,200 +1900,5,9,-20.000000,205 +1915,10,9,-20.000000,222 +1915,11,9,-21.000000,202 +1921,3,9,-21.000000,198 +1924,7,9,-21.000000,235 +1938,8,9,-21.000000,206 +1960,15,9,-18.000000,250 +2003,6,9,-21.000000,209 +2015,13,10,-21.000000,184 +2016,1,9,-20.000000,268 +2031,2,9,-20.000000,224 +2046,0,9,-21.000000,223 +2051,9,9,-21.000000,204 +2093,4,10,-20.000000,232 +2101,12,10,-21.000000,246 +2103,5,10,-21.000000,203 +2117,14,10,-20.000000,255 +2122,7,10,-21.000000,198 +2124,3,10,-21.000000,203 +2133,10,10,-21.000000,218 +2137,8,10,-21.000000,199 +2148,11,10,-20.000000,233 +2174,15,10,-20.000000,214 +2208,13,11,-21.000000,193 +2219,6,10,-21.000000,216 +2236,2,10,-21.000000,205 +2238,9,10,-21.000000,187 +2248,1,10,-20.000000,232 +2275,0,10,-21.000000,229 +2277,4,11,-21.000000,184 +2305,12,11,-21.000000,204 +2318,5,11,-21.000000,215 +2321,14,11,-21.000000,204 +2344,7,11,-20.000000,222 +2346,3,11,-20.000000,222 +2348,11,11,-21.000000,200 +2352,8,11,-21.000000,215 +2358,10,11,-21.000000,225 +2403,6,11,-21.000000,184 +2431,15,11,-20.000000,257 +2433,13,12,-21.000000,225 +2444,2,11,-21.000000,208 +2456,9,11,-20.000000,218 +2462,4,12,-21.000000,185 +2479,0,11,-21.000000,204 +2479,1,11,-21.000000,231 +2513,5,12,-21.000000,195 +2518,12,12,-21.000000,213 +2529,7,12,-21.000000,185 +2547,3,12,-21.000000,201 +2548,14,12,-21.000000,227 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed2/monitor.csv b/experiments/e53-image-observations/results/pong-bridge-seed2/monitor.csv new file mode 100644 index 0000000..d392a9c --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed2/monitor.csv @@ -0,0 +1,185 @@ +#{"t_start": 1790949735.987226, "env_id": "None"} +r,l,t +-21.0,183,4.754981 +-21.0,183,4.755004 +-21.0,186,4.797794 +-21.0,190,4.851143 +-21.0,197,4.961332 +-21.0,200,5.042733 +-21.0,203,5.103585 +-20.0,205,5.133261 +-20.0,210,5.208237 +-20.0,212,5.262604 +-21.0,213,5.284001 +-20.0,220,5.399045 +-21.0,224,5.505698 +-21.0,226,5.536206 +-20.0,237,5.679704 +-19.0,261,7.899789 +-21.0,200,9.646981 +-21.0,184,11.54677 +-21.0,194,11.575417 +-21.0,202,11.59395 +-21.0,198,11.677145 +-21.0,215,11.677163 +-21.0,183,12.028526 +-20.0,217,12.028556 +-21.0,238,12.088664 +-20.0,219,12.23557 +-21.0,227,12.480766 +-20.0,234,12.480783 +-21.0,240,12.846303 +-20.0,264,12.964546 +-20.0,278,13.410738 +-20.0,245,13.485657 +-21.0,190,16.379602 +-21.0,185,16.53074 +-21.0,202,16.7701 +-21.0,211,16.836308 +-20.0,238,17.236052 +-21.0,208,17.253168 +-21.0,184,17.296394 +-19.0,241,17.324546 +-21.0,217,17.376022 +-21.0,231,19.515004 +-21.0,218,19.653792 +-20.0,242,19.792623 +-21.0,213,19.886817 +-21.0,193,20.145555 +-21.0,233,20.32995 +-21.0,208,20.47388 +-21.0,187,21.147914 +-21.0,187,23.400596 +-21.0,230,23.817609 +-21.0,200,24.05285 +-21.0,229,24.071484 +-21.0,211,24.262601 +-21.0,202,24.283261 +-20.0,216,24.418192 +-21.0,220,24.466473 +-21.0,188,24.593506 +-21.0,203,24.704915 +-20.0,223,24.886267 +-21.0,182,25.043971 +-21.0,211,25.059997 +-21.0,202,27.001822 +-21.0,203,27.264138 +-20.0,211,28.113929 +-21.0,188,28.538312 +-20.0,237,30.769804 +-21.0,203,30.960979 +-21.0,194,31.014527 +-21.0,200,31.069182 +-21.0,186,31.140088 +-20.0,228,31.169451 +-21.0,216,31.439094 +-20.0,240,31.467356 +-20.0,207,31.546003 +-21.0,212,31.762462 +-18.0,279,31.864006 +-21.0,214,31.911672 +-19.0,234,32.119277 +-21.0,221,32.327086 +-21.0,222,35.042484 +-21.0,196,35.125112 +-21.0,205,35.630802 +-21.0,195,35.796619 +-21.0,199,35.979388 +-21.0,218,36.12219 +-21.0,188,36.285195 +-21.0,247,38.354025 +-21.0,215,38.411141 +-20.0,222,38.559925 +-21.0,197,38.641231 +-21.0,200,38.933164 +-19.0,279,39.13052 +-19.0,246,39.365208 +-21.0,236,39.365226 +-21.0,249,39.898865 +-21.0,203,40.037363 +-21.0,221,42.11256 +-20.0,219,42.665507 +-21.0,206,42.786723 +-21.0,221,42.911155 +-21.0,213,43.083148 +-21.0,207,43.137342 +-21.0,218,43.589468 +-21.0,215,43.753363 +-21.0,186,43.833054 +-20.0,232,43.87259 +-21.0,237,45.93929 +-21.0,198,45.993853 +-20.0,227,46.08321 +-21.0,233,46.592912 +-21.0,204,46.883961 +-21.0,219,46.972141 +-20.0,236,47.71316 +-21.0,203,47.818182 +-21.0,219,49.778107 +-20.0,224,50.074846 +-21.0,187,50.13285 +-21.0,215,50.176092 +-20.0,234,50.403935 +-21.0,197,50.445788 +-21.0,202,50.578923 +-21.0,190,50.722884 +-21.0,224,50.970049 +-21.0,243,53.604754 +-21.0,227,53.775557 +-18.0,278,53.99176 +-21.0,231,54.137118 +-20.0,241,54.348755 +-21.0,186,54.502681 +-21.0,207,54.579935 +-21.0,200,54.597464 +-20.0,205,55.143378 +-20.0,222,55.384537 +-21.0,202,55.384553 +-21.0,198,57.312496 +-21.0,235,57.360475 +-21.0,206,57.578995 +-18.0,250,57.913789 +-21.0,209,58.56883 +-21.0,184,58.762601 +-20.0,268,58.783338 +-20.0,224,59.015515 +-21.0,223,59.208807 +-21.0,204,61.120211 +-20.0,232,61.794975 +-21.0,246,61.956167 +-21.0,203,62.001336 +-20.0,255,62.227684 +-21.0,198,62.312895 +-21.0,203,62.342159 +-21.0,218,62.468963 +-21.0,199,62.568822 +-20.0,233,62.754121 +-20.0,214,63.179561 +-21.0,193,65.476373 +-21.0,216,65.619946 +-21.0,205,65.934952 +-21.0,187,65.966228 +-20.0,232,66.103827 +-21.0,229,66.527611 +-21.0,184,66.557145 +-21.0,204,68.818067 +-21.0,215,68.998615 +-21.0,204,69.049381 +-20.0,222,69.362219 +-20.0,222,69.392164 +-21.0,200,69.463613 +-21.0,215,69.535107 +-21.0,225,69.628604 +-21.0,184,70.28515 +-20.0,257,70.687684 +-21.0,225,72.503179 +-21.0,208,72.649757 +-20.0,218,72.823967 +-21.0,185,72.939616 +-21.0,204,73.247788 +-21.0,231,73.247806 +-21.0,195,73.724471 +-21.0,213,73.790708 +-21.0,185,73.975507 +-21.0,201,74.271071 +-21.0,227,74.28982 diff --git a/experiments/e53-image-observations/results/pong-bridge-seed2/result.json b/experiments/e53-image-observations/results/pong-bridge-seed2/result.json new file mode 100644 index 0000000..e2f2287 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-bridge-seed2/result.json @@ -0,0 +1,12 @@ +{ + "env": "ALE/Pong-v5", + "arm": "bridge", + "seed": 2, + "weights_sha256": "6bf04095eb14a02d5a4831dfa0d29d8e185cf3edfdcfa62c55df123f842ae611", + "learn_s": 75.771, + "steps": 40960, + "steps_per_s": 540.6, + "observation_space": "Box(0, 255, (84, 84, 4), uint8)", + "torch": "2.14.1+cpu", + "client_rc": 0 +} diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed0.log b/experiments/e53-image-observations/results/pong-inprocess-seed0.log new file mode 100644 index 0000000..5f0d9d5 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed0.log @@ -0,0 +1,3 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +{"env": "ALE/Pong-v5", "arm": "inprocess", "seed": 0, "weights_sha256": "82f80c02616bc24c4f2baa984af8162e0f94532e1f4c3a3ff49fb977ae1b489f", "learn_s": 67.381, "steps": 40960, "steps_per_s": 607.9, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu"} diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed0/monitor.csv b/experiments/e53-image-observations/results/pong-inprocess-seed0/monitor.csv new file mode 100644 index 0000000..60aaacf --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed0/monitor.csv @@ -0,0 +1,185 @@ +#{"t_start": 1790949364.3979475, "env_id": "None"} +r,l,t +-21.0,183,5.245734 +-21.0,186,5.284531 +-21.0,189,5.321161 +-21.0,192,5.360959 +-21.0,196,5.41564 +-21.0,196,5.415675 +-21.0,202,5.48691 +-21.0,204,5.513135 +-21.0,205,5.52856 +-21.0,215,5.64621 +-21.0,220,5.708428 +-21.0,236,5.896159 +-21.0,250,6.061136 +-20.0,283,8.263794 +-19.0,286,8.300809 +-19.0,295,8.408195 +-21.0,184,9.264317 +-21.0,200,9.418308 +-21.0,187,11.38395 +-21.0,220,11.586902 +-21.0,195,11.601781 +-20.0,219,11.617889 +-20.0,221,11.689403 +-21.0,213,11.707156 +-21.0,226,11.756922 +-21.0,211,11.863567 +-21.0,201,11.93392 +-20.0,236,11.950515 +-21.0,200,12.088182 +-20.0,212,12.631846 +-20.0,218,12.669533 +-21.0,229,14.819569 +-20.0,223,15.59925 +-21.0,186,15.648263 +-21.0,204,15.834988 +-21.0,187,15.89499 +-20.0,240,15.95516 +-20.0,234,15.983607 +-20.0,213,16.043165 +-21.0,187,16.135538 +-21.0,199,16.135572 +-21.0,231,18.16977 +-20.0,235,18.169812 +-20.0,243,18.169817 +-21.0,218,18.195493 +-20.0,203,18.712685 +-21.0,206,18.782955 +-21.0,211,19.09967 +-21.0,197,21.565301 +-21.0,199,21.778212 +-21.0,239,22.008645 +-21.0,199,22.057654 +-21.0,229,22.264983 +-21.0,205,22.314089 +-20.0,238,22.35206 +-21.0,233,22.380183 +-20.0,237,22.507536 +-21.0,223,22.534219 +-19.0,263,22.592757 +-21.0,240,22.754787 +-20.0,256,24.675727 +-20.0,219,24.87186 +-21.0,195,24.920357 +-20.0,254,25.201447 +-21.0,208,25.750278 +-20.0,211,28.014529 +-21.0,237,28.087515 +-21.0,192,28.159124 +-21.0,220,28.176009 +-21.0,200,28.2045 +-21.0,187,28.244447 +-20.0,223,28.430924 +-21.0,201,28.430961 +-21.0,227,28.785203 +-19.0,249,28.811234 +-20.0,216,28.827676 +-20.0,201,29.009774 +-20.0,223,29.071183 +-21.0,195,29.278161 +-21.0,244,31.367846 +-21.0,202,31.705678 +-21.0,190,32.100487 +-21.0,186,32.126296 +-21.0,182,32.154282 +-21.0,201,32.17037 +-21.0,226,34.408656 +-20.0,225,34.426427 +-21.0,208,34.454239 +-21.0,210,34.483261 +-21.0,214,34.9289 +-21.0,220,34.97162 +-21.0,218,34.971649 +-21.0,184,35.044765 +-20.0,236,35.384105 +-19.0,243,35.529594 +-21.0,210,35.636254 +-20.0,220,37.931848 +-21.0,208,38.194585 +-21.0,203,38.194623 +-21.0,223,38.389039 +-20.0,240,38.606179 +-21.0,205,38.667184 +-21.0,211,38.727407 +-20.0,230,39.000412 +-20.0,240,39.095879 +-21.0,213,41.06459 +-20.0,235,41.349736 +-21.0,208,41.442857 +-21.0,247,41.492388 +-21.0,209,41.69912 +-20.0,227,41.802561 +-21.0,195,41.997751 +-19.0,313,42.312762 +-21.0,202,42.341712 +-21.0,226,44.482916 +-21.0,185,44.482958 +-21.0,216,44.568097 +-21.0,240,45.086986 +-21.0,230,45.087024 +-20.0,209,45.216045 +-21.0,221,45.265929 +-21.0,199,45.542428 +-21.0,236,45.681907 +-21.0,185,45.733824 +-21.0,216,45.884665 +-21.0,203,47.915174 +-21.0,198,48.055612 +-21.0,247,48.071322 +-21.0,208,48.489854 +-19.0,233,48.797127 +-20.0,225,48.980022 +-21.0,226,48.995493 +-21.0,186,49.114004 +-21.0,193,49.198383 +-21.0,242,51.114759 +-21.0,186,51.147443 +-21.0,204,51.64997 +-20.0,220,52.015317 +-20.0,264,52.015356 +-20.0,227,52.055674 +-21.0,220,52.171836 +-21.0,204,52.286018 +-21.0,222,52.356531 +-21.0,184,52.496401 +-20.0,246,54.637551 +-20.0,206,54.910733 +-21.0,198,55.004303 +-21.0,183,55.044209 +-21.0,201,55.062403 +-21.0,184,55.158673 +-21.0,211,55.300563 +-21.0,200,55.327677 +-21.0,191,57.90574 +-20.0,208,58.103161 +-19.0,250,58.223073 +-21.0,203,58.387276 +-21.0,205,58.580876 +-19.0,257,58.872244 +-20.0,274,58.960227 +-19.0,254,58.960262 +-21.0,188,58.977813 +-21.0,183,59.048867 +-21.0,231,59.198281 +-21.0,207,59.301444 +-21.0,217,61.276448 +-21.0,212,61.316682 +-21.0,206,61.410787 +-21.0,221,61.562901 +-21.0,197,62.37924 +-21.0,215,62.471826 +-20.0,235,62.509998 +-20.0,203,64.603911 +-21.0,221,64.653748 +-21.0,197,64.894211 +-21.0,197,64.908509 +-21.0,185,65.081792 +-20.0,207,65.098705 +-21.0,223,65.124445 +-21.0,221,65.185784 +-21.0,183,65.269842 +-21.0,201,65.441404 +-21.0,203,65.591508 +-19.0,263,65.898403 diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed0/result.json b/experiments/e53-image-observations/results/pong-inprocess-seed0/result.json new file mode 100644 index 0000000..e8037c6 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed0/result.json @@ -0,0 +1,11 @@ +{ + "env": "ALE/Pong-v5", + "arm": "inprocess", + "seed": 0, + "weights_sha256": "82f80c02616bc24c4f2baa984af8162e0f94532e1f4c3a3ff49fb977ae1b489f", + "learn_s": 67.381, + "steps": 40960, + "steps_per_s": 607.9, + "observation_space": "Box(0, 255, (84, 84, 4), uint8)", + "torch": "2.14.1+cpu" +} diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed1.log b/experiments/e53-image-observations/results/pong-inprocess-seed1.log new file mode 100644 index 0000000..40f7c31 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed1.log @@ -0,0 +1,3 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +{"env": "ALE/Pong-v5", "arm": "inprocess", "seed": 1, "weights_sha256": "60acba42dad30f0cc497681237ee46bbc236d2374ad6502a6c65381ad2e0ddec", "learn_s": 69.586, "steps": 40960, "steps_per_s": 588.6, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu"} diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed1/monitor.csv b/experiments/e53-image-observations/results/pong-inprocess-seed1/monitor.csv new file mode 100644 index 0000000..b4535cb --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed1/monitor.csv @@ -0,0 +1,178 @@ +#{"t_start": 1790949512.6539161, "env_id": "None"} +r,l,t +-21.0,184,5.356394 +-21.0,187,5.397985 +-21.0,187,5.398018 +-21.0,193,5.472736 +-21.0,203,5.591659 +-20.0,205,5.618466 +-21.0,209,5.667898 +-21.0,220,5.798455 +-20.0,228,5.891556 +-20.0,230,5.920418 +-20.0,238,6.017419 +-21.0,239,6.034025 +-21.0,249,6.15485 +-19.0,250,6.170502 +-19.0,265,8.237361 +-20.0,285,8.467791 +-21.0,186,9.474099 +-21.0,203,11.531694 +-21.0,199,11.594774 +-21.0,218,11.949878 +-21.0,217,11.990255 +-21.0,203,12.078862 +-20.0,246,12.078898 +-21.0,187,12.118272 +-21.0,217,12.224947 +-21.0,207,12.242944 +-21.0,211,12.28246 +-21.0,280,12.673403 +-19.0,274,12.801502 +-21.0,247,12.872911 +-20.0,244,13.02575 +-21.0,228,15.001935 +-21.0,204,15.874008 +-21.0,204,16.133821 +-19.0,228,16.387372 +-21.0,208,16.679172 +-21.0,220,18.848826 +-21.0,204,18.848862 +-20.0,214,18.965806 +-21.0,227,18.984884 +-20.0,237,18.984909 +-20.0,226,19.011323 +-21.0,235,19.312247 +-21.0,198,19.312287 +-21.0,195,19.407802 +-21.0,212,19.83847 +-21.0,225,19.852914 +-21.0,217,19.946413 +-21.0,203,22.416188 +-21.0,188,22.464123 +-21.0,205,22.92322 +-21.0,213,23.4705 +-21.0,209,23.553857 +-21.0,187,23.613715 +-20.0,225,23.709414 +-18.0,251,23.725636 +-20.0,234,23.782509 +-20.0,242,25.892399 +-21.0,234,26.0531 +-19.0,238,26.103656 +-21.0,264,26.164664 +-21.0,208,26.380232 +-20.0,228,26.526015 +-19.0,248,26.778122 +-21.0,201,26.922958 +-20.0,222,27.261701 +-21.0,187,29.708588 +-19.0,254,29.837243 +-21.0,222,30.033857 +-21.0,206,30.105611 +-21.0,192,30.122286 +-20.0,226,30.161038 +-21.0,215,30.243672 +-21.0,230,30.424359 +-21.0,201,30.450707 +-20.0,213,30.658949 +-21.0,213,30.72131 +-21.0,207,30.85078 +-21.0,195,32.948091 +-21.0,214,32.948134 +-21.0,252,33.812152 +-21.0,192,34.403872 +-20.0,280,36.269755 +-21.0,193,36.284352 +-20.0,228,36.336257 +-21.0,188,36.468778 +-21.0,195,36.579803 +-20.0,238,36.59588 +-21.0,221,36.703117 +-19.0,230,36.730491 +-21.0,203,36.869214 +-21.0,200,36.898546 +-21.0,259,37.119803 +-21.0,216,37.215307 +-21.0,199,37.243692 +-21.0,214,37.439021 +-21.0,233,40.199606 +-21.0,201,40.441741 +-21.0,200,40.623399 +-21.0,202,40.755334 +-21.0,192,40.968509 +-21.0,221,41.023487 +-20.0,242,41.023525 +-21.0,218,42.960958 +-21.0,186,43.001068 +-21.0,220,43.016475 +-21.0,225,43.212959 +-21.0,210,43.365695 +-18.0,286,43.365732 +-21.0,309,43.551104 +-20.0,239,43.680583 +-21.0,271,44.245934 +-21.0,212,46.437037 +-21.0,191,46.990651 +-21.0,249,47.131088 +-20.0,224,47.147226 +-21.0,198,47.186139 +-20.0,208,47.203105 +-21.0,205,47.242745 +-21.0,189,47.302678 +-21.0,221,47.319008 +-21.0,188,47.448097 +-21.0,205,47.644439 +-20.0,236,47.661989 +-20.0,305,49.880777 +-21.0,208,49.896537 +-21.0,196,50.328053 +-21.0,257,50.346434 +-21.0,230,51.058572 +-21.0,187,53.102112 +-21.0,192,53.174407 +-21.0,200,53.227339 +-21.0,213,53.227372 +-20.0,216,53.400519 +-21.0,210,53.497816 +-21.0,219,53.539331 +-21.0,237,53.940547 +-21.0,221,53.956114 +-21.0,208,54.148093 +-21.0,207,54.148131 +-21.0,269,54.187165 +-21.0,228,56.655067 +-21.0,234,56.738372 +-19.0,314,56.889282 +-21.0,225,57.402256 +-21.0,204,57.498107 +-21.0,202,57.685213 +-21.0,201,57.767273 +-21.0,235,57.783421 +-21.0,210,57.912997 +-21.0,238,57.940874 +-20.0,257,59.9218 +-21.0,198,59.995349 +-21.0,200,60.03281 +-21.0,215,60.418645 +-20.0,219,60.501941 +-19.0,272,61.07033 +-21.0,239,63.211288 +-21.0,237,63.27186 +-20.0,227,63.309681 +-20.0,214,63.650575 +-20.0,217,64.002749 +-21.0,207,64.051034 +-20.0,213,64.099785 +-20.0,231,64.197824 +-21.0,201,64.282058 +-20.0,247,64.300516 +-19.0,271,64.431198 +-21.0,219,64.469494 +-20.0,230,64.531899 +-20.0,217,66.723607 +-20.0,242,67.087265 +-21.0,203,67.597197 +-20.0,251,67.771725 +-21.0,234,67.924378 +-21.0,247,68.018732 diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed1/result.json b/experiments/e53-image-observations/results/pong-inprocess-seed1/result.json new file mode 100644 index 0000000..5a67641 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed1/result.json @@ -0,0 +1,11 @@ +{ + "env": "ALE/Pong-v5", + "arm": "inprocess", + "seed": 1, + "weights_sha256": "60acba42dad30f0cc497681237ee46bbc236d2374ad6502a6c65381ad2e0ddec", + "learn_s": 69.586, + "steps": 40960, + "steps_per_s": 588.6, + "observation_space": "Box(0, 255, (84, 84, 4), uint8)", + "torch": "2.14.1+cpu" +} diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed2.log b/experiments/e53-image-observations/results/pong-inprocess-seed2.log new file mode 100644 index 0000000..8373c2e --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed2.log @@ -0,0 +1,3 @@ +A.L.E: Arcade Learning Environment (version 0.12.1+8a8fafb) +[Powered by Stella] +{"env": "ALE/Pong-v5", "arm": "inprocess", "seed": 2, "weights_sha256": "6bf04095eb14a02d5a4831dfa0d29d8e185cf3edfdcfa62c55df123f842ae611", "learn_s": 68.144, "steps": 40960, "steps_per_s": 601.1, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu"} diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed2/monitor.csv b/experiments/e53-image-observations/results/pong-inprocess-seed2/monitor.csv new file mode 100644 index 0000000..d97f25a --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed2/monitor.csv @@ -0,0 +1,185 @@ +#{"t_start": 1790949664.310362, "env_id": "None"} +r,l,t +-21.0,183,5.316048 +-21.0,183,5.316097 +-21.0,186,5.355489 +-21.0,190,5.405365 +-21.0,197,5.489892 +-21.0,200,5.530639 +-21.0,203,5.571786 +-20.0,205,5.600071 +-20.0,210,5.661764 +-20.0,212,5.689134 +-21.0,213,5.705905 +-20.0,220,5.793081 +-21.0,224,5.843368 +-21.0,226,5.870354 +-20.0,237,6.00417 +-19.0,261,8.158544 +-21.0,200,9.594829 +-21.0,184,11.48157 +-21.0,194,11.509265 +-21.0,202,11.526773 +-21.0,198,11.60359 +-21.0,215,11.603626 +-21.0,183,11.864637 +-20.0,217,11.864675 +-21.0,238,11.916842 +-20.0,219,12.001061 +-21.0,227,12.197915 +-20.0,234,12.197947 +-21.0,240,12.423389 +-20.0,264,12.518325 +-20.0,278,12.853186 +-20.0,245,12.931847 +-21.0,190,15.542512 +-21.0,185,15.660384 +-21.0,202,15.859004 +-21.0,211,15.886444 +-20.0,238,16.200489 +-21.0,208,16.216825 +-21.0,184,16.255834 +-19.0,241,16.281288 +-21.0,217,16.330695 +-21.0,231,18.370547 +-21.0,218,18.487118 +-20.0,242,18.580737 +-21.0,213,18.651281 +-21.0,193,18.836862 +-21.0,233,18.978929 +-21.0,208,19.067071 +-21.0,187,19.592761 +-21.0,187,21.815649 +-21.0,230,22.090847 +-21.0,200,22.264919 +-21.0,229,22.280208 +-21.0,211,22.386986 +-21.0,202,22.40258 +-20.0,216,22.51982 +-21.0,220,22.550792 +-21.0,188,22.675345 +-21.0,203,22.758344 +-20.0,223,22.874805 +-21.0,182,23.005391 +-21.0,211,23.020372 +-21.0,202,24.92756 +-21.0,203,25.159508 +-20.0,211,25.785567 +-21.0,188,26.174126 +-20.0,237,28.295935 +-21.0,203,28.471133 +-21.0,194,28.522806 +-21.0,200,28.573834 +-21.0,186,28.664815 +-20.0,228,28.69188 +-21.0,216,28.9345 +-20.0,240,28.961508 +-20.0,207,29.059596 +-21.0,212,29.246357 +-18.0,279,29.332979 +-21.0,214,29.372327 +-19.0,234,29.52548 +-21.0,221,29.689176 +-21.0,222,32.089505 +-21.0,196,32.139404 +-21.0,205,32.529847 +-21.0,195,32.681918 +-21.0,199,32.789352 +-21.0,218,32.906166 +-21.0,188,33.022542 +-21.0,247,34.958199 +-21.0,215,35.011589 +-20.0,222,35.118673 +-21.0,197,35.17079 +-21.0,200,35.392346 +-19.0,279,35.522254 +-19.0,246,35.669181 +-21.0,236,35.669221 +-21.0,249,36.186047 +-21.0,203,36.293986 +-21.0,221,38.279805 +-20.0,219,38.64736 +-21.0,206,38.756117 +-21.0,221,38.830472 +-21.0,213,38.961651 +-21.0,207,39.012821 +-21.0,218,39.34434 +-21.0,215,39.49372 +-21.0,186,39.557894 +-20.0,232,39.59688 +-21.0,237,41.605942 +-21.0,198,41.645976 +-20.0,227,41.71758 +-21.0,233,42.083401 +-21.0,204,42.314946 +-21.0,219,42.38572 +-20.0,236,42.93508 +-21.0,203,43.0334 +-21.0,219,44.917294 +-20.0,224,45.154622 +-21.0,187,45.20693 +-21.0,215,45.238152 +-20.0,234,45.46527 +-21.0,197,45.503721 +-21.0,202,45.625059 +-21.0,190,45.730355 +-21.0,224,45.957543 +-21.0,243,48.316756 +-21.0,227,48.477296 +-18.0,278,48.663598 +-21.0,231,48.759372 +-20.0,241,48.945317 +-21.0,186,49.075773 +-21.0,207,49.147404 +-21.0,200,49.164553 +-20.0,205,49.597948 +-20.0,222,49.777214 +-21.0,202,49.777253 +-21.0,198,51.654621 +-21.0,235,51.694106 +-21.0,206,51.872395 +-18.0,250,52.125736 +-21.0,209,52.620882 +-21.0,184,52.788934 +-20.0,268,52.806564 +-20.0,224,52.985049 +-21.0,223,53.177898 +-21.0,204,55.085081 +-20.0,232,55.573635 +-21.0,246,55.666568 +-21.0,203,55.69519 +-20.0,255,55.857443 +-21.0,198,55.918145 +-21.0,203,55.944512 +-21.0,218,56.051013 +-21.0,199,56.101702 +-20.0,233,56.232157 +-20.0,214,56.555692 +-21.0,193,58.755104 +-21.0,216,58.90404 +-21.0,205,59.101561 +-21.0,187,59.130024 +-20.0,232,59.249643 +-21.0,229,59.561849 +-21.0,184,59.589364 +-21.0,204,61.767353 +-21.0,215,61.921061 +-21.0,204,61.960097 +-20.0,222,62.230809 +-20.0,222,62.258409 +-21.0,200,62.325072 +-21.0,215,62.391676 +-21.0,225,62.468631 +-21.0,184,63.061822 +-20.0,257,63.385909 +-21.0,225,65.186544 +-21.0,208,65.314434 +-20.0,218,65.454109 +-21.0,185,65.525184 +-21.0,204,65.728407 +-21.0,231,65.728445 +-21.0,195,66.170932 +-21.0,213,66.23425 +-21.0,185,66.362091 +-21.0,201,66.571584 +-21.0,227,66.588935 diff --git a/experiments/e53-image-observations/results/pong-inprocess-seed2/result.json b/experiments/e53-image-observations/results/pong-inprocess-seed2/result.json new file mode 100644 index 0000000..74e5b37 --- /dev/null +++ b/experiments/e53-image-observations/results/pong-inprocess-seed2/result.json @@ -0,0 +1,11 @@ +{ + "env": "ALE/Pong-v5", + "arm": "inprocess", + "seed": 2, + "weights_sha256": "6bf04095eb14a02d5a4831dfa0d29d8e185cf3edfdcfa62c55df123f842ae611", + "learn_s": 68.144, + "steps": 40960, + "steps_per_s": 601.1, + "observation_space": "Box(0, 255, (84, 84, 4), uint8)", + "torch": "2.14.1+cpu" +} diff --git a/experiments/e53-image-observations/results/registered.out b/experiments/e53-image-observations/results/registered.out new file mode 100644 index 0000000..19bb662 --- /dev/null +++ b/experiments/e53-image-observations/results/registered.out @@ -0,0 +1,15 @@ + 09:56:03 up 27 days, 22:57, 1 user, load average: 0.96, 1.31, 0.93 +== pong-inprocess-seed0 09:56:03 +{"env": "ALE/Pong-v5", "arm": "inprocess", "seed": 0, "weights_sha256": "82f80c02616bc24c4f2baa984af8162e0f94532e1f4c3a3ff49fb977ae1b489f", "learn_s": 67.381, "steps": 40960, "steps_per_s": 607.9, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu"} +== pong-bridge-seed0 09:57:12 +{"env": "ALE/Pong-v5", "arm": "bridge", "seed": 0, "weights_sha256": "82f80c02616bc24c4f2baa984af8162e0f94532e1f4c3a3ff49fb977ae1b489f", "learn_s": 75.04, "steps": 40960, "steps_per_s": 545.8, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu", "client_rc": 0} +== pong-inprocess-seed1 09:58:31 +{"env": "ALE/Pong-v5", "arm": "inprocess", "seed": 1, "weights_sha256": "60acba42dad30f0cc497681237ee46bbc236d2374ad6502a6c65381ad2e0ddec", "learn_s": 69.586, "steps": 40960, "steps_per_s": 588.6, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu"} +== pong-bridge-seed1 09:59:42 +{"env": "ALE/Pong-v5", "arm": "bridge", "seed": 1, "weights_sha256": "60acba42dad30f0cc497681237ee46bbc236d2374ad6502a6c65381ad2e0ddec", "learn_s": 76.244, "steps": 40960, "steps_per_s": 537.2, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu", "client_rc": 0} +== pong-inprocess-seed2 10:01:02 +{"env": "ALE/Pong-v5", "arm": "inprocess", "seed": 2, "weights_sha256": "6bf04095eb14a02d5a4831dfa0d29d8e185cf3edfdcfa62c55df123f842ae611", "learn_s": 68.144, "steps": 40960, "steps_per_s": 601.1, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu"} +== pong-bridge-seed2 10:02:13 +{"env": "ALE/Pong-v5", "arm": "bridge", "seed": 2, "weights_sha256": "6bf04095eb14a02d5a4831dfa0d29d8e185cf3edfdcfa62c55df123f842ae611", "learn_s": 75.771, "steps": 40960, "steps_per_s": 540.6, "observation_space": "Box(0, 255, (84, 84, 4), uint8)", "torch": "2.14.1+cpu", "client_rc": 0} + 10:03:32 up 27 days, 23:05, 1 user, load average: 3.27, 3.00, 1.94 +E53_DONE diff --git a/experiments/e53-image-observations/results/verdicts.txt b/experiments/e53-image-observations/results/verdicts.txt new file mode 100644 index 0000000..bc6dcbe --- /dev/null +++ b/experiments/e53-image-observations/results/verdicts.txt @@ -0,0 +1,8 @@ +seed 0: weights same (82f80c02616b / 82f80c02616b), 183 episodes same, episode log agrees; 67.4 s vs 75.0 s (+11%, 2.99 ms per vector step) +seed 1: weights same (60acba42dad3 / 60acba42dad3), 176 episodes same, episode log agrees; 69.6 s vs 76.2 s (+10%, 2.60 ms per vector step) +seed 2: weights same (6bf04095eb14 / 6bf04095eb14), 183 episodes same, episode log agrees; 68.1 s vs 75.8 s (+11%, 2.98 ms per vector step) + +V1: holds +P1: holds +P2: holds +P3: holds