diff --git a/experiments/e31-dppo-longer/.gitignore b/experiments/e31-dppo-longer/.gitignore new file mode 100644 index 0000000..1498076 --- /dev/null +++ b/experiments/e31-dppo-longer/.gitignore @@ -0,0 +1,2 @@ +# Checkpoints and tensorboard event files; summarise.py reads out what they hold. +results/*/dppo/ diff --git a/experiments/e31-dppo-longer/FINDINGS.md b/experiments/e31-dppo-longer/FINDINGS.md new file mode 100644 index 0000000..e4f9605 --- /dev/null +++ b/experiments/e31-dppo-longer/FINDINGS.md @@ -0,0 +1,106 @@ +# E31: run three times as long, `dppo-policy` learns Hopper and Walker2d under DPPO + +2026-09-27 · Linux workstation (`guangzhao`), CPU only · two DPPO cells, +three seeds each, 300 iterations of 4,096 steps · protocol: +[`PROTOCOL.md`](PROTOCOL.md) (`f8ffec0`, before any run) + +--- + +## The result + +E28 ran `dppo-policy` under DPPO on Hopper and Walker2d for 100 iterations; +both rose on every seed and neither reached the bar of 500. E31 ran the same +two cells, unchanged, to 300 iterations (1,228,800 steps per seed), with the +same bar. + +| cell | mean of iterations 291-300, seeds 0 / 1 / 2 | status | +| --- | --- | --- | +| `dppo-policy` · DPPO · Hopper | **873 / 961 / 812**, from 11-16 | **learns** (3 of 3 past 500) | +| `dppo-policy` · DPPO · Walker2d | **528 / 622 / 585**, from −3 to −2 | **learns** (3 of 3 past 500) | + +* **P1 holds, 2 of 2**: all six seeds logged 300 iterations, wrote the + checkpoint at 300, have no traceback and a client that exited 0; 48 and 50 + minutes per cell. +* **P2 holds**: `dppo-policy` learns Hopper. +* **P3 holds**: `dppo-policy` learns Walker2d. I had written that I expected + this one falsified; see below. + +With E24, E27 and E28, the matrix of the two MLP policies now reads: + +| | HalfCheetah | Hopper | Walker2d | robomimic square | +| --- | --- | --- | --- | --- | +| `fpo-policy` · FPO | learns (E6, E16) | learns (E23) | learns (E24) | runs (E27) | +| `fpo-policy` · DPPO | runs, no learning (E17, E18) | runs, no learning (E28) | runs, no learning (E28) | runs (E27) | +| `dppo-policy` · DPPO | learns (E28) | **learns (E31)** | **learns (E31)** | runs (E27) | + +--- + +## The curves + +Mean return over windows of ten iterations (the first: iteration 1 alone), +and each seed's best single iteration: + +| cell, seed | 1 | 11-20 | 41-50 | 91-100 | 141-150 | 191-200 | 241-250 | 291-300 | best | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| Hopper, 0 | 11 | 76 | 189 | 336 | 490 | 685 | 629 | 873 | 1,073 (298) | +| Hopper, 1 | 12 | 73 | 188 | 405 | 707 | 831 | 1,005 | 961 | 1,336 (249) | +| Hopper, 2 | 16 | 80 | 207 | 350 | 609 | 753 | 731 | 812 | 1,122 (189) | +| Walker2d, 0 | −2 | 172 | 354 | 448 | 499 | 602 | 530 | 528 | 996 (186) | +| Walker2d, 1 | −2 | 172 | 243 | 436 | 591 | 540 | 671 | 622 | 983 (140) | +| Walker2d, 2 | −3 | 152 | 230 | 393 | 534 | 477 | 547 | 585 | 946 (240) | + +Hopper climbs through all 300 iterations; by 191-200 every seed is past 500, +and episodes at the end last 456 to 652 steps, from 17 to 21. Walker2d's +ten-iteration mean is past 500 by 141-150 on seeds 1 and 2 and by 191-200 on +seed 0, and from there the windows above stay between 477 and 671: seed 0 +ended below its own 191-200 mean. Its end, 528 to 622, is a pass, not a +plateau far above the bar. + +--- + +## Where my expectation went wrong + +For Walker2d I had written "falsified": between iterations 41-50 and 91-100 +E28 gained 9, 69 and 7, and at that pace it would have ended between about +300 and 560. E31 did not repeat E28's first hundred iterations. At the same +point, iterations 91-100, E31's Walker2d stood at 448 / 436 / 393 where +E28's stood at 260 / 292 / 274 - higher on every seed, by 101 to 188. +Hopper, by contrast, came out close to E28 at that point (336 / 405 / 350 +against 339 / 430 / 276). + +Nothing in the code tells the two runs apart. Between E28's `d1f4783` and +E31's `f8ffec0` the DPPO path changed twice, and neither changes a number: +the log-probability clamp became configurable with its old bounds as the +default (#64), and a storage read wraps its result in `np.asarray`, which +returns the same array for every index DPPO uses (#58). Every seed's first +iteration matches E28's, and after the first update runs are not +reproducible bit for bit (E20, declared as known item 2). What E28 read as +Walker2d flattening after about fifty iterations was one run's curve, not a +property of the cell: E28's 100 iterations and E31's first 100 are two draws +of the same thing, and they differ by more than E28's gain over its last +fifty. + +--- + +## What this does not show + +* That 300 iterations is what either task needs. Both were past 500 in the + ten-iteration windows by 191-200 on every seed; the protocol fixed 300 + before any data, and nothing shorter was read. +* How the cells compare with DPPO's published MuJoCo results. The bar is + this project's, set in E23 for these tasks, and 1.2 million steps is + about the length of the standard benchmark, not its protocol. +* Anything about `fpo-policy` under DPPO, which E30 is testing. + +--- + +## Files + +* `results/run.out` - the launcher's log: code commit, client commit, end + time and the failed-seed count per cell (0 and 0). +* `results/.out`, `results//{server,client}-seed.log` - each + cell's launcher output and each seed's server and client log. +* `results/verdicts.txt` - `summarise.py`'s output, run on `guangzhao` + against the tensorboards and checkpoints, which stay there + (`~/zuogou/plugrl/e31/experiments/e31-dppo-longer/results//dppo/`). +* `summary.tsv` - first and last-ten return and episode length per seed. diff --git a/experiments/e31-dppo-longer/PROTOCOL.md b/experiments/e31-dppo-longer/PROTOCOL.md new file mode 100644 index 0000000..60b9eb2 --- /dev/null +++ b/experiments/e31-dppo-longer/PROTOCOL.md @@ -0,0 +1,101 @@ +# E31 measurement protocol (pre-registered) + +**Written 2026-09-27, before any E31 run.** + +This file must not be edited after the first registered data point. Anything +learned afterwards goes in `AMENDMENT.md`, dated. + +--- + +## The question + +E28 ran `dppo-policy` under DPPO on Hopper and Walker2d for 100 iterations +(409,600 steps). Both rose on every seed and neither reached the bar of 500: +Hopper ended at 339, 430 and 276 and was still climbing; Walker2d ended at +260, 292 and 274 and had been flat since about iteration 50. **Run long +enough, does either learn its task?** + +Same cells, same settings, three times as long: 300 iterations, 1,228,800 +environment steps per seed - about the length of the standard million-step +MuJoCo benchmark, chosen before any data. + +### What this cannot settle + +* Why either stops where it does. Nothing but the length changes. +* Whether a different setting would do better. None is tried; the bar does + not move. + +--- + +## Declared in advance: what was already known + +1. **E28's two cells** (`experiments/e28-dppo-learns/`), mean return per ten + iterations, seeds 0 / 1 / 2: + + | cell | 11-20 | 41-50 | 91-100 | + | --- | --- | --- | --- | + | `dppo-hopper` | 77 / 74 / 80 | 195 / 214 / 201 | 339 / 430 / 276 | + | `dppo-walker` | 186 / 172 / 155 | 251 / 224 / 267 | 260 / 292 / 274 | + + The best single Hopper iteration was 493 (seed 1). +2. **Learning is not reproducible bit for bit** once the first update has + run (E20). E31's first 100 iterations are E28's settings on the same + machine, not E28's numbers. + +--- + +## Design + +| cell | policy | algorithm | task | iterations | buffer / batch | seeds | +| --- | --- | --- | --- | --- | --- | --- | +| `dppo-hopper` | `dppo-policy hopper` | `dppo hopper` | Hopper-v5 | 300 | 1,024 / 512 | 0, 1, 2 | +| `dppo-walker` | `dppo-policy walker` | `dppo walker` | Walker2d-v5 | 300 | 1,024 / 512 | 0, 1, 2 | + +Both run through E24's `run_cell.sh`, unchanged, exactly as E28 ran them: +chunks of 4, so 4,096 environment steps per iteration; the variants' +10-iteration warm-up and then a constant learning rate; checkpoints every 20 +iterations. Machine `guangzhao`, CPU, one thread per process. Code `main`, +client plugrl-env-client at 931ab56. + +--- + +## The status rule + +A cell **learns** if the mean return of iterations 291-300 is at least +**500** on at least **2 of 3** seeds - E23's, E24's and E28's bar for these +tasks. Otherwise it **did not learn in 1,228,800 steps**. + +--- + +## Predictions, and what falsifies each + +**P1 - both cells run end to end**: 300 iterations logged, the checkpoint at +300, no traceback, client exit 0, on every seed. + +**P2 - `dppo-policy` learns Hopper.** `dppo-hopper` meets the status rule. + +> Grounds: +62 to +66 by iterations 11-20, +184 to +202 by 41-50, +260 to +> +417 by 91-100, and still rising. Falsified if fewer than two seeds reach +> 500. + +**P3 - `dppo-policy` learns Walker2d.** `dppo-walker` meets the status rule. + +> What I expect: falsified. Between iterations 41-50 and 91-100 Walker2d +> gained 9, 69 and 7; at that pace it ends between about 300 and 560, most +> seeds below 500. Registered so that either outcome reads. + +**Reported, not predicted:** mean return over iterations 91-100 (E28's +length), 191-200 and 291-300 for every seed; wall clock. + +--- + +## Declared deviations allowed in advance + +1. One restart of any run that dies for a reason outside the experiment, + recorded in `AMENDMENT.md`. + +--- + +## Reading order + +P1, the status rule, P2, P3, then the reported figures. diff --git a/experiments/e31-dppo-longer/results/dppo-hopper.out b/experiments/e31-dppo-longer/results/dppo-hopper.out new file mode 100644 index 0000000..7218daa --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper.out @@ -0,0 +1,10 @@ +cell: dppo-hopper = dppo-policy/hopper x dppo/hopper x Hopper-v5 +iters: 300 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper +start 2026-09-27 04:42:08 +seed 1 finished rc=0 at 05:30:06 +seed 0 finished rc=0 at 05:30:18 +seed 2 finished rc=0 at 05:30:22 +end 2026-09-27 05:30:22 +failed seeds: 0 +CELL_DONE dppo-hopper diff --git a/experiments/e31-dppo-longer/results/dppo-hopper/client-seed0.log b/experiments/e31-dppo-longer/results/dppo-hopper/client-seed0.log new file mode 100644 index 0000000..0d005f6 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper/client-seed0.log @@ -0,0 +1,104 @@ +2026-09-27 04:42:10.447 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-27 04:42:10.447 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-27 04:42:10.448 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-27 04:42:10.448 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-27 04:42:10.449 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-27 04:42:10.490 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260927-044210-a5e44632 output_dir=runs/mujoco-v1-nenv1-20260927-044210-a5e44632 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-27 04:42:10.527 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9800... +2026-09-27 04:42:10.530 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-27 04:42:40.534 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14829 infer_calls=3999 feedback_calls=3998 infer_wait=27.646s infer_obs_pack=0.058s env_step=1.549s feedback_total=0.331s feedback_obs_pack=0.081s feedback_info_pack=0.021s effective_fps=501.25 +2026-09-27 04:43:10.534 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27668 infer_calls=7349 feedback_calls=7349 infer_wait=55.504s infer_obs_pack=0.108s env_step=2.973s feedback_total=0.648s feedback_obs_pack=0.159s feedback_info_pack=0.039s effective_fps=467.10 +2026-09-27 04:43:40.538 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42508 infer_calls=11179 feedback_calls=11178 infer_wait=82.892s infer_obs_pack=0.169s env_step=4.711s feedback_total=1.056s feedback_obs_pack=0.256s feedback_info_pack=0.060s effective_fps=478.54 +2026-09-27 04:44:11.103 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54737 infer_calls=14338 feedback_calls=14337 infer_wait=111.188s infer_obs_pack=0.220s env_step=6.206s feedback_total=1.432s feedback_obs_pack=0.352s feedback_info_pack=0.078s effective_fps=459.80 +2026-09-27 04:44:41.107 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68627 infer_calls=17922 feedback_calls=17921 infer_wait=138.611s infer_obs_pack=0.279s env_step=7.927s feedback_total=1.830s feedback_obs_pack=0.446s feedback_info_pack=0.100s effective_fps=461.68 +2026-09-27 04:45:13.590 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=82522 infer_calls=21506 feedback_calls=21505 infer_wait=168.634s infer_obs_pack=0.334s env_step=9.562s feedback_total=2.208s feedback_obs_pack=0.537s feedback_info_pack=0.121s effective_fps=456.58 +2026-09-27 04:45:43.593 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=97122 infer_calls=25263 feedback_calls=25262 infer_wait=196.068s infer_obs_pack=0.393s env_step=11.264s feedback_total=2.614s feedback_obs_pack=0.632s feedback_info_pack=0.142s effective_fps=461.74 +2026-09-27 04:46:14.602 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110450 infer_calls=28674 feedback_calls=28673 infer_wait=224.737s infer_obs_pack=0.445s env_step=12.823s feedback_total=2.985s feedback_obs_pack=0.718s feedback_info_pack=0.161s effective_fps=458.32 +2026-09-27 04:46:44.608 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=125409 infer_calls=32496 feedback_calls=32495 infer_wait=252.182s infer_obs_pack=0.506s env_step=14.526s feedback_total=3.384s feedback_obs_pack=0.811s feedback_info_pack=0.182s effective_fps=463.45 +2026-09-27 04:47:14.610 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=138672 infer_calls=35870 feedback_calls=35869 infer_wait=280.017s infer_obs_pack=0.557s env_step=15.979s feedback_total=3.710s feedback_obs_pack=0.890s feedback_info_pack=0.200s effective_fps=461.83 +2026-09-27 04:47:44.613 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=154129 infer_calls=39808 feedback_calls=39807 infer_wait=307.586s infer_obs_pack=0.613s env_step=17.614s feedback_total=4.075s feedback_obs_pack=0.981s feedback_info_pack=0.220s effective_fps=467.22 +2026-09-27 04:48:14.614 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=167398 infer_calls=43176 feedback_calls=43175 infer_wait=335.488s infer_obs_pack=0.661s env_step=19.028s feedback_total=4.385s feedback_obs_pack=1.058s feedback_info_pack=0.238s effective_fps=465.56 +2026-09-27 04:48:47.361 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=182883 infer_calls=47106 feedback_calls=47105 infer_wait=365.795s infer_obs_pack=0.718s env_step=20.674s feedback_total=4.746s feedback_obs_pack=1.149s feedback_info_pack=0.258s effective_fps=466.62 +2026-09-27 04:49:17.364 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=198349 infer_calls=51032 feedback_calls=51031 infer_wait=393.313s infer_obs_pack=0.774s env_step=22.349s feedback_total=5.119s feedback_obs_pack=1.241s feedback_info_pack=0.279s effective_fps=470.52 +2026-09-27 04:49:47.370 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=211622 infer_calls=54397 feedback_calls=54396 infer_wait=421.224s infer_obs_pack=0.823s env_step=23.768s feedback_total=5.435s feedback_obs_pack=1.316s feedback_info_pack=0.296s effective_fps=468.97 +2026-09-27 04:50:17.375 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=226939 infer_calls=58276 feedback_calls=58275 infer_wait=448.827s infer_obs_pack=0.878s env_step=25.402s feedback_total=5.790s feedback_obs_pack=1.403s feedback_info_pack=0.315s effective_fps=471.91 +2026-09-27 04:50:47.399 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=239453 infer_calls=61442 feedback_calls=61441 infer_wait=476.785s infer_obs_pack=0.928s env_step=26.799s feedback_total=6.100s feedback_obs_pack=1.478s feedback_info_pack=0.332s effective_fps=468.95 +2026-09-27 04:51:17.402 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=255198 infer_calls=65424 feedback_calls=65423 infer_wait=504.361s infer_obs_pack=0.984s env_step=28.441s feedback_total=6.460s feedback_obs_pack=1.566s feedback_info_pack=0.354s effective_fps=472.37 +2026-09-27 04:51:47.978 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=267805 infer_calls=68610 feedback_calls=68609 infer_wait=532.730s infer_obs_pack=1.035s env_step=29.930s feedback_total=6.804s feedback_obs_pack=1.650s feedback_info_pack=0.372s effective_fps=469.42 +2026-09-27 04:52:17.982 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=281059 infer_calls=71960 feedback_calls=71959 infer_wait=560.317s infer_obs_pack=1.091s env_step=31.569s feedback_total=7.177s feedback_obs_pack=1.736s feedback_info_pack=0.392s effective_fps=468.31 +2026-09-27 04:52:47.983 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=292825 infer_calls=74932 feedback_calls=74931 infer_wait=588.006s infer_obs_pack=1.144s env_step=33.144s feedback_total=7.529s feedback_obs_pack=1.820s feedback_info_pack=0.410s effective_fps=464.93 +2026-09-27 04:53:20.915 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=308338 infer_calls=78850 feedback_calls=78849 infer_wait=618.451s infer_obs_pack=1.201s env_step=34.824s feedback_total=7.904s feedback_obs_pack=1.912s feedback_info_pack=0.431s effective_fps=465.50 +2026-09-27 04:53:50.917 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=320743 infer_calls=81982 feedback_calls=81981 infer_wait=646.029s infer_obs_pack=1.255s env_step=36.485s feedback_total=8.268s feedback_obs_pack=2.004s feedback_info_pack=0.451s effective_fps=463.48 +2026-09-27 04:54:23.862 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=332678 infer_calls=84994 feedback_calls=84993 infer_wait=676.427s infer_obs_pack=1.313s env_step=38.213s feedback_total=8.659s feedback_obs_pack=2.101s feedback_info_pack=0.474s effective_fps=459.11 +2026-09-27 04:54:54.120 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=344849 infer_calls=88066 feedback_calls=88065 infer_wait=704.282s infer_obs_pack=1.369s env_step=39.823s feedback_total=9.041s feedback_obs_pack=2.195s feedback_info_pack=0.498s effective_fps=457.05 +2026-09-27 04:55:25.097 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=357026 infer_calls=91138 feedback_calls=91137 infer_wait=732.795s infer_obs_pack=1.426s env_step=41.480s feedback_total=9.442s feedback_obs_pack=2.298s feedback_info_pack=0.518s effective_fps=454.73 +2026-09-27 04:55:56.122 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=369237 infer_calls=94210 feedback_calls=94209 infer_wait=761.302s infer_obs_pack=1.487s env_step=43.191s feedback_total=9.824s feedback_obs_pack=2.395s feedback_info_pack=0.538s effective_fps=452.61 +2026-09-27 04:56:27.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=381449 infer_calls=97282 feedback_calls=97281 infer_wait=790.289s infer_obs_pack=1.545s env_step=44.907s feedback_total=10.233s feedback_obs_pack=2.488s feedback_info_pack=0.561s effective_fps=450.37 +2026-09-27 04:56:58.095 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=393622 infer_calls=100354 feedback_calls=100353 infer_wait=818.322s infer_obs_pack=1.604s env_step=46.542s feedback_total=10.612s feedback_obs_pack=2.578s feedback_info_pack=0.580s effective_fps=448.79 +2026-09-27 04:57:29.160 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=405820 infer_calls=103426 feedback_calls=103425 infer_wait=846.912s infer_obs_pack=1.664s env_step=48.209s feedback_total=11.009s feedback_obs_pack=2.674s feedback_info_pack=0.600s effective_fps=447.04 +2026-09-27 04:58:00.602 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=418026 infer_calls=106497 feedback_calls=106496 infer_wait=875.778s infer_obs_pack=1.726s env_step=49.946s feedback_total=11.422s feedback_obs_pack=2.769s feedback_info_pack=0.624s effective_fps=445.24 +2026-09-27 04:58:32.456 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=430226 infer_calls=109570 feedback_calls=109569 infer_wait=905.023s infer_obs_pack=1.788s env_step=51.696s feedback_total=11.837s feedback_obs_pack=2.867s feedback_info_pack=0.646s effective_fps=443.37 +2026-09-27 04:59:05.818 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=442444 infer_calls=112642 feedback_calls=112641 infer_wait=935.775s infer_obs_pack=1.848s env_step=53.451s feedback_total=12.254s feedback_obs_pack=2.968s feedback_info_pack=0.667s effective_fps=440.98 +2026-09-27 04:59:35.819 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=454252 infer_calls=115616 feedback_calls=115615 infer_wait=963.197s infer_obs_pack=1.912s env_step=55.162s feedback_total=12.681s feedback_obs_pack=3.068s feedback_info_pack=0.687s effective_fps=439.76 +2026-09-27 05:00:05.823 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=463358 infer_calls=117906 feedback_calls=117905 infer_wait=991.167s infer_obs_pack=1.956s env_step=56.541s feedback_total=13.012s feedback_obs_pack=3.145s feedback_info_pack=0.707s effective_fps=436.03 +2026-09-27 05:00:35.826 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=473976 infer_calls=120578 feedback_calls=120577 infer_wait=1018.784s infer_obs_pack=2.017s env_step=58.135s feedback_total=13.397s feedback_obs_pack=3.240s feedback_info_pack=0.726s effective_fps=433.91 +2026-09-27 05:01:05.828 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=485059 infer_calls=123367 feedback_calls=123366 infer_wait=1046.494s infer_obs_pack=2.071s env_step=59.665s feedback_total=13.756s feedback_obs_pack=3.324s feedback_info_pack=0.744s effective_fps=432.32 +2026-09-27 05:01:35.833 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=497546 infer_calls=126510 feedback_calls=126509 infer_wait=1074.038s infer_obs_pack=2.129s env_step=61.331s feedback_total=14.134s feedback_obs_pack=3.414s feedback_info_pack=0.764s effective_fps=432.04 +2026-09-27 05:02:05.834 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=510162 infer_calls=129684 feedback_calls=129683 infer_wait=1101.565s infer_obs_pack=2.189s env_step=63.027s feedback_total=14.503s feedback_obs_pack=3.507s feedback_info_pack=0.785s effective_fps=431.87 +2026-09-27 05:02:35.839 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=522837 infer_calls=132877 feedback_calls=132876 infer_wait=1129.102s infer_obs_pack=2.246s env_step=64.696s feedback_total=14.878s feedback_obs_pack=3.597s feedback_info_pack=0.805s effective_fps=431.77 +2026-09-27 05:03:05.843 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=535555 infer_calls=136077 feedback_calls=136076 infer_wait=1156.678s infer_obs_pack=2.303s env_step=66.338s feedback_total=15.258s feedback_obs_pack=3.689s feedback_info_pack=0.826s effective_fps=431.70 +2026-09-27 05:03:37.793 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=548222 infer_calls=139266 feedback_calls=139265 infer_wait=1186.228s infer_obs_pack=2.358s env_step=67.958s feedback_total=15.636s feedback_obs_pack=3.781s feedback_info_pack=0.846s effective_fps=430.93 +2026-09-27 05:04:07.795 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=560909 infer_calls=142456 feedback_calls=142456 infer_wait=1213.788s infer_obs_pack=2.414s env_step=69.612s feedback_total=16.011s feedback_obs_pack=3.875s feedback_info_pack=0.865s effective_fps=430.86 +2026-09-27 05:04:37.795 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=573993 infer_calls=145753 feedback_calls=145752 infer_wait=1241.305s infer_obs_pack=2.476s env_step=71.295s feedback_total=16.389s feedback_obs_pack=3.967s feedback_info_pack=0.886s effective_fps=431.10 +2026-09-27 05:05:07.798 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=586921 infer_calls=149002 feedback_calls=149001 infer_wait=1268.803s infer_obs_pack=2.538s env_step=72.999s feedback_total=16.773s feedback_obs_pack=4.061s feedback_info_pack=0.906s effective_fps=431.21 +2026-09-27 05:05:37.799 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=599621 infer_calls=152194 feedback_calls=152193 infer_wait=1296.347s infer_obs_pack=2.595s env_step=74.678s feedback_total=17.139s feedback_obs_pack=4.152s feedback_info_pack=0.925s effective_fps=431.15 +2026-09-27 05:06:07.799 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=613164 infer_calls=155596 feedback_calls=155595 infer_wait=1323.837s infer_obs_pack=2.653s env_step=76.368s feedback_total=17.539s feedback_obs_pack=4.248s feedback_info_pack=0.945s effective_fps=431.68 +2026-09-27 05:06:40.734 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=625597 infer_calls=158722 feedback_calls=158721 infer_wait=1354.372s infer_obs_pack=2.710s env_step=78.004s feedback_total=17.905s feedback_obs_pack=4.338s feedback_info_pack=0.965s effective_fps=430.56 +2026-09-27 05:07:10.739 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=638639 infer_calls=162001 feedback_calls=162000 infer_wait=1381.846s infer_obs_pack=2.769s env_step=79.720s feedback_total=18.296s feedback_obs_pack=4.432s feedback_info_pack=0.986s effective_fps=430.75 +2026-09-27 05:07:40.745 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=651055 infer_calls=165122 feedback_calls=165121 infer_wait=1409.373s infer_obs_pack=2.828s env_step=81.415s feedback_total=18.665s feedback_obs_pack=4.523s feedback_info_pack=1.006s effective_fps=430.51 +2026-09-27 05:08:10.752 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=663180 infer_calls=168167 feedback_calls=168166 infer_wait=1436.976s infer_obs_pack=2.886s env_step=83.059s feedback_total=19.022s feedback_obs_pack=4.619s feedback_info_pack=1.025s effective_fps=430.09 +2026-09-27 05:08:40.758 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=676444 infer_calls=171497 feedback_calls=171496 infer_wait=1464.426s infer_obs_pack=2.944s env_step=84.793s feedback_total=19.420s feedback_obs_pack=4.716s feedback_info_pack=1.046s effective_fps=430.42 +2026-09-27 05:09:10.762 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=690267 infer_calls=174963 feedback_calls=174962 infer_wait=1491.888s infer_obs_pack=3.003s env_step=86.502s feedback_total=19.835s feedback_obs_pack=4.809s feedback_info_pack=1.066s effective_fps=431.09 +2026-09-27 05:09:43.639 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=703078 infer_calls=178178 feedback_calls=178177 infer_wait=1522.262s infer_obs_pack=3.061s env_step=88.217s feedback_total=20.210s feedback_obs_pack=4.905s feedback_info_pack=1.086s effective_fps=430.35 +2026-09-27 05:10:13.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=716564 infer_calls=181564 feedback_calls=181563 infer_wait=1549.725s infer_obs_pack=3.119s env_step=89.932s feedback_total=20.611s feedback_obs_pack=5.001s feedback_info_pack=1.107s effective_fps=430.79 +2026-09-27 05:10:43.644 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=729522 infer_calls=184817 feedback_calls=184816 infer_wait=1577.199s infer_obs_pack=3.177s env_step=91.650s feedback_total=21.007s feedback_obs_pack=5.096s feedback_info_pack=1.127s effective_fps=430.90 +2026-09-27 05:11:13.647 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=741918 infer_calls=187925 feedback_calls=187924 infer_wait=1604.745s infer_obs_pack=3.234s env_step=93.335s feedback_total=21.376s feedback_obs_pack=5.197s feedback_info_pack=1.146s effective_fps=430.67 +2026-09-27 05:11:43.651 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=754933 infer_calls=191187 feedback_calls=191186 infer_wait=1632.222s infer_obs_pack=3.299s env_step=95.052s feedback_total=21.762s feedback_obs_pack=5.290s feedback_info_pack=1.166s effective_fps=430.82 +2026-09-27 05:12:13.651 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=767676 infer_calls=194381 feedback_calls=194380 infer_wait=1659.784s infer_obs_pack=3.354s env_step=96.711s feedback_total=22.146s feedback_obs_pack=5.379s feedback_info_pack=1.187s effective_fps=430.80 +2026-09-27 05:12:45.883 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=780671 infer_calls=197634 feedback_calls=197633 infer_wait=1689.577s infer_obs_pack=3.411s env_step=98.351s feedback_total=22.540s feedback_obs_pack=5.472s feedback_info_pack=1.206s effective_fps=430.39 +2026-09-27 05:13:15.883 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=793805 infer_calls=200925 feedback_calls=200924 infer_wait=1717.095s infer_obs_pack=3.470s env_step=100.036s feedback_total=22.931s feedback_obs_pack=5.564s feedback_info_pack=1.227s effective_fps=430.59 +2026-09-27 05:13:45.888 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=806945 infer_calls=204223 feedback_calls=204222 infer_wait=1744.540s infer_obs_pack=3.530s env_step=101.779s feedback_total=23.328s feedback_obs_pack=5.662s feedback_info_pack=1.247s effective_fps=430.79 +2026-09-27 05:14:15.891 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=819563 infer_calls=207390 feedback_calls=207389 infer_wait=1772.103s infer_obs_pack=3.587s env_step=103.441s feedback_total=23.699s feedback_obs_pack=5.753s feedback_info_pack=1.267s effective_fps=430.71 +2026-09-27 05:14:45.892 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=832382 infer_calls=210605 feedback_calls=210604 infer_wait=1799.690s infer_obs_pack=3.644s env_step=105.079s feedback_total=24.073s feedback_obs_pack=5.842s feedback_info_pack=1.287s effective_fps=430.73 +2026-09-27 05:15:15.894 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=845445 infer_calls=213882 feedback_calls=213881 infer_wait=1827.221s infer_obs_pack=3.702s env_step=106.760s feedback_total=24.451s feedback_obs_pack=5.934s feedback_info_pack=1.310s effective_fps=430.88 +2026-09-27 05:15:45.895 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=858090 infer_calls=217055 feedback_calls=217054 infer_wait=1854.747s infer_obs_pack=3.762s env_step=108.439s feedback_total=24.829s feedback_obs_pack=6.026s feedback_info_pack=1.330s effective_fps=430.82 +2026-09-27 05:16:17.716 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=870470 infer_calls=220162 feedback_calls=220161 infer_wait=1884.252s infer_obs_pack=3.816s env_step=110.005s feedback_total=25.191s feedback_obs_pack=6.112s feedback_info_pack=1.349s effective_fps=430.23 +2026-09-27 05:16:47.725 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=883747 infer_calls=223493 feedback_calls=223492 infer_wait=1911.693s infer_obs_pack=3.875s env_step=111.763s feedback_total=25.581s feedback_obs_pack=6.209s feedback_info_pack=1.372s effective_fps=430.48 +2026-09-27 05:17:17.726 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=896751 infer_calls=226754 feedback_calls=226753 infer_wait=1939.208s infer_obs_pack=3.933s env_step=113.470s feedback_total=25.951s feedback_obs_pack=6.298s feedback_info_pack=1.392s effective_fps=430.60 +2026-09-27 05:17:47.730 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=910901 infer_calls=230298 feedback_calls=230297 infer_wait=1966.653s infer_obs_pack=3.992s env_step=115.213s feedback_total=26.345s feedback_obs_pack=6.393s feedback_info_pack=1.413s effective_fps=431.26 +2026-09-27 05:18:18.522 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=923561 infer_calls=233473 feedback_calls=233472 infer_wait=1995.137s infer_obs_pack=4.046s env_step=116.783s feedback_total=26.699s feedback_obs_pack=6.476s feedback_info_pack=1.431s effective_fps=431.03 +2026-09-27 05:18:48.525 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=938180 infer_calls=237140 feedback_calls=237139 infer_wait=2022.524s infer_obs_pack=4.111s env_step=118.548s feedback_total=27.116s feedback_obs_pack=6.571s feedback_info_pack=1.452s effective_fps=431.88 +2026-09-27 05:19:20.825 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=952150 infer_calls=240642 feedback_calls=240641 infer_wait=2052.314s infer_obs_pack=4.169s env_step=120.249s feedback_total=27.507s feedback_obs_pack=6.662s feedback_info_pack=1.472s effective_fps=431.96 +2026-09-27 05:19:50.827 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=966695 infer_calls=244288 feedback_calls=244287 infer_wait=2079.704s infer_obs_pack=4.230s env_step=122.014s feedback_total=27.918s feedback_obs_pack=6.758s feedback_info_pack=1.493s effective_fps=432.75 +2026-09-27 05:20:23.151 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=980723 infer_calls=247810 feedback_calls=247809 infer_wait=2109.498s infer_obs_pack=4.290s env_step=123.731s feedback_total=28.304s feedback_obs_pack=6.851s feedback_info_pack=1.513s effective_fps=432.83 +2026-09-27 05:20:53.151 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=995196 infer_calls=251441 feedback_calls=251440 infer_wait=2136.918s infer_obs_pack=4.351s env_step=125.467s feedback_total=28.710s feedback_obs_pack=6.946s feedback_info_pack=1.534s effective_fps=433.55 +2026-09-27 05:21:25.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1009260 infer_calls=254978 feedback_calls=254977 infer_wait=2166.667s infer_obs_pack=4.411s env_step=127.174s feedback_total=29.096s feedback_obs_pack=7.040s feedback_info_pack=1.555s effective_fps=433.65 +2026-09-27 05:21:55.420 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1023394 infer_calls=258526 feedback_calls=258525 infer_wait=2194.129s infer_obs_pack=4.473s env_step=128.899s feedback_total=29.477s feedback_obs_pack=7.133s feedback_info_pack=1.575s effective_fps=434.20 +2026-09-27 05:22:28.078 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1037808 infer_calls=262146 feedback_calls=262145 infer_wait=2224.220s infer_obs_pack=4.533s env_step=130.626s feedback_total=29.885s feedback_obs_pack=7.228s feedback_info_pack=1.596s effective_fps=434.36 +2026-09-27 05:22:58.081 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1052388 infer_calls=265807 feedback_calls=265806 infer_wait=2251.698s infer_obs_pack=4.594s env_step=132.335s feedback_total=30.276s feedback_obs_pack=7.320s feedback_info_pack=1.617s effective_fps=435.07 +2026-09-27 05:23:30.261 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1066339 infer_calls=269314 feedback_calls=269313 infer_wait=2281.459s infer_obs_pack=4.651s env_step=133.965s feedback_total=30.652s feedback_obs_pack=7.410s feedback_info_pack=1.636s effective_fps=435.11 +2026-09-27 05:24:00.261 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1079814 infer_calls=272697 feedback_calls=272696 infer_wait=2308.990s infer_obs_pack=4.711s env_step=135.635s feedback_total=31.032s feedback_obs_pack=7.510s feedback_info_pack=1.657s effective_fps=435.34 +2026-09-27 05:24:30.264 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1094731 infer_calls=276440 feedback_calls=276439 infer_wait=2336.434s infer_obs_pack=4.771s env_step=137.371s feedback_total=31.419s feedback_obs_pack=7.602s feedback_info_pack=1.678s effective_fps=436.15 +2026-09-27 05:25:01.226 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1107143 infer_calls=279554 feedback_calls=279553 infer_wait=2365.129s infer_obs_pack=4.824s env_step=138.903s feedback_total=31.770s feedback_obs_pack=7.687s feedback_info_pack=1.696s effective_fps=435.78 +2026-09-27 05:25:34.028 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1119386 infer_calls=282626 feedback_calls=282625 infer_wait=2395.373s infer_obs_pack=4.888s env_step=140.622s feedback_total=32.167s feedback_obs_pack=7.782s feedback_info_pack=1.717s effective_fps=435.04 +2026-09-27 05:26:04.029 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1131540 infer_calls=285674 feedback_calls=285673 infer_wait=2422.861s infer_obs_pack=4.950s env_step=142.309s feedback_total=32.562s feedback_obs_pack=7.876s feedback_info_pack=1.739s effective_fps=434.76 +2026-09-27 05:26:34.031 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1143099 infer_calls=288574 feedback_calls=288573 infer_wait=2450.463s infer_obs_pack=5.007s env_step=143.921s feedback_total=32.948s feedback_obs_pack=7.971s feedback_info_pack=1.759s effective_fps=434.25 +2026-09-27 05:27:04.034 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1153140 infer_calls=291095 feedback_calls=291094 infer_wait=2478.334s infer_obs_pack=5.059s env_step=145.341s feedback_total=33.292s feedback_obs_pack=8.050s feedback_info_pack=1.778s effective_fps=433.18 +2026-09-27 05:27:34.040 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1163301 infer_calls=293646 feedback_calls=293645 infer_wait=2506.042s infer_obs_pack=5.115s env_step=146.878s feedback_total=33.670s feedback_obs_pack=8.139s feedback_info_pack=1.799s effective_fps=432.18 +2026-09-27 05:28:04.045 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1172596 infer_calls=295980 feedback_calls=295979 infer_wait=2533.963s infer_obs_pack=5.164s env_step=148.261s feedback_total=34.016s feedback_obs_pack=8.222s feedback_info_pack=1.816s effective_fps=430.88 +2026-09-27 05:28:34.053 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1182906 infer_calls=298568 feedback_calls=298567 infer_wait=2561.653s infer_obs_pack=5.221s env_step=149.800s feedback_total=34.399s feedback_obs_pack=8.312s feedback_info_pack=1.836s effective_fps=429.98 +2026-09-27 05:29:05.401 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1192821 infer_calls=301058 feedback_calls=301057 infer_wait=2590.880s infer_obs_pack=5.271s env_step=151.201s feedback_total=34.758s feedback_obs_pack=8.394s feedback_info_pack=1.854s effective_fps=428.75 +2026-09-27 05:29:35.411 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1203962 infer_calls=303854 feedback_calls=303853 infer_wait=2618.496s infer_obs_pack=5.329s env_step=152.783s feedback_total=35.163s feedback_obs_pack=8.488s feedback_info_pack=1.874s effective_fps=428.19 +2026-09-27 05:30:06.403 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1213233 infer_calls=306178 feedback_calls=306177 infer_wait=2647.470s infer_obs_pack=5.376s env_step=154.130s feedback_total=35.492s feedback_obs_pack=8.566s feedback_info_pack=1.891s effective_fps=426.82 +2026-09-27 05:30:18.822 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=1217313 infer_calls=307201 feedback_calls=307201 infer_wait=2655.704s infer_obs_pack=5.397s env_step=154.736s feedback_total=35.644s feedback_obs_pack=8.599s feedback_info_pack=1.900s effective_fps=426.91 +2026-09-27 05:30:18.822 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 7664/1228800 episodes. diff --git a/experiments/e31-dppo-longer/results/dppo-hopper/client-seed1.log b/experiments/e31-dppo-longer/results/dppo-hopper/client-seed1.log new file mode 100644 index 0000000..89ba9e4 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper/client-seed1.log @@ -0,0 +1,104 @@ +2026-09-27 04:42:15.436 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-27 04:42:15.436 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-27 04:42:15.436 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-27 04:42:15.436 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-27 04:42:15.437 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-27 04:42:15.478 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260927-044215-76627aac output_dir=runs/mujoco-v1-nenv1-20260927-044215-76627aac recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-27 04:42:15.517 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9801... +2026-09-27 04:42:15.520 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-27 04:42:45.523 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14807 infer_calls=4001 feedback_calls=4000 infer_wait=27.620s infer_obs_pack=0.056s env_step=1.548s feedback_total=0.334s feedback_obs_pack=0.085s feedback_info_pack=0.018s effective_fps=500.94 +2026-09-27 04:43:15.523 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27921 infer_calls=7427 feedback_calls=7426 infer_wait=55.416s infer_obs_pack=0.110s env_step=3.000s feedback_total=0.674s feedback_obs_pack=0.165s feedback_info_pack=0.035s effective_fps=471.64 +2026-09-27 04:43:45.523 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42585 infer_calls=11214 feedback_calls=11214 infer_wait=82.824s infer_obs_pack=0.169s env_step=4.728s feedback_total=1.071s feedback_obs_pack=0.262s feedback_info_pack=0.056s effective_fps=479.61 +2026-09-27 04:44:15.624 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54659 infer_calls=14338 feedback_calls=14337 infer_wait=110.614s infer_obs_pack=0.220s env_step=6.243s feedback_total=1.457s feedback_obs_pack=0.375s feedback_info_pack=0.073s effective_fps=461.13 +2026-09-27 04:44:45.627 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=68666 infer_calls=17938 feedback_calls=17937 infer_wait=138.001s infer_obs_pack=0.279s env_step=7.994s feedback_total=1.861s feedback_obs_pack=0.473s feedback_info_pack=0.092s effective_fps=463.53 +2026-09-27 04:45:17.997 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=82567 infer_calls=21506 feedback_calls=21505 infer_wait=167.852s infer_obs_pack=0.339s env_step=9.681s feedback_total=2.245s feedback_obs_pack=0.567s feedback_info_pack=0.111s effective_fps=458.41 +2026-09-27 04:45:48.001 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=97457 infer_calls=25319 feedback_calls=25318 infer_wait=195.280s infer_obs_pack=0.398s env_step=11.400s feedback_total=2.646s feedback_obs_pack=0.662s feedback_info_pack=0.130s effective_fps=464.69 +2026-09-27 04:46:18.845 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110599 infer_calls=28674 feedback_calls=28673 infer_wait=223.769s infer_obs_pack=0.450s env_step=12.976s feedback_total=3.013s feedback_obs_pack=0.749s feedback_info_pack=0.148s effective_fps=460.43 +2026-09-27 04:46:48.846 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=125636 infer_calls=32502 feedback_calls=32501 infer_wait=251.259s infer_obs_pack=0.508s env_step=14.663s feedback_total=3.389s feedback_obs_pack=0.844s feedback_info_pack=0.167s effective_fps=465.63 +2026-09-27 04:47:18.849 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139275 infer_calls=35969 feedback_calls=35968 infer_wait=279.096s infer_obs_pack=0.558s env_step=16.117s feedback_total=3.711s feedback_obs_pack=0.925s feedback_info_pack=0.183s effective_fps=465.05 +2026-09-27 04:47:51.640 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=154894 infer_calls=39938 feedback_calls=39937 infer_wait=309.422s infer_obs_pack=0.617s env_step=17.770s feedback_total=4.079s feedback_obs_pack=1.017s feedback_info_pack=0.201s effective_fps=466.71 +2026-09-27 04:48:21.641 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=170318 infer_calls=43852 feedback_calls=43851 infer_wait=336.963s infer_obs_pack=0.675s env_step=19.432s feedback_total=4.436s feedback_obs_pack=1.107s feedback_info_pack=0.220s effective_fps=471.14 +2026-09-27 04:48:51.641 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=183762 infer_calls=47262 feedback_calls=47261 infer_wait=364.814s infer_obs_pack=0.725s env_step=20.879s feedback_total=4.754s feedback_obs_pack=1.187s feedback_info_pack=0.236s effective_fps=469.77 +2026-09-27 04:49:24.333 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=199303 infer_calls=51202 feedback_calls=51201 infer_wait=395.049s infer_obs_pack=0.781s env_step=22.550s feedback_total=5.106s feedback_obs_pack=1.278s feedback_info_pack=0.254s effective_fps=470.62 +2026-09-27 04:49:54.334 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=212906 infer_calls=54643 feedback_calls=54642 infer_wait=422.838s infer_obs_pack=0.833s env_step=24.069s feedback_total=5.424s feedback_obs_pack=1.355s feedback_info_pack=0.270s effective_fps=469.82 +2026-09-27 04:50:26.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=227650 infer_calls=58370 feedback_calls=58369 infer_wait=452.749s infer_obs_pack=0.888s env_step=25.700s feedback_total=5.783s feedback_obs_pack=1.442s feedback_info_pack=0.289s effective_fps=469.26 +2026-09-27 04:50:56.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=243211 infer_calls=62313 feedback_calls=62312 infer_wait=480.252s infer_obs_pack=0.945s env_step=27.392s feedback_total=6.147s feedback_obs_pack=1.531s feedback_info_pack=0.308s effective_fps=472.50 +2026-09-27 04:51:26.660 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=256955 infer_calls=65790 feedback_calls=65789 infer_wait=508.048s infer_obs_pack=0.995s env_step=28.888s feedback_total=6.471s feedback_obs_pack=1.612s feedback_info_pack=0.326s effective_fps=471.99 +2026-09-27 04:51:56.663 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=271453 infer_calls=69456 feedback_calls=69455 infer_wait=535.477s infer_obs_pack=1.054s env_step=30.625s feedback_total=6.864s feedback_obs_pack=1.704s feedback_info_pack=0.346s effective_fps=472.90 +2026-09-27 04:52:26.667 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=282974 infer_calls=72373 feedback_calls=72372 infer_wait=563.167s infer_obs_pack=1.114s env_step=32.199s feedback_total=7.212s feedback_obs_pack=1.789s feedback_info_pack=0.365s effective_fps=468.74 +2026-09-27 04:52:56.669 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=296052 infer_calls=75681 feedback_calls=75680 infer_wait=590.779s infer_obs_pack=1.169s env_step=33.829s feedback_total=7.567s feedback_obs_pack=1.881s feedback_info_pack=0.383s effective_fps=467.44 +2026-09-27 04:53:26.671 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=309084 infer_calls=78971 feedback_calls=78970 infer_wait=618.576s infer_obs_pack=1.219s env_step=35.328s feedback_total=7.895s feedback_obs_pack=1.961s feedback_info_pack=0.399s effective_fps=466.18 +2026-09-27 04:53:56.675 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=321159 infer_calls=82015 feedback_calls=82014 infer_wait=646.139s infer_obs_pack=1.278s env_step=36.985s feedback_total=8.270s feedback_obs_pack=2.054s feedback_info_pack=0.419s effective_fps=463.65 +2026-09-27 04:54:26.678 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=333493 infer_calls=85123 feedback_calls=85122 infer_wait=673.730s infer_obs_pack=1.335s env_step=38.607s feedback_total=8.652s feedback_obs_pack=2.145s feedback_info_pack=0.438s effective_fps=461.69 +2026-09-27 04:54:56.681 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=345557 infer_calls=88167 feedback_calls=88166 infer_wait=701.322s infer_obs_pack=1.392s env_step=40.246s feedback_total=9.007s feedback_obs_pack=2.234s feedback_info_pack=0.455s effective_fps=459.54 +2026-09-27 04:55:26.684 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=358185 infer_calls=91347 feedback_calls=91346 infer_wait=728.861s infer_obs_pack=1.447s env_step=41.909s feedback_total=9.395s feedback_obs_pack=2.327s feedback_info_pack=0.474s effective_fps=458.26 +2026-09-27 04:55:56.687 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=369624 infer_calls=94225 feedback_calls=94224 infer_wait=756.495s infer_obs_pack=1.502s env_step=43.517s feedback_total=9.765s feedback_obs_pack=2.420s feedback_info_pack=0.490s effective_fps=455.61 +2026-09-27 04:56:27.708 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=381786 infer_calls=97282 feedback_calls=97281 infer_wait=785.074s infer_obs_pack=1.567s env_step=45.156s feedback_total=10.153s feedback_obs_pack=2.512s feedback_info_pack=0.513s effective_fps=453.45 +2026-09-27 04:56:57.712 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=394042 infer_calls=100365 feedback_calls=100364 infer_wait=812.605s infer_obs_pack=1.623s env_step=46.831s feedback_total=10.550s feedback_obs_pack=2.615s feedback_info_pack=0.530s effective_fps=452.09 +2026-09-27 04:57:29.026 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=406208 infer_calls=103426 feedback_calls=103425 infer_wait=841.441s infer_obs_pack=1.681s env_step=48.521s feedback_total=10.923s feedback_obs_pack=2.717s feedback_info_pack=0.548s effective_fps=450.06 +2026-09-27 04:58:00.573 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=418410 infer_calls=106498 feedback_calls=106497 infer_wait=870.461s infer_obs_pack=1.739s env_step=50.224s feedback_total=11.328s feedback_obs_pack=2.816s feedback_info_pack=0.568s effective_fps=448.10 +2026-09-27 04:58:31.427 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=430625 infer_calls=109570 feedback_calls=109569 infer_wait=898.790s infer_obs_pack=1.795s env_step=51.926s feedback_total=11.736s feedback_obs_pack=2.922s feedback_info_pack=0.588s effective_fps=446.59 +2026-09-27 04:59:01.428 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=442779 infer_calls=112622 feedback_calls=112621 infer_wait=926.179s infer_obs_pack=1.856s env_step=53.682s feedback_total=12.148s feedback_obs_pack=3.027s feedback_info_pack=0.608s effective_fps=445.51 +2026-09-27 04:59:31.436 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=452318 infer_calls=115016 feedback_calls=115015 infer_wait=954.112s infer_obs_pack=1.911s env_step=55.077s feedback_total=12.483s feedback_obs_pack=3.105s feedback_info_pack=0.624s effective_fps=441.90 +2026-09-27 05:00:01.445 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=463141 infer_calls=117730 feedback_calls=117729 infer_wait=981.698s infer_obs_pack=1.970s env_step=56.714s feedback_total=12.876s feedback_obs_pack=3.200s feedback_info_pack=0.644s effective_fps=439.72 +2026-09-27 05:00:31.448 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=472475 infer_calls=120075 feedback_calls=120074 infer_wait=1009.631s infer_obs_pack=2.018s env_step=58.120s feedback_total=13.201s feedback_obs_pack=3.281s feedback_info_pack=0.663s effective_fps=436.28 +2026-09-27 05:01:02.649 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=483665 infer_calls=122882 feedback_calls=122881 infer_wait=1038.429s infer_obs_pack=2.084s env_step=59.749s feedback_total=13.564s feedback_obs_pack=3.374s feedback_info_pack=0.681s effective_fps=434.24 +2026-09-27 05:01:32.654 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=495930 infer_calls=125961 feedback_calls=125960 infer_wait=1065.999s infer_obs_pack=2.139s env_step=61.400s feedback_total=13.951s feedback_obs_pack=3.468s feedback_info_pack=0.701s effective_fps=433.70 +2026-09-27 05:02:02.656 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=508499 infer_calls=129115 feedback_calls=129114 infer_wait=1093.576s infer_obs_pack=2.193s env_step=63.055s feedback_total=14.320s feedback_obs_pack=3.558s feedback_info_pack=0.719s effective_fps=433.45 +2026-09-27 05:02:32.664 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=521842 infer_calls=132458 feedback_calls=132457 infer_wait=1121.090s infer_obs_pack=2.249s env_step=64.759s feedback_total=14.699s feedback_obs_pack=3.650s feedback_info_pack=0.737s effective_fps=433.86 +2026-09-27 05:03:02.672 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=534527 infer_calls=135639 feedback_calls=135638 infer_wait=1148.626s infer_obs_pack=2.306s env_step=66.454s feedback_total=15.070s feedback_obs_pack=3.745s feedback_info_pack=0.760s effective_fps=433.71 +2026-09-27 05:03:32.679 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=547365 infer_calls=138857 feedback_calls=138856 infer_wait=1176.159s infer_obs_pack=2.366s env_step=68.144s feedback_total=15.437s feedback_obs_pack=3.839s feedback_info_pack=0.780s effective_fps=433.69 +2026-09-27 05:04:02.682 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=559732 infer_calls=141956 feedback_calls=141955 infer_wait=1203.706s infer_obs_pack=2.429s env_step=69.826s feedback_total=15.801s feedback_obs_pack=3.928s feedback_info_pack=0.802s effective_fps=433.31 +2026-09-27 05:04:32.682 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=572583 infer_calls=145178 feedback_calls=145178 infer_wait=1231.175s infer_obs_pack=2.489s env_step=71.567s feedback_total=16.177s feedback_obs_pack=4.024s feedback_info_pack=0.824s effective_fps=433.31 +2026-09-27 05:05:02.682 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=585495 infer_calls=148415 feedback_calls=148414 infer_wait=1258.700s infer_obs_pack=2.549s env_step=73.255s feedback_total=16.553s feedback_obs_pack=4.116s feedback_info_pack=0.845s effective_fps=433.36 +2026-09-27 05:05:35.276 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=598029 infer_calls=151554 feedback_calls=151553 infer_wait=1288.921s infer_obs_pack=2.603s env_step=74.879s feedback_total=16.911s feedback_obs_pack=4.209s feedback_info_pack=0.865s effective_fps=432.32 +2026-09-27 05:06:05.280 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=611263 infer_calls=154870 feedback_calls=154869 infer_wait=1316.403s infer_obs_pack=2.664s env_step=76.610s feedback_total=17.285s feedback_obs_pack=4.303s feedback_info_pack=0.885s effective_fps=432.61 +2026-09-27 05:06:35.284 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=624096 infer_calls=158081 feedback_calls=158080 infer_wait=1343.916s infer_obs_pack=2.720s env_step=78.312s feedback_total=17.660s feedback_obs_pack=4.396s feedback_info_pack=0.906s effective_fps=432.62 +2026-09-27 05:07:05.287 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=637119 infer_calls=161342 feedback_calls=161341 infer_wait=1371.409s infer_obs_pack=2.777s env_step=80.027s feedback_total=18.051s feedback_obs_pack=4.486s feedback_info_pack=0.930s effective_fps=432.75 +2026-09-27 05:07:35.288 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=649918 infer_calls=164551 feedback_calls=164550 infer_wait=1398.917s infer_obs_pack=2.834s env_step=81.724s feedback_total=18.433s feedback_obs_pack=4.579s feedback_info_pack=0.950s effective_fps=432.73 +2026-09-27 05:08:05.290 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=662413 infer_calls=167681 feedback_calls=167680 infer_wait=1426.513s infer_obs_pack=2.889s env_step=83.379s feedback_total=18.788s feedback_obs_pack=4.668s feedback_info_pack=0.969s effective_fps=432.51 +2026-09-27 05:08:37.954 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=675702 infer_calls=171010 feedback_calls=171009 infer_wait=1456.715s infer_obs_pack=2.946s env_step=85.061s feedback_total=19.166s feedback_obs_pack=4.761s feedback_info_pack=0.990s effective_fps=432.07 +2026-09-27 05:09:07.956 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=689043 infer_calls=174352 feedback_calls=174351 infer_wait=1484.223s infer_obs_pack=3.002s env_step=86.754s feedback_total=19.554s feedback_obs_pack=4.852s feedback_info_pack=1.010s effective_fps=432.40 +2026-09-27 05:09:37.957 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=702145 infer_calls=177633 feedback_calls=177632 infer_wait=1511.719s infer_obs_pack=3.059s env_step=88.457s feedback_total=19.945s feedback_obs_pack=4.944s feedback_info_pack=1.029s effective_fps=432.57 +2026-09-27 05:10:07.960 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=714851 infer_calls=180819 feedback_calls=180818 infer_wait=1539.301s infer_obs_pack=3.115s env_step=90.102s feedback_total=20.322s feedback_obs_pack=5.033s feedback_info_pack=1.049s effective_fps=432.50 +2026-09-27 05:10:37.964 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=728433 infer_calls=184222 feedback_calls=184221 infer_wait=1566.752s infer_obs_pack=3.175s env_step=91.833s feedback_total=20.721s feedback_obs_pack=5.128s feedback_info_pack=1.070s effective_fps=432.95 +2026-09-27 05:11:10.600 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=741106 infer_calls=187394 feedback_calls=187393 infer_wait=1596.978s infer_obs_pack=3.232s env_step=93.471s feedback_total=21.095s feedback_obs_pack=5.222s feedback_info_pack=1.090s effective_fps=432.19 +2026-09-27 05:11:40.607 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=754392 infer_calls=190718 feedback_calls=190717 infer_wait=1624.445s infer_obs_pack=3.290s env_step=95.200s feedback_total=21.481s feedback_obs_pack=5.320s feedback_info_pack=1.111s effective_fps=432.46 +2026-09-27 05:12:10.612 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=767717 infer_calls=194053 feedback_calls=194052 infer_wait=1651.940s infer_obs_pack=3.348s env_step=96.913s feedback_total=21.869s feedback_obs_pack=5.416s feedback_info_pack=1.132s effective_fps=432.74 +2026-09-27 05:12:40.614 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=780794 infer_calls=197331 feedback_calls=197330 infer_wait=1679.437s infer_obs_pack=3.407s env_step=98.619s feedback_total=22.247s feedback_obs_pack=5.509s feedback_info_pack=1.153s effective_fps=432.88 +2026-09-27 05:13:13.282 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=794271 infer_calls=200706 feedback_calls=200705 infer_wait=1709.632s infer_obs_pack=3.465s env_step=100.295s feedback_total=22.637s feedback_obs_pack=5.603s feedback_info_pack=1.173s effective_fps=432.60 +2026-09-27 05:13:43.284 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=807542 infer_calls=204029 feedback_calls=204028 infer_wait=1737.105s infer_obs_pack=3.526s env_step=102.027s feedback_total=23.021s feedback_obs_pack=5.699s feedback_info_pack=1.193s effective_fps=432.84 +2026-09-27 05:14:13.290 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=820736 infer_calls=207332 feedback_calls=207331 infer_wait=1764.627s infer_obs_pack=3.591s env_step=103.714s feedback_total=23.393s feedback_obs_pack=5.794s feedback_info_pack=1.213s effective_fps=433.03 +2026-09-27 05:14:43.292 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=833996 infer_calls=210653 feedback_calls=210652 infer_wait=1792.185s infer_obs_pack=3.651s env_step=105.372s feedback_total=23.771s feedback_obs_pack=5.890s feedback_info_pack=1.232s effective_fps=433.25 +2026-09-27 05:15:15.960 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=847436 infer_calls=214018 feedback_calls=214017 infer_wait=1822.380s infer_obs_pack=3.710s env_step=107.041s feedback_total=24.161s feedback_obs_pack=5.983s feedback_info_pack=1.252s effective_fps=432.96 +2026-09-27 05:15:45.963 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=860937 infer_calls=217399 feedback_calls=217398 infer_wait=1849.879s infer_obs_pack=3.769s env_step=108.739s feedback_total=24.548s feedback_obs_pack=6.078s feedback_info_pack=1.270s effective_fps=433.30 +2026-09-27 05:16:15.968 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=873890 infer_calls=220644 feedback_calls=220643 infer_wait=1877.437s infer_obs_pack=3.827s env_step=110.404s feedback_total=24.920s feedback_obs_pack=6.170s feedback_info_pack=1.288s effective_fps=433.35 +2026-09-27 05:16:45.972 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=886850 infer_calls=223892 feedback_calls=223891 infer_wait=1904.976s infer_obs_pack=3.884s env_step=112.068s feedback_total=25.304s feedback_obs_pack=6.266s feedback_info_pack=1.309s effective_fps=433.41 +2026-09-27 05:17:15.973 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=899826 infer_calls=227145 feedback_calls=227144 infer_wait=1932.487s infer_obs_pack=3.941s env_step=113.762s feedback_total=25.684s feedback_obs_pack=6.360s feedback_info_pack=1.330s effective_fps=433.47 +2026-09-27 05:17:46.818 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=912823 infer_calls=230402 feedback_calls=230401 infer_wait=1960.990s infer_obs_pack=3.996s env_step=115.349s feedback_total=26.048s feedback_obs_pack=6.449s feedback_info_pack=1.348s effective_fps=433.36 +2026-09-27 05:18:16.821 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=927416 infer_calls=234062 feedback_calls=234061 infer_wait=1988.413s infer_obs_pack=4.057s env_step=117.103s feedback_total=26.442s feedback_obs_pack=6.544s feedback_info_pack=1.369s effective_fps=434.18 +2026-09-27 05:18:48.602 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=941412 infer_calls=237570 feedback_calls=237569 infer_wait=2017.777s infer_obs_pack=4.114s env_step=118.748s feedback_total=26.809s feedback_obs_pack=6.634s feedback_info_pack=1.387s effective_fps=434.34 +2026-09-27 05:19:18.606 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=956254 infer_calls=241286 feedback_calls=241285 infer_wait=2045.210s infer_obs_pack=4.175s env_step=120.498s feedback_total=27.202s feedback_obs_pack=6.729s feedback_info_pack=1.406s effective_fps=435.24 +2026-09-27 05:19:50.266 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=970036 infer_calls=244738 feedback_calls=244737 infer_wait=2074.503s infer_obs_pack=4.232s env_step=122.113s feedback_total=27.556s feedback_obs_pack=6.817s feedback_info_pack=1.423s effective_fps=435.31 +2026-09-27 05:20:20.272 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=984421 infer_calls=248338 feedback_calls=248337 infer_wait=2101.972s infer_obs_pack=4.294s env_step=123.841s feedback_total=27.938s feedback_obs_pack=6.911s feedback_info_pack=1.442s effective_fps=435.96 +2026-09-27 05:20:52.748 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=998675 infer_calls=251906 feedback_calls=251905 infer_wait=2131.958s infer_obs_pack=4.352s env_step=125.532s feedback_total=28.325s feedback_obs_pack=7.003s feedback_info_pack=1.460s effective_fps=436.07 +2026-09-27 05:21:22.752 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1013361 infer_calls=255580 feedback_calls=255579 infer_wait=2159.341s infer_obs_pack=4.414s env_step=127.319s feedback_total=28.726s feedback_obs_pack=7.099s feedback_info_pack=1.479s effective_fps=436.83 +2026-09-27 05:21:54.520 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1027325 infer_calls=259074 feedback_calls=259073 infer_wait=2188.624s infer_obs_pack=4.472s env_step=129.007s feedback_total=29.107s feedback_obs_pack=7.191s feedback_info_pack=1.497s effective_fps=436.93 +2026-09-27 05:22:24.520 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1042019 infer_calls=262752 feedback_calls=262752 infer_wait=2216.031s infer_obs_pack=4.532s env_step=130.771s feedback_total=29.502s feedback_obs_pack=7.287s feedback_info_pack=1.517s effective_fps=437.67 +2026-09-27 05:22:56.939 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1055960 infer_calls=266242 feedback_calls=266241 infer_wait=2245.953s infer_obs_pack=4.593s env_step=132.470s feedback_total=29.881s feedback_obs_pack=7.384s feedback_info_pack=1.536s effective_fps=437.63 +2026-09-27 05:23:26.943 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1070445 infer_calls=269871 feedback_calls=269870 infer_wait=2273.401s infer_obs_pack=4.654s env_step=134.214s feedback_total=30.265s feedback_obs_pack=7.478s feedback_info_pack=1.557s effective_fps=438.25 +2026-09-27 05:23:56.947 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1084203 infer_calls=273318 feedback_calls=273317 infer_wait=2300.898s infer_obs_pack=4.714s env_step=135.916s feedback_total=30.649s feedback_obs_pack=7.572s feedback_info_pack=1.576s effective_fps=438.56 +2026-09-27 05:24:27.009 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1096836 infer_calls=276482 feedback_calls=276481 infer_wait=2328.746s infer_obs_pack=4.766s env_step=137.425s feedback_total=30.986s feedback_obs_pack=7.655s feedback_info_pack=1.593s effective_fps=438.40 +2026-09-27 05:24:57.017 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1111175 infer_calls=280074 feedback_calls=280073 infer_wait=2356.215s infer_obs_pack=4.829s env_step=139.139s feedback_total=31.382s feedback_obs_pack=7.754s feedback_info_pack=1.615s effective_fps=438.93 +2026-09-27 05:25:27.024 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1121855 infer_calls=282749 feedback_calls=282748 infer_wait=2383.931s infer_obs_pack=4.884s env_step=140.684s feedback_total=31.745s feedback_obs_pack=7.840s feedback_info_pack=1.633s effective_fps=438.01 +2026-09-27 05:25:59.130 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1133642 infer_calls=285698 feedback_calls=285697 infer_wait=2413.586s infer_obs_pack=4.944s env_step=142.338s feedback_total=32.133s feedback_obs_pack=7.933s feedback_info_pack=1.652s effective_fps=437.19 +2026-09-27 05:26:29.756 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1145889 infer_calls=288770 feedback_calls=288769 infer_wait=2441.774s infer_obs_pack=5.002s env_step=143.976s feedback_total=32.525s feedback_obs_pack=8.024s feedback_info_pack=1.671s effective_fps=436.82 +2026-09-27 05:26:59.757 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1158092 infer_calls=291826 feedback_calls=291825 infer_wait=2469.202s infer_obs_pack=5.062s env_step=145.710s feedback_total=32.937s feedback_obs_pack=8.126s feedback_info_pack=1.691s effective_fps=436.54 +2026-09-27 05:27:29.760 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1167126 infer_calls=294089 feedback_calls=294088 infer_wait=2497.190s infer_obs_pack=5.110s env_step=147.045s feedback_total=33.283s feedback_obs_pack=8.202s feedback_info_pack=1.710s effective_fps=435.07 +2026-09-27 05:27:59.764 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1177632 infer_calls=296718 feedback_calls=296717 infer_wait=2524.826s infer_obs_pack=5.166s env_step=148.625s feedback_total=33.674s feedback_obs_pack=8.291s feedback_info_pack=1.734s effective_fps=434.18 +2026-09-27 05:28:29.918 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1186785 infer_calls=299010 feedback_calls=299009 infer_wait=2552.970s infer_obs_pack=5.213s env_step=149.961s feedback_total=34.010s feedback_obs_pack=8.374s feedback_info_pack=1.750s effective_fps=432.79 +2026-09-27 05:28:59.918 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1197658 infer_calls=301731 feedback_calls=301730 infer_wait=2580.672s infer_obs_pack=5.269s env_step=151.493s feedback_total=34.389s feedback_obs_pack=8.461s feedback_info_pack=1.767s effective_fps=432.08 +2026-09-27 05:29:30.310 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1207238 infer_calls=304130 feedback_calls=304129 infer_wait=2609.008s infer_obs_pack=5.319s env_step=152.866s feedback_total=34.722s feedback_obs_pack=8.541s feedback_info_pack=1.785s effective_fps=430.86 +2026-09-27 05:30:00.316 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1218228 infer_calls=306884 feedback_calls=306883 infer_wait=2636.605s infer_obs_pack=5.379s env_step=154.457s feedback_total=35.125s feedback_obs_pack=8.635s feedback_info_pack=1.803s effective_fps=430.23 +2026-09-27 05:30:06.747 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=1219497 infer_calls=307201 feedback_calls=307201 infer_wait=2639.437s infer_obs_pack=5.386s env_step=154.652s feedback_total=35.173s feedback_obs_pack=8.645s feedback_info_pack=1.805s effective_fps=430.21 +2026-09-27 05:30:06.747 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 6551/1228800 episodes. diff --git a/experiments/e31-dppo-longer/results/dppo-hopper/client-seed2.log b/experiments/e31-dppo-longer/results/dppo-hopper/client-seed2.log new file mode 100644 index 0000000..ebb7fd7 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper/client-seed2.log @@ -0,0 +1,104 @@ +2026-09-27 04:42:20.450 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-27 04:42:20.450 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-27 04:42:20.450 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-27 04:42:20.450 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-27 04:42:20.451 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-27 04:42:20.492 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260927-044220-f66395a5 output_dir=runs/mujoco-v1-nenv1-20260927-044220-f66395a5 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-27 04:42:20.532 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9802... +2026-09-27 04:42:20.534 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 3, 'action_horizon': 4} +2026-09-27 04:42:50.538 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14957 infer_calls=3972 feedback_calls=3971 infer_wait=27.567s infer_obs_pack=0.055s env_step=1.606s feedback_total=0.356s feedback_obs_pack=0.086s feedback_info_pack=0.018s effective_fps=505.57 +2026-09-27 04:43:20.543 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27943 infer_calls=7359 feedback_calls=7358 infer_wait=55.383s infer_obs_pack=0.104s env_step=3.057s feedback_total=0.694s feedback_obs_pack=0.165s feedback_info_pack=0.035s effective_fps=471.71 +2026-09-27 04:43:50.548 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=42345 infer_calls=11073 feedback_calls=11072 infer_wait=82.780s infer_obs_pack=0.161s env_step=4.810s feedback_total=1.090s feedback_obs_pack=0.257s feedback_info_pack=0.055s effective_fps=476.63 +2026-09-27 04:44:21.550 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=55050 infer_calls=14338 feedback_calls=14337 infer_wait=111.453s infer_obs_pack=0.234s env_step=6.354s feedback_total=1.454s feedback_obs_pack=0.343s feedback_info_pack=0.074s effective_fps=460.69 +2026-09-27 04:44:51.555 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=69014 infer_calls=17920 feedback_calls=17919 infer_wait=138.883s infer_obs_pack=0.292s env_step=8.092s feedback_total=1.845s feedback_obs_pack=0.438s feedback_info_pack=0.094s effective_fps=462.83 +2026-09-27 04:45:23.565 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=83000 infer_calls=21506 feedback_calls=21505 infer_wait=168.447s infer_obs_pack=0.346s env_step=9.729s feedback_total=2.228s feedback_obs_pack=0.530s feedback_info_pack=0.113s effective_fps=459.20 +2026-09-27 04:45:53.566 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=97914 infer_calls=25326 feedback_calls=25325 infer_wait=195.907s infer_obs_pack=0.403s env_step=11.431s feedback_total=2.623s feedback_obs_pack=0.622s feedback_info_pack=0.133s effective_fps=465.45 +2026-09-27 04:46:24.334 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=111027 infer_calls=28674 feedback_calls=28673 infer_wait=224.351s infer_obs_pack=0.456s env_step=12.978s feedback_total=2.998s feedback_obs_pack=0.709s feedback_info_pack=0.151s effective_fps=461.11 +2026-09-27 04:46:54.339 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=126274 infer_calls=32553 feedback_calls=32552 infer_wait=251.789s infer_obs_pack=0.515s env_step=14.708s feedback_total=3.388s feedback_obs_pack=0.801s feedback_info_pack=0.171s effective_fps=466.99 +2026-09-27 04:47:24.342 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139808 infer_calls=35997 feedback_calls=35996 infer_wait=279.644s infer_obs_pack=0.564s env_step=16.154s feedback_total=3.710s feedback_obs_pack=0.878s feedback_info_pack=0.188s effective_fps=465.91 +2026-09-27 04:47:56.984 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=155321 infer_calls=39938 feedback_calls=39937 infer_wait=309.881s infer_obs_pack=0.620s env_step=17.783s feedback_total=4.063s feedback_obs_pack=0.966s feedback_info_pack=0.206s effective_fps=467.35 +2026-09-27 04:48:26.987 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=170749 infer_calls=43851 feedback_calls=43850 infer_wait=337.452s infer_obs_pack=0.676s env_step=19.430s feedback_total=4.420s feedback_obs_pack=1.054s feedback_info_pack=0.225s effective_fps=471.71 +2026-09-27 04:48:56.987 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=184349 infer_calls=47307 feedback_calls=47307 infer_wait=365.319s infer_obs_pack=0.726s env_step=20.872s feedback_total=4.735s feedback_obs_pack=1.131s feedback_info_pack=0.242s effective_fps=470.70 +2026-09-27 04:49:29.585 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=199723 infer_calls=51202 feedback_calls=51201 infer_wait=395.475s infer_obs_pack=0.785s env_step=22.522s feedback_total=5.096s feedback_obs_pack=1.218s feedback_info_pack=0.260s effective_fps=471.18 +2026-09-27 04:49:59.588 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=214296 infer_calls=54897 feedback_calls=54896 infer_wait=423.125s infer_obs_pack=0.839s env_step=24.113s feedback_total=5.460s feedback_obs_pack=1.300s feedback_info_pack=0.278s effective_fps=472.50 +2026-09-27 04:50:30.845 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=228007 infer_calls=58369 feedback_calls=58368 infer_wait=452.115s infer_obs_pack=0.891s env_step=25.642s feedback_total=5.803s feedback_obs_pack=1.386s feedback_info_pack=0.296s effective_fps=470.65 +2026-09-27 04:51:00.849 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=243922 infer_calls=62399 feedback_calls=62398 infer_wait=479.691s infer_obs_pack=0.947s env_step=27.279s feedback_total=6.170s feedback_obs_pack=1.473s feedback_info_pack=0.314s effective_fps=474.48 +2026-09-27 04:51:30.855 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=256926 infer_calls=65693 feedback_calls=65692 infer_wait=507.571s infer_obs_pack=0.997s env_step=28.710s feedback_total=6.501s feedback_obs_pack=1.550s feedback_info_pack=0.330s effective_fps=472.48 +2026-09-27 04:52:00.855 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=271721 infer_calls=69437 feedback_calls=69436 infer_wait=535.111s infer_obs_pack=1.053s env_step=30.375s feedback_total=6.874s feedback_obs_pack=1.641s feedback_info_pack=0.349s effective_fps=473.87 +2026-09-27 04:52:30.858 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=282536 infer_calls=72167 feedback_calls=72166 infer_wait=562.938s infer_obs_pack=1.103s env_step=31.858s feedback_total=7.206s feedback_obs_pack=1.720s feedback_info_pack=0.366s effective_fps=468.47 +2026-09-27 04:53:00.859 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=296054 infer_calls=75587 feedback_calls=75586 infer_wait=590.649s infer_obs_pack=1.155s env_step=33.424s feedback_total=7.539s feedback_obs_pack=1.803s feedback_info_pack=0.384s effective_fps=467.87 +2026-09-27 04:53:31.144 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=308973 infer_calls=78850 feedback_calls=78849 infer_wait=618.762s infer_obs_pack=1.208s env_step=34.894s feedback_total=7.869s feedback_obs_pack=1.882s feedback_info_pack=0.401s effective_fps=466.21 +2026-09-27 04:54:01.145 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=321132 infer_calls=81923 feedback_calls=81922 infer_wait=646.386s infer_obs_pack=1.263s env_step=36.499s feedback_total=8.246s feedback_obs_pack=1.970s feedback_info_pack=0.420s effective_fps=463.80 +2026-09-27 04:54:31.917 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=333279 infer_calls=84994 feedback_calls=84993 infer_wait=674.700s infer_obs_pack=1.320s env_step=38.166s feedback_total=8.618s feedback_obs_pack=2.060s feedback_info_pack=0.440s effective_fps=461.09 +2026-09-27 04:55:02.471 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=345482 infer_calls=88066 feedback_calls=88065 infer_wait=702.807s infer_obs_pack=1.374s env_step=39.831s feedback_total=9.003s feedback_obs_pack=2.151s feedback_info_pack=0.466s effective_fps=458.80 +2026-09-27 04:55:33.172 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=357664 infer_calls=91138 feedback_calls=91137 infer_wait=731.042s infer_obs_pack=1.428s env_step=41.494s feedback_total=9.398s feedback_obs_pack=2.242s feedback_info_pack=0.487s effective_fps=456.58 +2026-09-27 04:56:03.555 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=369860 infer_calls=94210 feedback_calls=94209 infer_wait=758.985s infer_obs_pack=1.488s env_step=43.133s feedback_total=9.790s feedback_obs_pack=2.338s feedback_info_pack=0.510s effective_fps=454.71 +2026-09-27 04:56:35.409 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=382046 infer_calls=97282 feedback_calls=97281 infer_wait=788.337s infer_obs_pack=1.543s env_step=44.817s feedback_total=10.194s feedback_obs_pack=2.438s feedback_info_pack=0.531s effective_fps=452.18 +2026-09-27 04:57:07.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=394240 infer_calls=100354 feedback_calls=100353 infer_wait=817.760s infer_obs_pack=1.606s env_step=46.576s feedback_total=10.590s feedback_obs_pack=2.537s feedback_info_pack=0.554s effective_fps=449.77 +2026-09-27 04:57:39.290 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=406444 infer_calls=103426 feedback_calls=103425 infer_wait=847.201s infer_obs_pack=1.670s env_step=48.228s feedback_total=10.967s feedback_obs_pack=2.625s feedback_info_pack=0.575s effective_fps=447.59 +2026-09-27 04:58:11.203 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=418639 infer_calls=106498 feedback_calls=106497 infer_wait=876.564s infer_obs_pack=1.734s env_step=49.961s feedback_total=11.362s feedback_obs_pack=2.725s feedback_info_pack=0.596s effective_fps=445.54 +2026-09-27 04:58:43.353 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=430852 infer_calls=109570 feedback_calls=109569 infer_wait=906.096s infer_obs_pack=1.794s env_step=51.738s feedback_total=11.774s feedback_obs_pack=2.821s feedback_info_pack=0.622s effective_fps=443.54 +2026-09-27 04:59:13.356 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=442623 infer_calls=112530 feedback_calls=112529 infer_wait=933.579s infer_obs_pack=1.853s env_step=53.420s feedback_total=12.186s feedback_obs_pack=2.918s feedback_info_pack=0.642s effective_fps=442.16 +2026-09-27 04:59:43.363 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=452487 infer_calls=115018 feedback_calls=115017 infer_wait=961.408s infer_obs_pack=1.909s env_step=54.881s feedback_total=12.547s feedback_obs_pack=3.006s feedback_info_pack=0.660s effective_fps=438.99 +2026-09-27 05:00:16.290 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=463396 infer_calls=117762 feedback_calls=117761 infer_wait=991.939s infer_obs_pack=1.962s env_step=56.496s feedback_total=12.935s feedback_obs_pack=3.100s feedback_info_pack=0.680s effective_fps=435.80 +2026-09-27 05:00:46.292 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=474730 infer_calls=120614 feedback_calls=120613 infer_wait=1019.477s infer_obs_pack=2.020s env_step=58.166s feedback_total=13.331s feedback_obs_pack=3.201s feedback_info_pack=0.699s effective_fps=434.34 +2026-09-27 05:01:16.298 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=487077 infer_calls=123721 feedback_calls=123720 infer_wait=1047.059s infer_obs_pack=2.074s env_step=59.812s feedback_total=13.716s feedback_obs_pack=3.293s feedback_info_pack=0.721s effective_fps=433.86 +2026-09-27 05:01:46.299 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=499575 infer_calls=126862 feedback_calls=126861 infer_wait=1074.606s infer_obs_pack=2.130s env_step=61.485s feedback_total=14.097s feedback_obs_pack=3.382s feedback_info_pack=0.742s effective_fps=433.54 +2026-09-27 05:02:18.068 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=512264 infer_calls=130050 feedback_calls=130049 infer_wait=1103.959s infer_obs_pack=2.189s env_step=63.105s feedback_total=14.481s feedback_obs_pack=3.471s feedback_info_pack=0.762s effective_fps=432.75 +2026-09-27 05:02:48.069 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=525383 infer_calls=133348 feedback_calls=133347 infer_wait=1131.419s infer_obs_pack=2.249s env_step=64.831s feedback_total=14.883s feedback_obs_pack=3.567s feedback_info_pack=0.783s effective_fps=432.99 +2026-09-27 05:03:18.071 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=538423 infer_calls=136623 feedback_calls=136622 infer_wait=1158.909s infer_obs_pack=2.306s env_step=66.551s feedback_total=15.264s feedback_obs_pack=3.660s feedback_info_pack=0.803s effective_fps=433.15 +2026-09-27 05:03:48.074 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=551180 infer_calls=139830 feedback_calls=139829 infer_wait=1186.437s infer_obs_pack=2.363s env_step=68.242s feedback_total=15.646s feedback_obs_pack=3.756s feedback_info_pack=0.824s effective_fps=433.08 +2026-09-27 05:04:18.075 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=563628 infer_calls=142956 feedback_calls=142955 infer_wait=1213.978s infer_obs_pack=2.422s env_step=69.924s feedback_total=16.018s feedback_obs_pack=3.850s feedback_info_pack=0.844s effective_fps=432.78 +2026-09-27 05:04:48.076 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=576331 infer_calls=146144 feedback_calls=146143 infer_wait=1241.448s infer_obs_pack=2.482s env_step=71.659s feedback_total=16.398s feedback_obs_pack=3.949s feedback_info_pack=0.864s effective_fps=432.69 +2026-09-27 05:05:18.080 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=588831 infer_calls=149279 feedback_calls=149278 infer_wait=1269.026s infer_obs_pack=2.540s env_step=73.314s feedback_total=16.767s feedback_obs_pack=4.043s feedback_info_pack=0.882s effective_fps=432.44 +2026-09-27 05:05:48.083 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=601738 infer_calls=152518 feedback_calls=152517 infer_wait=1296.561s infer_obs_pack=2.597s env_step=74.995s feedback_total=17.155s feedback_obs_pack=4.138s feedback_info_pack=0.901s effective_fps=432.50 +2026-09-27 05:06:19.653 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=614223 infer_calls=155650 feedback_calls=155649 infer_wait=1325.784s infer_obs_pack=2.654s env_step=76.591s feedback_total=17.520s feedback_obs_pack=4.225s feedback_info_pack=0.918s effective_fps=431.78 +2026-09-27 05:06:49.654 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=626800 infer_calls=158802 feedback_calls=158801 infer_wait=1353.377s infer_obs_pack=2.713s env_step=78.242s feedback_total=17.881s feedback_obs_pack=4.317s feedback_info_pack=0.938s effective_fps=431.62 +2026-09-27 05:07:19.658 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=639904 infer_calls=162088 feedback_calls=162087 infer_wait=1380.883s infer_obs_pack=2.771s env_step=79.950s feedback_total=18.262s feedback_obs_pack=4.412s feedback_info_pack=0.957s effective_fps=431.82 +2026-09-27 05:07:49.659 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=653138 infer_calls=165408 feedback_calls=165407 infer_wait=1408.375s infer_obs_pack=2.836s env_step=81.651s feedback_total=18.652s feedback_obs_pack=4.510s feedback_info_pack=0.976s effective_fps=432.11 +2026-09-27 05:08:19.668 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=666708 infer_calls=168813 feedback_calls=168812 infer_wait=1435.853s infer_obs_pack=2.895s env_step=83.369s feedback_total=19.045s feedback_obs_pack=4.606s feedback_info_pack=0.994s effective_fps=432.60 +2026-09-27 05:08:52.167 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=679561 infer_calls=172034 feedback_calls=172033 infer_wait=1465.910s infer_obs_pack=2.950s env_step=85.035s feedback_total=19.419s feedback_obs_pack=4.698s feedback_info_pack=1.013s effective_fps=431.93 +2026-09-27 05:09:22.172 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=692939 infer_calls=175389 feedback_calls=175388 infer_wait=1493.380s infer_obs_pack=3.009s env_step=86.765s feedback_total=19.808s feedback_obs_pack=4.794s feedback_info_pack=1.032s effective_fps=432.29 +2026-09-27 05:09:52.172 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=705717 infer_calls=178594 feedback_calls=178593 infer_wait=1520.932s infer_obs_pack=3.064s env_step=88.439s feedback_total=20.182s feedback_obs_pack=4.882s feedback_info_pack=1.051s effective_fps=432.26 +2026-09-27 05:10:22.179 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=718384 infer_calls=181769 feedback_calls=181768 infer_wait=1548.512s infer_obs_pack=3.119s env_step=90.091s feedback_total=20.553s feedback_obs_pack=4.971s feedback_info_pack=1.069s effective_fps=432.17 +2026-09-27 05:10:52.185 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=731151 infer_calls=184969 feedback_calls=184968 infer_wait=1576.068s infer_obs_pack=3.176s env_step=91.764s feedback_total=20.923s feedback_obs_pack=5.060s feedback_info_pack=1.087s effective_fps=432.14 +2026-09-27 05:11:22.185 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=744323 infer_calls=188270 feedback_calls=188270 infer_wait=1603.591s infer_obs_pack=3.231s env_step=93.461s feedback_total=21.298s feedback_obs_pack=5.152s feedback_info_pack=1.105s effective_fps=432.35 +2026-09-27 05:11:54.101 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=757180 infer_calls=191490 feedback_calls=191489 infer_wait=1633.099s infer_obs_pack=3.287s env_step=95.098s feedback_total=21.675s feedback_obs_pack=5.242s feedback_info_pack=1.123s effective_fps=431.89 +2026-09-27 05:12:24.106 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=770308 infer_calls=194777 feedback_calls=194776 infer_wait=1660.676s infer_obs_pack=3.343s env_step=96.745s feedback_total=22.052s feedback_obs_pack=5.333s feedback_info_pack=1.143s effective_fps=432.07 +2026-09-27 05:12:54.112 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=783403 infer_calls=198058 feedback_calls=198057 infer_wait=1688.162s infer_obs_pack=3.400s env_step=98.468s feedback_total=22.439s feedback_obs_pack=5.431s feedback_info_pack=1.161s effective_fps=432.23 +2026-09-27 05:13:24.118 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=796363 infer_calls=201303 feedback_calls=201302 infer_wait=1715.662s infer_obs_pack=3.461s env_step=100.179s feedback_total=22.822s feedback_obs_pack=5.522s feedback_info_pack=1.180s effective_fps=432.31 +2026-09-27 05:13:54.122 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=809378 infer_calls=204562 feedback_calls=204561 infer_wait=1743.184s infer_obs_pack=3.517s env_step=101.875s feedback_total=23.208s feedback_obs_pack=5.617s feedback_info_pack=1.201s effective_fps=432.41 +2026-09-27 05:14:24.124 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=822065 infer_calls=207740 feedback_calls=207739 infer_wait=1770.717s infer_obs_pack=3.576s env_step=103.559s feedback_total=23.585s feedback_obs_pack=5.711s feedback_info_pack=1.219s effective_fps=432.34 +2026-09-27 05:14:54.129 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=834807 infer_calls=210928 feedback_calls=210927 infer_wait=1798.219s infer_obs_pack=3.640s env_step=105.262s feedback_total=23.973s feedback_obs_pack=5.803s feedback_info_pack=1.236s effective_fps=432.30 +2026-09-27 05:15:25.477 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=847148 infer_calls=214018 feedback_calls=214017 infer_wait=1827.256s infer_obs_pack=3.693s env_step=106.840s feedback_total=24.332s feedback_obs_pack=5.888s feedback_info_pack=1.253s effective_fps=431.75 +2026-09-27 05:15:55.860 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=859421 infer_calls=217090 feedback_calls=217089 infer_wait=1855.253s infer_obs_pack=3.748s env_step=108.466s feedback_total=24.698s feedback_obs_pack=5.980s feedback_info_pack=1.272s effective_fps=431.40 +2026-09-27 05:16:25.862 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=872604 infer_calls=220396 feedback_calls=220395 infer_wait=1882.756s infer_obs_pack=3.806s env_step=110.157s feedback_total=25.091s feedback_obs_pack=6.074s feedback_info_pack=1.291s effective_fps=431.60 +2026-09-27 05:16:55.865 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=886438 infer_calls=223865 feedback_calls=223864 infer_wait=1910.198s infer_obs_pack=3.865s env_step=111.887s feedback_total=25.505s feedback_obs_pack=6.171s feedback_info_pack=1.311s effective_fps=432.10 +2026-09-27 05:17:25.869 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=899660 infer_calls=227179 feedback_calls=227178 infer_wait=1937.727s infer_obs_pack=3.921s env_step=113.581s feedback_total=25.879s feedback_obs_pack=6.263s feedback_info_pack=1.330s effective_fps=432.30 +2026-09-27 05:17:56.499 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=912540 infer_calls=230402 feedback_calls=230401 infer_wait=1966.038s infer_obs_pack=3.974s env_step=115.170s feedback_total=26.231s feedback_obs_pack=6.348s feedback_info_pack=1.346s effective_fps=432.19 +2026-09-27 05:18:26.499 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=927074 infer_calls=234040 feedback_calls=234040 infer_wait=1993.439s infer_obs_pack=4.035s env_step=116.946s feedback_total=26.626s feedback_obs_pack=6.441s feedback_info_pack=1.366s effective_fps=433.00 +2026-09-27 05:18:58.606 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=941176 infer_calls=237570 feedback_calls=237569 infer_wait=2023.002s infer_obs_pack=4.095s env_step=118.681s feedback_total=27.020s feedback_obs_pack=6.534s feedback_info_pack=1.384s effective_fps=433.16 +2026-09-27 05:19:28.610 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=955681 infer_calls=241200 feedback_calls=241199 infer_wait=2050.416s infer_obs_pack=4.154s env_step=120.462s feedback_total=27.401s feedback_obs_pack=6.626s feedback_info_pack=1.403s effective_fps=433.92 +2026-09-27 05:20:01.043 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=969817 infer_calls=244738 feedback_calls=244737 infer_wait=2080.318s infer_obs_pack=4.214s env_step=122.206s feedback_total=27.775s feedback_obs_pack=6.718s feedback_info_pack=1.421s effective_fps=434.02 +2026-09-27 05:20:31.046 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=984203 infer_calls=248340 feedback_calls=248339 infer_wait=2107.717s infer_obs_pack=4.273s env_step=123.993s feedback_total=28.170s feedback_obs_pack=6.814s feedback_info_pack=1.443s effective_fps=434.69 +2026-09-27 05:21:03.447 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=998443 infer_calls=251906 feedback_calls=251905 infer_wait=2137.620s infer_obs_pack=4.335s env_step=125.705s feedback_total=28.545s feedback_obs_pack=6.903s feedback_info_pack=1.460s effective_fps=434.82 +2026-09-27 05:21:33.452 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1013168 infer_calls=255598 feedback_calls=255597 infer_wait=2165.031s infer_obs_pack=4.394s env_step=127.478s feedback_total=28.941s feedback_obs_pack=6.995s feedback_info_pack=1.479s effective_fps=435.61 +2026-09-27 05:22:05.153 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1027044 infer_calls=259074 feedback_calls=259073 infer_wait=2194.280s infer_obs_pack=4.449s env_step=129.157s feedback_total=29.310s feedback_obs_pack=7.084s feedback_info_pack=1.497s effective_fps=435.71 +2026-09-27 05:22:35.158 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1041237 infer_calls=262632 feedback_calls=262631 infer_wait=2221.691s infer_obs_pack=4.509s env_step=130.941s feedback_total=29.693s feedback_obs_pack=7.178s feedback_info_pack=1.517s effective_fps=436.24 +2026-09-27 05:23:05.158 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1055616 infer_calls=266236 feedback_calls=266236 infer_wait=2249.051s infer_obs_pack=4.569s env_step=132.762s feedback_total=30.083s feedback_obs_pack=7.274s feedback_info_pack=1.536s effective_fps=436.84 +2026-09-27 05:23:35.158 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1068086 infer_calls=269366 feedback_calls=269366 infer_wait=2276.776s infer_obs_pack=4.623s env_step=134.315s feedback_total=30.431s feedback_obs_pack=7.355s feedback_info_pack=1.553s effective_fps=436.64 +2026-09-27 05:24:05.160 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1081638 infer_calls=272762 feedback_calls=272761 infer_wait=2304.226s infer_obs_pack=4.683s env_step=136.065s feedback_total=30.817s feedback_obs_pack=7.447s feedback_info_pack=1.573s effective_fps=436.89 +2026-09-27 05:24:37.615 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1096509 infer_calls=276482 feedback_calls=276481 infer_wait=2334.058s infer_obs_pack=4.741s env_step=137.869s feedback_total=31.208s feedback_obs_pack=7.538s feedback_info_pack=1.592s effective_fps=437.23 +2026-09-27 05:25:07.622 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1109544 infer_calls=279743 feedback_calls=279742 infer_wait=2361.472s infer_obs_pack=4.801s env_step=139.648s feedback_total=31.600s feedback_obs_pack=7.631s feedback_info_pack=1.611s effective_fps=437.26 +2026-09-27 05:25:39.402 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1121068 infer_calls=282626 feedback_calls=282625 infer_wait=2390.699s infer_obs_pack=4.858s env_step=141.387s feedback_total=32.004s feedback_obs_pack=7.730s feedback_info_pack=1.631s effective_fps=436.39 +2026-09-27 05:26:11.187 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1133341 infer_calls=285698 feedback_calls=285697 infer_wait=2419.924s infer_obs_pack=4.916s env_step=143.135s feedback_total=32.398s feedback_obs_pack=7.827s feedback_info_pack=1.649s effective_fps=435.84 +2026-09-27 05:26:42.913 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1145606 infer_calls=288770 feedback_calls=288769 infer_wait=2449.076s infer_obs_pack=4.975s env_step=144.885s feedback_total=32.803s feedback_obs_pack=7.926s feedback_info_pack=1.668s effective_fps=435.30 +2026-09-27 05:27:12.913 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1156819 infer_calls=291574 feedback_calls=291573 infer_wait=2476.555s infer_obs_pack=5.032s env_step=146.610s feedback_total=33.203s feedback_obs_pack=8.022s feedback_info_pack=1.687s effective_fps=434.67 +2026-09-27 05:27:43.713 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1166081 infer_calls=293890 feedback_calls=293889 infer_wait=2505.249s infer_obs_pack=5.079s env_step=148.025s feedback_total=33.557s feedback_obs_pack=8.100s feedback_info_pack=1.703s effective_fps=433.18 +2026-09-27 05:28:13.713 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1177088 infer_calls=296643 feedback_calls=296643 infer_wait=2532.801s infer_obs_pack=5.138s env_step=149.671s feedback_total=33.961s feedback_obs_pack=8.193s feedback_info_pack=1.723s effective_fps=432.50 +2026-09-27 05:28:44.293 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1186543 infer_calls=299010 feedback_calls=299009 infer_wait=2561.253s infer_obs_pack=5.191s env_step=151.105s feedback_total=34.302s feedback_obs_pack=8.273s feedback_info_pack=1.740s effective_fps=431.18 +2026-09-27 05:29:14.300 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1197536 infer_calls=301764 feedback_calls=301763 infer_wait=2588.788s infer_obs_pack=5.247s env_step=152.778s feedback_total=34.693s feedback_obs_pack=8.362s feedback_info_pack=1.759s effective_fps=430.54 +2026-09-27 05:29:45.143 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1206980 infer_calls=304130 feedback_calls=304129 infer_wait=2617.452s infer_obs_pack=5.301s env_step=154.250s feedback_total=35.047s feedback_obs_pack=8.444s feedback_info_pack=1.775s effective_fps=429.22 +2026-09-27 05:30:15.149 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1217481 infer_calls=306759 feedback_calls=306758 infer_wait=2645.078s infer_obs_pack=5.360s env_step=155.858s feedback_total=35.425s feedback_obs_pack=8.533s feedback_info_pack=1.792s effective_fps=428.43 +2026-09-27 05:30:22.291 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=1219252 infer_calls=307201 feedback_calls=307201 infer_wait=2648.615s infer_obs_pack=5.369s env_step=156.124s feedback_total=35.489s feedback_obs_pack=8.548s feedback_info_pack=1.795s effective_fps=428.47 +2026-09-27 05:30:22.292 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 6620/1228800 episodes. diff --git a/experiments/e31-dppo-longer/results/dppo-hopper/server-seed0.log b/experiments/e31-dppo-longer/results/dppo-hopper/server-seed0.log new file mode 100644 index 0000000..7a4f626 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper/server-seed0.log @@ -0,0 +1,36 @@ +04:42:10|INFO|plugrl_server version: 0.1.0 +04:42:10|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=307200, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, logprob_clamp_min=-5.0, logprob_clamp_max=2.0, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=300, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +04:42:10|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +04:42:10|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +04:42:10|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0 +04:42:10|INFO|Number of network parameters: 553020 +04:42:10|INFO|Policy created... +04:42:10|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +04:42:10|INFO|Algorithm created: + +04:42:10|INFO|Agent Server is listening on 0.0.0.0:9800 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +04:45:04|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/20481 +04:47:56|INFO|Checkpoint saved at step 40960 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/40960 +04:50:47|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/61441 +04:53:50|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/81921 +04:57:18|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/102401 +05:01:02|INFO|Checkpoint saved at step 122881 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/122881 +05:04:16|INFO|Checkpoint saved at step 143361 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/143361 +05:07:28|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/163841 +05:10:40|INFO|Checkpoint saved at step 184321 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/184321 +05:13:52|INFO|Checkpoint saved at step 204801 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/204801 +05:17:05|INFO|Checkpoint saved at step 225281 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/225281 +05:20:05|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/245761 +05:23:03|INFO|Checkpoint saved at step 266241 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/266241 +05:26:17|INFO|Checkpoint saved at step 286721 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/286721 +05:30:18|INFO|Checkpoint saved at step 307201 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed0/307201 +05:30:18|INFO|Stopping server as the algorithm signaled to stop. +05:30:18|INFO|Shutdown started: aborting pending infer requests. +05:30:18|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +05:30:18|INFO|Shutdown closing 1 websocket connection(s). +05:30:18|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 49052). +05:30:18|INFO|WebSocket server closed. +05:30:18|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e31-dppo-longer/results/dppo-hopper/server-seed1.log b/experiments/e31-dppo-longer/results/dppo-hopper/server-seed1.log new file mode 100644 index 0000000..ffbee75 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper/server-seed1.log @@ -0,0 +1,36 @@ +04:42:15|INFO|plugrl_server version: 0.1.0 +04:42:15|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=307200, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, logprob_clamp_min=-5.0, logprob_clamp_max=2.0, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=300, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +04:42:15|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +04:42:15|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +04:42:15|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1 +04:42:15|INFO|Number of network parameters: 553020 +04:42:15|INFO|Policy created... +04:42:15|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +04:42:15|INFO|Algorithm created: + +04:42:15|INFO|Agent Server is listening on 0.0.0.0:9801 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +04:45:09|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/20481 +04:48:00|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/40961 +04:50:52|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/61441 +04:53:56|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/81921 +04:57:18|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/102401 +05:01:02|INFO|Checkpoint saved at step 122881 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/122881 +05:04:18|INFO|Checkpoint saved at step 143361 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/143361 +05:07:30|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/163841 +05:10:41|INFO|Checkpoint saved at step 184321 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/184321 +05:13:51|INFO|Checkpoint saved at step 204801 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/204801 +05:17:01|INFO|Checkpoint saved at step 225281 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/225281 +05:19:59|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/245761 +05:22:56|INFO|Checkpoint saved at step 266241 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/266241 +05:26:09|INFO|Checkpoint saved at step 286720 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/286720 +05:30:06|INFO|Checkpoint saved at step 307201 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed1/307201 +05:30:06|INFO|Stopping server as the algorithm signaled to stop. +05:30:06|INFO|Shutdown started: aborting pending infer requests. +05:30:06|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +05:30:06|INFO|Shutdown closing 1 websocket connection(s). +05:30:06|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 46724). +05:30:06|INFO|WebSocket server closed. +05:30:06|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e31-dppo-longer/results/dppo-hopper/server-seed2.log b/experiments/e31-dppo-longer/results/dppo-hopper/server-seed2.log new file mode 100644 index 0000000..083ec41 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-hopper/server-seed2.log @@ -0,0 +1,36 @@ +04:42:20|INFO|plugrl_server version: 0.1.0 +04:42:20|INFO|Algorithm: dppo, Config: DPPOAlgoConfigHopper(global_steps=307200, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, logprob_clamp_min=-5.0, logprob_clamp_max=2.0, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=300, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +04:42:20|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigHopper(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='hopper-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +04:42:20|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +04:42:20|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2 +04:42:20|INFO|Number of network parameters: 553020 +04:42:20|INFO|Policy created... +04:42:20|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 3) value_shape=(1024,) +04:42:20|INFO|Algorithm created: + +04:42:20|INFO|Agent Server is listening on 0.0.0.0:9802 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +04:45:15|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/20481 +04:48:05|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/40961 +04:50:55|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/61441 +04:54:01|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/81921 +04:57:29|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/102401 +05:01:10|INFO|Checkpoint saved at step 122881 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/122881 +05:04:23|INFO|Checkpoint saved at step 143361 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/143361 +05:07:36|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/163841 +05:10:48|INFO|Checkpoint saved at step 184321 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/184321 +05:13:58|INFO|Checkpoint saved at step 204801 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/204801 +05:17:11|INFO|Checkpoint saved at step 225281 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/225281 +05:20:10|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/245761 +05:23:08|INFO|Checkpoint saved at step 266241 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/266241 +05:26:21|INFO|Checkpoint saved at step 286721 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/286721 +05:30:22|INFO|Checkpoint saved at step 307201 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-hopper/dppo/dppo-policy/dppo-hopper-seed2/307201 +05:30:22|INFO|Stopping server as the algorithm signaled to stop. +05:30:22|INFO|Shutdown started: aborting pending infer requests. +05:30:22|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +05:30:22|INFO|Shutdown closing 1 websocket connection(s). +05:30:22|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 47732). +05:30:22|INFO|WebSocket server closed. +05:30:22|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e31-dppo-longer/results/dppo-walker.out b/experiments/e31-dppo-longer/results/dppo-walker.out new file mode 100644 index 0000000..1dbfa69 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker.out @@ -0,0 +1,10 @@ +cell: dppo-walker = dppo-policy/walker x dppo/walker x Walker2d-v5 +iters: 300 x 1024 batch: 512 replan: 4 seeds: 0 1 2 +out: /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker +start 2026-09-27 04:42:28 +seed 1 finished rc=0 at 05:31:54 +seed 2 finished rc=0 at 05:31:58 +seed 0 finished rc=0 at 05:32:02 +end 2026-09-27 05:32:02 +failed seeds: 0 +CELL_DONE dppo-walker diff --git a/experiments/e31-dppo-longer/results/dppo-walker/client-seed0.log b/experiments/e31-dppo-longer/results/dppo-walker/client-seed0.log new file mode 100644 index 0000000..4624d96 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker/client-seed0.log @@ -0,0 +1,107 @@ +2026-09-27 04:42:31.448 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-27 04:42:31.448 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-27 04:42:31.448 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-27 04:42:31.448 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-27 04:42:31.449 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-27 04:42:31.492 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260927-044231-090b2886 output_dir=runs/mujoco-v1-nenv1-20260927-044231-090b2886 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-27 04:42:31.528 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9810... +2026-09-27 04:42:31.530 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-27 04:43:01.534 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14216 infer_calls=3804 feedback_calls=3803 infer_wait=27.458s infer_obs_pack=0.063s env_step=1.686s feedback_total=0.366s feedback_obs_pack=0.091s feedback_info_pack=0.019s effective_fps=480.70 +2026-09-27 04:43:32.366 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27256 infer_calls=7170 feedback_calls=7169 infer_wait=55.938s infer_obs_pack=0.115s env_step=3.284s feedback_total=0.711s feedback_obs_pack=0.173s feedback_info_pack=0.036s effective_fps=453.91 +2026-09-27 04:44:02.370 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=41176 infer_calls=10693 feedback_calls=10692 infer_wait=83.247s infer_obs_pack=0.173s env_step=5.144s feedback_total=1.113s feedback_obs_pack=0.269s feedback_info_pack=0.054s effective_fps=459.16 +2026-09-27 04:44:32.372 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=55061 infer_calls=14195 feedback_calls=14194 infer_wait=110.554s infer_obs_pack=0.247s env_step=6.998s feedback_total=1.517s feedback_obs_pack=0.363s feedback_info_pack=0.072s effective_fps=461.47 +2026-09-27 04:45:03.438 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67851 infer_calls=17410 feedback_calls=17409 infer_wait=139.108s infer_obs_pack=0.300s env_step=8.756s feedback_total=1.877s feedback_obs_pack=0.450s feedback_info_pack=0.089s effective_fps=452.21 +2026-09-27 04:45:33.441 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=82152 infer_calls=21012 feedback_calls=21011 infer_wait=166.424s infer_obs_pack=0.356s env_step=10.641s feedback_total=2.258s feedback_obs_pack=0.542s feedback_info_pack=0.107s effective_fps=457.21 +2026-09-27 04:46:06.161 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=96327 infer_calls=24578 feedback_calls=24577 infer_wait=196.535s infer_obs_pack=0.412s env_step=12.461s feedback_total=2.632s feedback_obs_pack=0.633s feedback_info_pack=0.126s effective_fps=454.29 +2026-09-27 04:46:36.161 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110912 infer_calls=28251 feedback_calls=28250 infer_wait=223.831s infer_obs_pack=0.475s env_step=14.353s feedback_total=3.008s feedback_obs_pack=0.727s feedback_info_pack=0.144s effective_fps=458.95 +2026-09-27 04:47:07.619 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=124809 infer_calls=31746 feedback_calls=31745 infer_wait=252.879s infer_obs_pack=0.527s env_step=16.044s feedback_total=3.333s feedback_obs_pack=0.809s feedback_info_pack=0.160s effective_fps=457.54 +2026-09-27 04:47:37.622 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=140314 infer_calls=35646 feedback_calls=35645 infer_wait=280.273s infer_obs_pack=0.582s env_step=17.884s feedback_total=3.682s feedback_obs_pack=0.899s feedback_info_pack=0.178s effective_fps=463.97 +2026-09-27 04:48:07.623 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=153716 infer_calls=39020 feedback_calls=39019 infer_wait=308.028s infer_obs_pack=0.632s env_step=19.461s feedback_total=3.985s feedback_obs_pack=0.975s feedback_info_pack=0.193s effective_fps=462.85 +2026-09-27 04:48:37.627 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=168778 infer_calls=42806 feedback_calls=42805 infer_wait=335.457s infer_obs_pack=0.687s env_step=21.277s feedback_total=4.328s feedback_obs_pack=1.062s feedback_info_pack=0.210s effective_fps=466.56 +2026-09-27 04:49:07.630 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=181837 infer_calls=46090 feedback_calls=46089 infer_wait=363.213s infer_obs_pack=0.735s env_step=22.865s feedback_total=4.627s feedback_obs_pack=1.139s feedback_info_pack=0.225s effective_fps=464.53 +2026-09-27 04:49:37.632 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=197305 infer_calls=49976 feedback_calls=49975 infer_wait=390.579s infer_obs_pack=0.796s env_step=24.734s feedback_total=4.970s feedback_obs_pack=1.231s feedback_info_pack=0.243s effective_fps=468.57 +2026-09-27 04:50:09.260 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=210344 infer_calls=53250 feedback_calls=53249 infer_wait=419.984s infer_obs_pack=0.843s env_step=26.337s feedback_total=5.251s feedback_obs_pack=1.305s feedback_info_pack=0.257s effective_fps=464.94 +2026-09-27 04:50:39.261 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=224764 infer_calls=56868 feedback_calls=56867 infer_wait=447.397s infer_obs_pack=0.900s env_step=28.153s feedback_total=5.608s feedback_obs_pack=1.392s feedback_info_pack=0.276s effective_fps=466.26 +2026-09-27 04:51:10.456 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=238935 infer_calls=60418 feedback_calls=60417 infer_wait=476.190s infer_obs_pack=0.952s env_step=29.850s feedback_total=5.929s feedback_obs_pack=1.474s feedback_info_pack=0.292s effective_fps=465.83 +2026-09-27 04:51:40.460 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=253125 infer_calls=63974 feedback_calls=63973 infer_wait=503.589s infer_obs_pack=1.010s env_step=31.677s feedback_total=6.292s feedback_obs_pack=1.562s feedback_info_pack=0.310s effective_fps=466.53 +2026-09-27 04:52:10.462 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266857 infer_calls=67421 feedback_calls=67420 infer_wait=531.089s infer_obs_pack=1.064s env_step=33.435s feedback_total=6.640s feedback_obs_pack=1.650s feedback_info_pack=0.327s effective_fps=466.35 +2026-09-27 04:52:40.468 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=277743 infer_calls=70158 feedback_calls=70157 infer_wait=558.748s infer_obs_pack=1.116s env_step=35.080s feedback_total=6.975s feedback_obs_pack=1.730s feedback_info_pack=0.343s effective_fps=461.43 +2026-09-27 04:53:12.900 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=291948 infer_calls=73730 feedback_calls=73729 infer_wait=588.713s infer_obs_pack=1.170s env_step=36.806s feedback_total=7.319s feedback_obs_pack=1.821s feedback_info_pack=0.363s effective_fps=460.48 +2026-09-27 04:53:42.903 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=304875 infer_calls=76970 feedback_calls=76969 infer_wait=616.149s infer_obs_pack=1.227s env_step=38.610s feedback_total=7.674s feedback_obs_pack=1.913s feedback_info_pack=0.380s effective_fps=459.38 +2026-09-27 04:54:13.266 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=316463 infer_calls=79874 feedback_calls=79873 infer_wait=643.969s infer_obs_pack=1.282s env_step=40.406s feedback_total=8.039s feedback_obs_pack=2.006s feedback_info_pack=0.397s effective_fps=456.20 +2026-09-27 04:54:45.782 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=328718 infer_calls=82946 feedback_calls=82945 infer_wait=673.748s infer_obs_pack=1.342s env_step=42.332s feedback_total=8.411s feedback_obs_pack=2.102s feedback_info_pack=0.416s effective_fps=452.88 +2026-09-27 04:55:18.029 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=340970 infer_calls=86018 feedback_calls=86017 infer_wait=703.305s infer_obs_pack=1.400s env_step=44.235s feedback_total=8.791s feedback_obs_pack=2.196s feedback_info_pack=0.434s effective_fps=449.99 +2026-09-27 04:55:48.509 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=353221 infer_calls=89090 feedback_calls=89089 infer_wait=731.139s infer_obs_pack=1.459s env_step=46.081s feedback_total=9.171s feedback_obs_pack=2.295s feedback_info_pack=0.452s effective_fps=448.34 +2026-09-27 04:56:20.646 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=365455 infer_calls=92162 feedback_calls=92161 infer_wait=760.531s infer_obs_pack=1.521s env_step=48.001s feedback_total=9.565s feedback_obs_pack=2.393s feedback_info_pack=0.474s effective_fps=445.88 +2026-09-27 04:56:51.169 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=377689 infer_calls=95234 feedback_calls=95233 infer_wait=788.424s infer_obs_pack=1.576s env_step=49.841s feedback_total=9.948s feedback_obs_pack=2.484s feedback_info_pack=0.492s effective_fps=444.45 +2026-09-27 04:57:21.169 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=389790 infer_calls=98270 feedback_calls=98269 infer_wait=815.643s infer_obs_pack=1.641s env_step=51.803s feedback_total=10.325s feedback_obs_pack=2.584s feedback_info_pack=0.510s effective_fps=443.24 +2026-09-27 04:57:51.176 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=401636 infer_calls=101243 feedback_calls=101242 infer_wait=842.969s infer_obs_pack=1.704s env_step=53.661s feedback_total=10.732s feedback_obs_pack=2.688s feedback_info_pack=0.529s effective_fps=441.81 +2026-09-27 04:58:21.184 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=412978 infer_calls=104088 feedback_calls=104087 infer_wait=870.497s infer_obs_pack=1.764s env_step=55.380s feedback_total=11.098s feedback_obs_pack=2.772s feedback_info_pack=0.548s effective_fps=439.93 +2026-09-27 04:58:51.184 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=424041 infer_calls=106859 feedback_calls=106858 infer_wait=898.012s infer_obs_pack=1.820s env_step=57.121s feedback_total=11.458s feedback_obs_pack=2.861s feedback_info_pack=0.567s effective_fps=437.87 +2026-09-27 04:59:24.182 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=434857 infer_calls=109570 feedback_calls=109569 infer_wait=928.434s infer_obs_pack=1.891s env_step=58.905s feedback_total=11.842s feedback_obs_pack=2.960s feedback_info_pack=0.587s effective_fps=434.39 +2026-09-27 04:59:54.185 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=445721 infer_calls=112296 feedback_calls=112295 infer_wait=955.835s infer_obs_pack=1.951s env_step=60.694s feedback_total=12.250s feedback_obs_pack=3.057s feedback_info_pack=0.604s effective_fps=432.43 +2026-09-27 05:00:25.187 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=455268 infer_calls=114690 feedback_calls=114689 infer_wait=984.522s infer_obs_pack=2.012s env_step=62.306s feedback_total=12.592s feedback_obs_pack=3.140s feedback_info_pack=0.620s effective_fps=428.92 +2026-09-27 05:00:58.144 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=467502 infer_calls=117762 feedback_calls=117761 infer_wait=1014.698s infer_obs_pack=2.069s env_step=64.248s feedback_total=12.998s feedback_obs_pack=3.238s feedback_info_pack=0.642s effective_fps=427.33 +2026-09-27 05:01:29.357 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=479726 infer_calls=120834 feedback_calls=120833 infer_wait=1043.287s infer_obs_pack=2.124s env_step=66.098s feedback_total=13.371s feedback_obs_pack=3.329s feedback_info_pack=0.659s effective_fps=426.47 +2026-09-27 05:01:59.901 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=491969 infer_calls=123906 feedback_calls=123905 infer_wait=1071.228s infer_obs_pack=2.182s env_step=67.943s feedback_total=13.727s feedback_obs_pack=3.418s feedback_info_pack=0.677s effective_fps=425.92 +2026-09-27 05:02:30.032 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=504202 infer_calls=126978 feedback_calls=126977 infer_wait=1098.728s infer_obs_pack=2.241s env_step=69.815s feedback_total=14.089s feedback_obs_pack=3.508s feedback_info_pack=0.697s effective_fps=425.53 +2026-09-27 05:03:00.034 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=516785 infer_calls=130142 feedback_calls=130141 infer_wait=1126.079s infer_obs_pack=2.298s env_step=71.687s feedback_total=14.459s feedback_obs_pack=3.602s feedback_info_pack=0.715s effective_fps=425.50 +2026-09-27 05:03:30.038 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=529931 infer_calls=133442 feedback_calls=133441 infer_wait=1153.362s infer_obs_pack=2.355s env_step=73.605s feedback_total=14.849s feedback_obs_pack=3.697s feedback_info_pack=0.736s effective_fps=425.93 +2026-09-27 05:04:00.042 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=542404 infer_calls=136578 feedback_calls=136577 infer_wait=1180.728s infer_obs_pack=2.412s env_step=75.470s feedback_total=15.213s feedback_obs_pack=3.787s feedback_info_pack=0.753s effective_fps=425.81 +2026-09-27 05:04:30.043 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=554815 infer_calls=139693 feedback_calls=139692 infer_wait=1208.163s infer_obs_pack=2.467s env_step=77.294s feedback_total=15.557s feedback_obs_pack=3.874s feedback_info_pack=0.770s effective_fps=425.64 +2026-09-27 05:05:00.046 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=566961 infer_calls=142746 feedback_calls=142745 infer_wait=1235.635s infer_obs_pack=2.520s env_step=79.079s feedback_total=15.906s feedback_obs_pack=3.960s feedback_info_pack=0.786s effective_fps=425.28 +2026-09-27 05:05:30.049 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=579109 infer_calls=145793 feedback_calls=145792 infer_wait=1263.064s infer_obs_pack=2.577s env_step=80.905s feedback_total=16.257s feedback_obs_pack=4.050s feedback_info_pack=0.804s effective_fps=424.94 +2026-09-27 05:06:00.052 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=591603 infer_calls=148925 feedback_calls=148924 infer_wait=1290.480s infer_obs_pack=2.631s env_step=82.736s feedback_total=16.609s feedback_obs_pack=4.138s feedback_info_pack=0.821s effective_fps=424.86 +2026-09-27 05:06:30.056 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=603946 infer_calls=152022 feedback_calls=152021 infer_wait=1317.889s infer_obs_pack=2.686s env_step=84.573s feedback_total=16.967s feedback_obs_pack=4.227s feedback_info_pack=0.841s effective_fps=424.68 +2026-09-27 05:07:00.057 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=616330 infer_calls=155131 feedback_calls=155130 infer_wait=1345.282s infer_obs_pack=2.740s env_step=86.418s feedback_total=17.335s feedback_obs_pack=4.316s feedback_info_pack=0.858s effective_fps=424.54 +2026-09-27 05:07:30.057 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=629112 infer_calls=158340 feedback_calls=158339 infer_wait=1372.664s infer_obs_pack=2.795s env_step=88.263s feedback_total=17.703s feedback_obs_pack=4.410s feedback_info_pack=0.875s effective_fps=424.67 +2026-09-27 05:08:00.062 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=641449 infer_calls=161436 feedback_calls=161435 infer_wait=1400.048s infer_obs_pack=2.850s env_step=90.114s feedback_total=18.072s feedback_obs_pack=4.498s feedback_info_pack=0.893s effective_fps=424.50 +2026-09-27 05:08:30.063 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=654410 infer_calls=164685 feedback_calls=164684 infer_wait=1427.369s infer_obs_pack=2.905s env_step=92.014s feedback_total=18.444s feedback_obs_pack=4.590s feedback_info_pack=0.911s effective_fps=424.74 +2026-09-27 05:09:00.064 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=667173 infer_calls=167883 feedback_calls=167882 infer_wait=1454.754s infer_obs_pack=2.961s env_step=93.866s feedback_total=18.802s feedback_obs_pack=4.681s feedback_info_pack=0.930s effective_fps=424.85 +2026-09-27 05:09:32.549 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=679656 infer_calls=171010 feedback_calls=171009 infer_wait=1484.625s infer_obs_pack=3.016s env_step=95.728s feedback_total=19.156s feedback_obs_pack=4.771s feedback_info_pack=0.947s effective_fps=424.12 +2026-09-27 05:10:02.717 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=691903 infer_calls=174082 feedback_calls=174081 infer_wait=1512.237s infer_obs_pack=3.071s env_step=97.530s feedback_total=19.516s feedback_obs_pack=4.857s feedback_info_pack=0.964s effective_fps=423.87 +2026-09-27 05:10:32.721 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=704887 infer_calls=177340 feedback_calls=177339 infer_wait=1539.555s infer_obs_pack=3.130s env_step=99.423s feedback_total=19.898s feedback_obs_pack=4.951s feedback_info_pack=0.981s effective_fps=424.12 +2026-09-27 05:11:02.725 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=717599 infer_calls=180531 feedback_calls=180530 infer_wait=1566.889s infer_obs_pack=3.184s env_step=101.307s feedback_total=20.269s feedback_obs_pack=5.043s feedback_info_pack=0.999s effective_fps=424.20 +2026-09-27 05:11:32.727 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=730455 infer_calls=183756 feedback_calls=183755 infer_wait=1594.259s infer_obs_pack=3.240s env_step=103.165s feedback_total=20.641s feedback_obs_pack=5.138s feedback_info_pack=1.016s effective_fps=424.36 +2026-09-27 05:12:02.729 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=743317 infer_calls=186980 feedback_calls=186979 infer_wait=1621.679s infer_obs_pack=3.294s env_step=104.984s feedback_total=21.007s feedback_obs_pack=5.226s feedback_info_pack=1.033s effective_fps=424.52 +2026-09-27 05:12:32.730 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=755383 infer_calls=190006 feedback_calls=190005 infer_wait=1649.093s infer_obs_pack=3.348s env_step=106.827s feedback_total=21.356s feedback_obs_pack=5.313s feedback_info_pack=1.050s effective_fps=424.22 +2026-09-27 05:13:02.730 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=767681 infer_calls=193087 feedback_calls=193087 infer_wait=1676.500s infer_obs_pack=3.403s env_step=108.650s feedback_total=21.724s feedback_obs_pack=5.404s feedback_info_pack=1.067s effective_fps=424.07 +2026-09-27 05:13:32.730 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=780586 infer_calls=196321 feedback_calls=196320 infer_wait=1703.833s infer_obs_pack=3.460s env_step=110.535s feedback_total=22.099s feedback_obs_pack=5.495s feedback_info_pack=1.085s effective_fps=424.25 +2026-09-27 05:14:02.734 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=793314 infer_calls=199508 feedback_calls=199507 infer_wait=1731.149s infer_obs_pack=3.516s env_step=112.423s feedback_total=22.483s feedback_obs_pack=5.586s feedback_info_pack=1.103s effective_fps=424.33 +2026-09-27 05:14:32.737 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=805869 infer_calls=202651 feedback_calls=202650 infer_wait=1758.526s infer_obs_pack=3.573s env_step=114.286s feedback_total=22.842s feedback_obs_pack=5.676s feedback_info_pack=1.122s effective_fps=424.31 +2026-09-27 05:15:05.753 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=818533 infer_calls=205826 feedback_calls=205825 infer_wait=1788.932s infer_obs_pack=3.631s env_step=116.123s feedback_total=23.212s feedback_obs_pack=5.768s feedback_info_pack=1.139s effective_fps=423.69 +2026-09-27 05:15:35.759 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=831147 infer_calls=208986 feedback_calls=208985 infer_wait=1816.381s infer_obs_pack=3.686s env_step=117.924s feedback_total=23.575s feedback_obs_pack=5.857s feedback_info_pack=1.156s effective_fps=423.72 +2026-09-27 05:16:05.763 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=843360 infer_calls=212050 feedback_calls=212049 infer_wait=1843.811s infer_obs_pack=3.741s env_step=119.754s feedback_total=23.915s feedback_obs_pack=5.945s feedback_info_pack=1.173s effective_fps=423.54 +2026-09-27 05:16:35.772 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=855981 infer_calls=215214 feedback_calls=215213 infer_wait=1871.234s infer_obs_pack=3.795s env_step=121.571s feedback_total=24.282s feedback_obs_pack=6.032s feedback_info_pack=1.192s effective_fps=423.57 +2026-09-27 05:17:05.816 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=867518 infer_calls=218114 feedback_calls=218113 infer_wait=1898.793s infer_obs_pack=3.849s env_step=123.325s feedback_total=24.630s feedback_obs_pack=6.117s feedback_info_pack=1.208s effective_fps=423.06 +2026-09-27 05:17:35.820 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=881145 infer_calls=221531 feedback_calls=221530 infer_wait=1926.137s infer_obs_pack=3.907s env_step=125.202s feedback_total=25.003s feedback_obs_pack=6.207s feedback_info_pack=1.228s effective_fps=423.58 +2026-09-27 05:18:05.825 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=894987 infer_calls=225001 feedback_calls=225000 infer_wait=1953.490s infer_obs_pack=3.964s env_step=127.075s feedback_total=25.371s feedback_obs_pack=6.298s feedback_info_pack=1.245s effective_fps=424.18 +2026-09-27 05:18:38.133 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=908355 infer_calls=228354 feedback_calls=228353 infer_wait=1983.226s infer_obs_pack=4.019s env_step=128.885s feedback_total=25.734s feedback_obs_pack=6.387s feedback_info_pack=1.262s effective_fps=424.10 +2026-09-27 05:19:08.138 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=922284 infer_calls=231851 feedback_calls=231850 infer_wait=2010.551s infer_obs_pack=4.077s env_step=130.774s feedback_total=26.114s feedback_obs_pack=6.480s feedback_info_pack=1.280s effective_fps=424.72 +2026-09-27 05:19:38.140 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=936266 infer_calls=235359 feedback_calls=235358 infer_wait=2037.891s infer_obs_pack=4.134s env_step=132.656s feedback_total=26.483s feedback_obs_pack=6.572s feedback_info_pack=1.297s effective_fps=425.35 +2026-09-27 05:20:09.604 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=949150 infer_calls=238593 feedback_calls=238592 infer_wait=2066.861s infer_obs_pack=4.189s env_step=134.421s feedback_total=26.826s feedback_obs_pack=6.657s feedback_info_pack=1.313s effective_fps=425.19 +2026-09-27 05:20:39.608 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=963209 infer_calls=242118 feedback_calls=242117 infer_wait=2094.173s infer_obs_pack=4.249s env_step=136.330s feedback_total=27.190s feedback_obs_pack=6.748s feedback_info_pack=1.331s effective_fps=425.83 +2026-09-27 05:21:09.611 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=977221 infer_calls=245634 feedback_calls=245633 infer_wait=2121.528s infer_obs_pack=4.305s env_step=138.210s feedback_total=27.550s feedback_obs_pack=6.841s feedback_info_pack=1.348s effective_fps=426.44 +2026-09-27 05:21:40.729 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=989984 infer_calls=248834 feedback_calls=248833 infer_wait=2150.158s infer_obs_pack=4.361s env_step=139.962s feedback_total=27.899s feedback_obs_pack=6.923s feedback_info_pack=1.364s effective_fps=426.28 +2026-09-27 05:22:10.736 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1003936 infer_calls=252338 feedback_calls=252337 infer_wait=2177.449s infer_obs_pack=4.420s env_step=141.885s feedback_total=28.274s feedback_obs_pack=7.015s feedback_info_pack=1.382s effective_fps=426.84 +2026-09-27 05:22:40.741 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1017808 infer_calls=255817 feedback_calls=255816 infer_wait=2204.742s infer_obs_pack=4.478s env_step=143.819s feedback_total=28.631s feedback_obs_pack=7.107s feedback_info_pack=1.400s effective_fps=427.35 +2026-09-27 05:23:12.145 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1030800 infer_calls=259074 feedback_calls=259073 infer_wait=2233.645s infer_obs_pack=4.532s env_step=145.595s feedback_total=28.971s feedback_obs_pack=7.190s feedback_info_pack=1.416s effective_fps=427.23 +2026-09-27 05:23:42.149 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1044553 infer_calls=262521 feedback_calls=262520 infer_wait=2261.000s infer_obs_pack=4.590s env_step=147.477s feedback_total=29.330s feedback_obs_pack=7.279s feedback_info_pack=1.436s effective_fps=427.68 +2026-09-27 05:24:12.150 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1057998 infer_calls=265899 feedback_calls=265898 infer_wait=2288.351s infer_obs_pack=4.646s env_step=149.339s feedback_total=29.713s feedback_obs_pack=7.372s feedback_info_pack=1.453s effective_fps=427.98 +2026-09-27 05:24:43.767 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1071627 infer_calls=269314 feedback_calls=269313 infer_wait=2317.429s infer_obs_pack=4.702s env_step=151.136s feedback_total=30.058s feedback_obs_pack=7.457s feedback_info_pack=1.470s effective_fps=428.08 +2026-09-27 05:25:15.000 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1083875 infer_calls=272386 feedback_calls=272385 infer_wait=2345.939s infer_obs_pack=4.762s env_step=153.064s feedback_total=30.438s feedback_obs_pack=7.552s feedback_info_pack=1.489s effective_fps=427.70 +2026-09-27 05:25:45.001 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1095383 infer_calls=275275 feedback_calls=275274 infer_wait=2373.295s infer_obs_pack=4.817s env_step=154.910s feedback_total=30.822s feedback_obs_pack=7.643s feedback_info_pack=1.506s effective_fps=427.24 +2026-09-27 05:26:15.002 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1106419 infer_calls=278041 feedback_calls=278040 infer_wait=2400.882s infer_obs_pack=4.871s env_step=156.597s feedback_total=31.173s feedback_obs_pack=7.728s feedback_info_pack=1.522s effective_fps=426.61 +2026-09-27 05:26:45.005 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1117220 infer_calls=280750 feedback_calls=280749 infer_wait=2428.419s infer_obs_pack=4.926s env_step=158.321s feedback_total=31.526s feedback_obs_pack=7.816s feedback_info_pack=1.538s effective_fps=425.90 +2026-09-27 05:27:15.010 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1127305 infer_calls=283279 feedback_calls=283278 infer_wait=2455.989s infer_obs_pack=4.978s env_step=160.028s feedback_total=31.885s feedback_obs_pack=7.900s feedback_info_pack=1.554s effective_fps=424.94 +2026-09-27 05:27:47.414 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1136956 infer_calls=285698 feedback_calls=285697 infer_wait=2486.021s infer_obs_pack=5.030s env_step=161.671s feedback_total=32.250s feedback_obs_pack=7.984s feedback_info_pack=1.572s effective_fps=423.45 +2026-09-27 05:28:17.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1146961 infer_calls=288211 feedback_calls=288210 infer_wait=2513.603s infer_obs_pack=5.087s env_step=163.347s feedback_total=32.618s feedback_obs_pack=8.070s feedback_info_pack=1.589s effective_fps=422.51 +2026-09-27 05:28:47.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1156761 infer_calls=290666 feedback_calls=290666 infer_wait=2541.166s infer_obs_pack=5.141s env_step=165.046s feedback_total=32.975s feedback_obs_pack=8.161s feedback_info_pack=1.607s effective_fps=421.51 +2026-09-27 05:29:18.043 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1165540 infer_calls=292866 feedback_calls=292865 infer_wait=2569.705s infer_obs_pack=5.187s env_step=166.501s feedback_total=33.276s feedback_obs_pack=8.235s feedback_info_pack=1.620s effective_fps=420.06 +2026-09-27 05:29:48.046 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1175923 infer_calls=295465 feedback_calls=295464 infer_wait=2597.215s infer_obs_pack=5.240s env_step=168.242s feedback_total=33.645s feedback_obs_pack=8.322s feedback_info_pack=1.637s effective_fps=419.32 +2026-09-27 05:30:20.592 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1185993 infer_calls=297986 feedback_calls=297985 infer_wait=2627.329s infer_obs_pack=5.300s env_step=169.934s feedback_total=34.005s feedback_obs_pack=8.404s feedback_info_pack=1.653s effective_fps=418.11 +2026-09-27 05:30:50.597 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1197898 infer_calls=300969 feedback_calls=300968 infer_wait=2654.607s infer_obs_pack=5.364s env_step=171.863s feedback_total=34.386s feedback_obs_pack=8.497s feedback_info_pack=1.671s effective_fps=417.94 +2026-09-27 05:31:20.601 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1208245 infer_calls=303563 feedback_calls=303562 infer_wait=2682.271s infer_obs_pack=5.414s env_step=173.506s feedback_total=34.724s feedback_obs_pack=8.584s feedback_info_pack=1.687s effective_fps=417.22 +2026-09-27 05:31:51.210 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1218667 infer_calls=306178 feedback_calls=306177 infer_wait=2710.538s infer_obs_pack=5.465s env_step=175.139s feedback_total=35.067s feedback_obs_pack=8.667s feedback_info_pack=1.706s effective_fps=416.47 +2026-09-27 05:32:02.533 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=1222748 infer_calls=307201 feedback_calls=307201 infer_wait=2717.860s infer_obs_pack=5.486s env_step=175.778s feedback_total=35.194s feedback_obs_pack=8.698s feedback_info_pack=1.712s effective_fps=416.71 +2026-09-27 05:32:02.533 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 4386/1228800 episodes. diff --git a/experiments/e31-dppo-longer/results/dppo-walker/client-seed1.log b/experiments/e31-dppo-longer/results/dppo-walker/client-seed1.log new file mode 100644 index 0000000..7f86615 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker/client-seed1.log @@ -0,0 +1,107 @@ +2026-09-27 04:42:35.440 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-27 04:42:35.441 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-27 04:42:35.441 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-27 04:42:35.441 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-27 04:42:35.442 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-27 04:42:35.487 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260927-044235-a49611e3 output_dir=runs/mujoco-v1-nenv1-20260927-044235-a49611e3 recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-27 04:42:35.526 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9811... +2026-09-27 04:42:35.528 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-27 04:43:05.532 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=14232 infer_calls=3797 feedback_calls=3796 infer_wait=27.458s infer_obs_pack=0.055s env_step=1.689s feedback_total=0.378s feedback_obs_pack=0.096s feedback_info_pack=0.019s effective_fps=481.14 +2026-09-27 04:43:36.589 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=27394 infer_calls=7170 feedback_calls=7169 infer_wait=56.142s infer_obs_pack=0.107s env_step=3.310s feedback_total=0.727s feedback_obs_pack=0.179s feedback_info_pack=0.039s effective_fps=454.39 +2026-09-27 04:44:06.592 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40983 infer_calls=10613 feedback_calls=10612 infer_wait=83.541s infer_obs_pack=0.163s env_step=5.130s feedback_total=1.093s feedback_obs_pack=0.270s feedback_info_pack=0.057s effective_fps=455.74 +2026-09-27 04:44:36.596 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=54601 infer_calls=14052 feedback_calls=14051 infer_wait=110.919s infer_obs_pack=0.234s env_step=6.936s feedback_total=1.477s feedback_obs_pack=0.364s feedback_info_pack=0.076s effective_fps=456.66 +2026-09-27 04:45:08.503 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67912 infer_calls=17410 feedback_calls=17409 infer_wait=140.280s infer_obs_pack=0.291s env_step=8.708s feedback_total=1.839s feedback_obs_pack=0.452s feedback_info_pack=0.093s effective_fps=449.40 +2026-09-27 04:45:38.508 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=82229 infer_calls=21013 feedback_calls=21012 infer_wait=167.656s infer_obs_pack=0.346s env_step=10.552s feedback_total=2.211s feedback_obs_pack=0.543s feedback_info_pack=0.114s effective_fps=454.89 +2026-09-27 04:46:11.354 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=96382 infer_calls=24578 feedback_calls=24578 infer_wait=197.825s infer_obs_pack=0.402s env_step=12.414s feedback_total=2.602s feedback_obs_pack=0.635s feedback_info_pack=0.133s effective_fps=451.98 +2026-09-27 04:46:41.356 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=111123 infer_calls=28292 feedback_calls=28291 infer_wait=225.064s infer_obs_pack=0.463s env_step=14.349s feedback_total=2.991s feedback_obs_pack=0.731s feedback_info_pack=0.152s effective_fps=457.55 +2026-09-27 04:47:12.272 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=124848 infer_calls=31746 feedback_calls=31745 infer_wait=253.586s infer_obs_pack=0.515s env_step=16.030s feedback_total=3.320s feedback_obs_pack=0.815s feedback_info_pack=0.168s effective_fps=456.56 +2026-09-27 04:47:42.275 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=140402 infer_calls=35662 feedback_calls=35661 infer_wait=280.982s infer_obs_pack=0.571s env_step=17.858s feedback_total=3.675s feedback_obs_pack=0.907s feedback_info_pack=0.187s effective_fps=463.24 +2026-09-27 04:48:12.278 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=153593 infer_calls=38985 feedback_calls=38984 infer_wait=308.703s infer_obs_pack=0.619s env_step=19.458s feedback_total=3.992s feedback_obs_pack=0.987s feedback_info_pack=0.203s effective_fps=461.56 +2026-09-27 04:48:42.280 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=169023 infer_calls=42871 feedback_calls=42870 infer_wait=336.099s infer_obs_pack=0.678s env_step=21.295s feedback_total=4.342s feedback_obs_pack=1.078s feedback_info_pack=0.221s effective_fps=466.38 +2026-09-27 04:49:12.283 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=182290 infer_calls=46205 feedback_calls=46204 infer_wait=363.848s infer_obs_pack=0.725s env_step=22.879s feedback_total=4.653s feedback_obs_pack=1.154s feedback_info_pack=0.237s effective_fps=464.90 +2026-09-27 04:49:42.284 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=197724 infer_calls=50092 feedback_calls=50091 infer_wait=391.245s infer_obs_pack=0.781s env_step=24.711s feedback_total=5.012s feedback_obs_pack=1.243s feedback_info_pack=0.256s effective_fps=468.82 +2026-09-27 04:50:12.633 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=210268 infer_calls=53250 feedback_calls=53249 infer_wait=419.458s infer_obs_pack=0.827s env_step=26.215s feedback_total=5.309s feedback_obs_pack=1.319s feedback_info_pack=0.271s effective_fps=465.39 +2026-09-27 04:50:42.634 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=224540 infer_calls=56841 feedback_calls=56840 infer_wait=446.892s infer_obs_pack=0.882s env_step=28.030s feedback_total=5.657s feedback_obs_pack=1.407s feedback_info_pack=0.289s effective_fps=466.37 +2026-09-27 04:51:13.798 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=238761 infer_calls=60418 feedback_calls=60417 infer_wait=475.664s infer_obs_pack=0.935s env_step=29.715s feedback_total=5.982s feedback_obs_pack=1.491s feedback_info_pack=0.306s effective_fps=466.06 +2026-09-27 04:51:43.799 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=252461 infer_calls=63868 feedback_calls=63867 infer_wait=503.055s infer_obs_pack=0.992s env_step=31.550s feedback_total=6.343s feedback_obs_pack=1.583s feedback_info_pack=0.324s effective_fps=465.85 +2026-09-27 04:52:13.806 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=266125 infer_calls=67306 feedback_calls=67305 infer_wait=530.487s infer_obs_pack=1.045s env_step=33.369s feedback_total=6.695s feedback_obs_pack=1.670s feedback_info_pack=0.341s effective_fps=465.58 +2026-09-27 04:52:43.810 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=276822 infer_calls=69998 feedback_calls=69997 infer_wait=558.058s infer_obs_pack=1.097s env_step=35.093s feedback_total=7.031s feedback_obs_pack=1.755s feedback_info_pack=0.357s effective_fps=460.39 +2026-09-27 04:53:15.986 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=291629 infer_calls=73730 feedback_calls=73729 infer_wait=587.642s infer_obs_pack=1.151s env_step=36.907s feedback_total=7.393s feedback_obs_pack=1.846s feedback_info_pack=0.376s effective_fps=460.64 +2026-09-27 04:53:45.994 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=304312 infer_calls=76920 feedback_calls=76919 infer_wait=615.007s infer_obs_pack=1.208s env_step=38.776s feedback_total=7.762s feedback_obs_pack=1.939s feedback_info_pack=0.395s effective_fps=459.16 +2026-09-27 04:54:16.927 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=316035 infer_calls=79874 feedback_calls=79873 infer_wait=643.349s infer_obs_pack=1.263s env_step=40.597s feedback_total=8.125s feedback_obs_pack=2.040s feedback_info_pack=0.412s effective_fps=455.82 +2026-09-27 04:54:48.688 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=328263 infer_calls=82946 feedback_calls=82945 infer_wait=672.397s infer_obs_pack=1.319s env_step=42.488s feedback_total=8.514s feedback_obs_pack=2.139s feedback_info_pack=0.431s effective_fps=452.95 +2026-09-27 04:55:20.103 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=340486 infer_calls=86018 feedback_calls=86017 infer_wait=701.117s infer_obs_pack=1.380s env_step=44.376s feedback_total=8.900s feedback_obs_pack=2.243s feedback_info_pack=0.450s effective_fps=450.51 +2026-09-27 04:55:52.751 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=352716 infer_calls=89090 feedback_calls=89089 infer_wait=730.985s infer_obs_pack=1.443s env_step=46.320s feedback_total=9.304s feedback_obs_pack=2.343s feedback_info_pack=0.471s effective_fps=447.58 +2026-09-27 04:56:22.756 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364844 infer_calls=92135 feedback_calls=92134 infer_wait=758.247s infer_obs_pack=1.503s env_step=48.248s feedback_total=9.700s feedback_obs_pack=2.447s feedback_info_pack=0.493s effective_fps=446.18 +2026-09-27 04:56:52.757 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=375733 infer_calls=94870 feedback_calls=94869 infer_wait=785.823s infer_obs_pack=1.558s env_step=49.949s feedback_total=10.040s feedback_obs_pack=2.533s feedback_info_pack=0.510s effective_fps=443.41 +2026-09-27 04:57:22.760 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=387108 infer_calls=97725 feedback_calls=97724 infer_wait=813.395s infer_obs_pack=1.612s env_step=51.649s feedback_total=10.391s feedback_obs_pack=2.621s feedback_info_pack=0.529s effective_fps=441.38 +2026-09-27 04:57:52.762 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=397750 infer_calls=100395 feedback_calls=100394 infer_wait=840.935s infer_obs_pack=1.668s env_step=53.361s feedback_total=10.763s feedback_obs_pack=2.714s feedback_info_pack=0.550s effective_fps=438.67 +2026-09-27 04:58:26.197 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=409834 infer_calls=103426 feedback_calls=103425 infer_wait=871.574s infer_obs_pack=1.740s env_step=55.326s feedback_total=11.152s feedback_obs_pack=2.819s feedback_info_pack=0.568s effective_fps=436.09 +2026-09-27 04:58:59.412 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=422045 infer_calls=106498 feedback_calls=106497 infer_wait=901.963s infer_obs_pack=1.802s env_step=57.311s feedback_total=11.559s feedback_obs_pack=2.916s feedback_info_pack=0.586s effective_fps=433.92 +2026-09-27 04:59:29.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=432723 infer_calls=109189 feedback_calls=109188 infer_wait=929.397s infer_obs_pack=1.854s env_step=59.095s feedback_total=11.954s feedback_obs_pack=3.027s feedback_info_pack=0.604s effective_fps=431.73 +2026-09-27 04:59:59.598 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=442394 infer_calls=111618 feedback_calls=111617 infer_wait=957.309s infer_obs_pack=1.900s env_step=60.667s feedback_total=12.303s feedback_obs_pack=3.112s feedback_info_pack=0.619s effective_fps=428.60 +2026-09-27 05:00:29.606 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=452803 infer_calls=114234 feedback_calls=114233 infer_wait=984.753s infer_obs_pack=1.958s env_step=62.449s feedback_total=12.689s feedback_obs_pack=3.219s feedback_info_pack=0.636s effective_fps=426.43 +2026-09-27 05:00:59.607 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=463972 infer_calls=117035 feedback_calls=117034 infer_wait=1012.303s infer_obs_pack=2.009s env_step=64.171s feedback_total=13.043s feedback_obs_pack=3.307s feedback_info_pack=0.654s effective_fps=425.07 +2026-09-27 05:01:29.614 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=476165 infer_calls=120094 feedback_calls=120093 infer_wait=1039.744s infer_obs_pack=2.063s env_step=65.980s feedback_total=13.411s feedback_obs_pack=3.396s feedback_info_pack=0.671s effective_fps=424.69 +2026-09-27 05:01:59.614 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=488643 infer_calls=123223 feedback_calls=123222 infer_wait=1067.071s infer_obs_pack=2.123s env_step=67.868s feedback_total=13.790s feedback_obs_pack=3.492s feedback_info_pack=0.689s effective_fps=424.59 +2026-09-27 05:02:29.615 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=501503 infer_calls=126446 feedback_calls=126445 infer_wait=1094.402s infer_obs_pack=2.179s env_step=69.749s feedback_total=14.175s feedback_obs_pack=3.583s feedback_info_pack=0.711s effective_fps=424.82 +2026-09-27 05:02:59.621 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=513573 infer_calls=129472 feedback_calls=129471 infer_wait=1121.827s infer_obs_pack=2.236s env_step=71.576s feedback_total=14.534s feedback_obs_pack=3.679s feedback_info_pack=0.731s effective_fps=424.38 +2026-09-27 05:03:29.627 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=526613 infer_calls=132740 feedback_calls=132739 infer_wait=1149.180s infer_obs_pack=2.293s env_step=73.455s feedback_total=14.904s feedback_obs_pack=3.771s feedback_info_pack=0.749s effective_fps=424.75 +2026-09-27 05:03:59.630 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=538894 infer_calls=135818 feedback_calls=135817 infer_wait=1176.611s infer_obs_pack=2.345s env_step=75.290s feedback_total=15.257s feedback_obs_pack=3.863s feedback_info_pack=0.769s effective_fps=424.49 +2026-09-27 05:04:29.633 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=550985 infer_calls=138848 feedback_calls=138847 infer_wait=1203.983s infer_obs_pack=2.405s env_step=77.156s feedback_total=15.612s feedback_obs_pack=3.954s feedback_info_pack=0.787s effective_fps=424.11 +2026-09-27 05:04:59.633 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=563471 infer_calls=141982 feedback_calls=141982 infer_wait=1231.270s infer_obs_pack=2.465s env_step=79.087s feedback_total=15.977s feedback_obs_pack=4.046s feedback_info_pack=0.807s effective_fps=424.05 +2026-09-27 05:05:29.635 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=576051 infer_calls=145135 feedback_calls=145134 infer_wait=1258.685s infer_obs_pack=2.520s env_step=80.919s feedback_total=16.336s feedback_obs_pack=4.135s feedback_info_pack=0.824s effective_fps=424.05 +2026-09-27 05:05:59.640 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=589358 infer_calls=148468 feedback_calls=148467 infer_wait=1286.000s infer_obs_pack=2.577s env_step=82.814s feedback_total=16.717s feedback_obs_pack=4.231s feedback_info_pack=0.844s effective_fps=424.58 +2026-09-27 05:06:31.906 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=601673 infer_calls=151554 feedback_calls=151553 infer_wait=1315.709s infer_obs_pack=2.630s env_step=84.631s feedback_total=17.072s feedback_obs_pack=4.318s feedback_info_pack=0.864s effective_fps=423.70 +2026-09-27 05:07:01.914 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=614340 infer_calls=154730 feedback_calls=154729 infer_wait=1343.137s infer_obs_pack=2.686s env_step=86.447s feedback_total=17.445s feedback_obs_pack=4.412s feedback_info_pack=0.883s effective_fps=423.77 +2026-09-27 05:07:31.916 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=627105 infer_calls=157929 feedback_calls=157928 infer_wait=1370.484s infer_obs_pack=2.744s env_step=88.324s feedback_total=17.818s feedback_obs_pack=4.504s feedback_info_pack=0.903s effective_fps=423.90 +2026-09-27 05:08:01.919 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=639679 infer_calls=161077 feedback_calls=161076 infer_wait=1397.844s infer_obs_pack=2.799s env_step=90.204s feedback_total=18.179s feedback_obs_pack=4.600s feedback_info_pack=0.921s effective_fps=423.90 +2026-09-27 05:08:31.920 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=652600 infer_calls=164315 feedback_calls=164314 infer_wait=1425.204s infer_obs_pack=2.857s env_step=92.064s feedback_total=18.561s feedback_obs_pack=4.691s feedback_info_pack=0.939s effective_fps=424.13 +2026-09-27 05:09:01.926 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=665118 infer_calls=167450 feedback_calls=167449 infer_wait=1452.614s infer_obs_pack=2.912s env_step=93.898s feedback_total=18.930s feedback_obs_pack=4.780s feedback_info_pack=0.958s effective_fps=424.09 +2026-09-27 05:09:31.930 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=677690 infer_calls=170602 feedback_calls=170601 infer_wait=1480.012s infer_obs_pack=2.969s env_step=95.737s feedback_total=19.300s feedback_obs_pack=4.869s feedback_info_pack=0.977s effective_fps=424.08 +2026-09-27 05:10:01.934 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=689840 infer_calls=173647 feedback_calls=173646 infer_wait=1507.410s infer_obs_pack=3.025s env_step=97.577s feedback_total=19.671s feedback_obs_pack=4.961s feedback_info_pack=0.995s effective_fps=423.82 +2026-09-27 05:10:31.935 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=702419 infer_calls=176799 feedback_calls=176798 infer_wait=1534.796s infer_obs_pack=3.081s env_step=99.425s feedback_total=20.040s feedback_obs_pack=5.053s feedback_info_pack=1.014s effective_fps=423.82 +2026-09-27 05:11:01.939 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=715274 infer_calls=180017 feedback_calls=180016 infer_wait=1562.131s infer_obs_pack=3.138s env_step=101.307s feedback_total=20.415s feedback_obs_pack=5.152s feedback_info_pack=1.033s effective_fps=423.99 +2026-09-27 05:11:31.939 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=727990 infer_calls=183206 feedback_calls=183205 infer_wait=1589.513s infer_obs_pack=3.193s env_step=103.161s feedback_total=20.781s feedback_obs_pack=5.246s feedback_info_pack=1.052s effective_fps=424.08 +2026-09-27 05:12:01.942 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=740460 infer_calls=186331 feedback_calls=186330 infer_wait=1616.970s infer_obs_pack=3.249s env_step=104.960s feedback_total=21.138s feedback_obs_pack=5.338s feedback_info_pack=1.071s effective_fps=424.01 +2026-09-27 05:12:34.043 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=752876 infer_calls=189442 feedback_calls=189441 infer_wait=1646.536s infer_obs_pack=3.305s env_step=106.750s feedback_total=21.495s feedback_obs_pack=5.425s feedback_info_pack=1.089s effective_fps=423.42 +2026-09-27 05:13:04.043 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=765534 infer_calls=192612 feedback_calls=192611 infer_wait=1673.919s infer_obs_pack=3.360s env_step=108.599s feedback_total=21.861s feedback_obs_pack=5.517s feedback_info_pack=1.110s effective_fps=423.48 +2026-09-27 05:13:34.045 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=778284 infer_calls=195810 feedback_calls=195809 infer_wait=1701.346s infer_obs_pack=3.414s env_step=110.413s feedback_total=22.225s feedback_obs_pack=5.607s feedback_info_pack=1.129s effective_fps=423.58 +2026-09-27 05:14:04.052 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=791122 infer_calls=199025 feedback_calls=199024 infer_wait=1728.727s infer_obs_pack=3.470s env_step=112.262s feedback_total=22.595s feedback_obs_pack=5.698s feedback_info_pack=1.148s effective_fps=423.73 +2026-09-27 05:14:34.053 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=804077 infer_calls=202275 feedback_calls=202274 infer_wait=1756.055s infer_obs_pack=3.527s env_step=114.141s feedback_total=22.975s feedback_obs_pack=5.793s feedback_info_pack=1.168s effective_fps=423.94 +2026-09-27 05:15:04.059 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=816411 infer_calls=205368 feedback_calls=205367 infer_wait=1783.503s infer_obs_pack=3.581s env_step=115.941s feedback_total=23.345s feedback_obs_pack=5.887s feedback_info_pack=1.185s effective_fps=423.81 +2026-09-27 05:15:34.063 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=828705 infer_calls=208450 feedback_calls=208449 infer_wait=1810.920s infer_obs_pack=3.641s env_step=117.761s feedback_total=23.709s feedback_obs_pack=5.978s feedback_info_pack=1.203s effective_fps=423.67 +2026-09-27 05:16:04.070 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=841472 infer_calls=211648 feedback_calls=211647 infer_wait=1838.306s infer_obs_pack=3.699s env_step=119.599s feedback_total=24.082s feedback_obs_pack=6.068s feedback_info_pack=1.222s effective_fps=423.77 +2026-09-27 05:16:34.075 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=854607 infer_calls=214938 feedback_calls=214937 infer_wait=1865.629s infer_obs_pack=3.755s env_step=121.490s feedback_total=24.463s feedback_obs_pack=6.161s feedback_info_pack=1.240s effective_fps=424.05 +2026-09-27 05:17:04.075 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=867157 infer_calls=218084 feedback_calls=218083 infer_wait=1893.024s infer_obs_pack=3.815s env_step=123.327s feedback_total=24.832s feedback_obs_pack=6.253s feedback_info_pack=1.259s effective_fps=424.04 +2026-09-27 05:17:35.392 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=879548 infer_calls=221186 feedback_calls=221185 infer_wait=1921.918s infer_obs_pack=3.867s env_step=125.043s feedback_total=25.167s feedback_obs_pack=6.337s feedback_info_pack=1.275s effective_fps=423.68 +2026-09-27 05:18:05.394 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=893748 infer_calls=224749 feedback_calls=224748 infer_wait=1949.195s infer_obs_pack=3.925s env_step=126.953s feedback_total=25.558s feedback_obs_pack=6.433s feedback_info_pack=1.296s effective_fps=424.46 +2026-09-27 05:18:35.396 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=907967 infer_calls=228317 feedback_calls=228316 infer_wait=1976.499s infer_obs_pack=3.983s env_step=128.849s feedback_total=25.940s feedback_obs_pack=6.527s feedback_info_pack=1.315s effective_fps=425.22 +2026-09-27 05:19:05.397 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=920481 infer_calls=231455 feedback_calls=231454 infer_wait=2004.214s infer_obs_pack=4.033s env_step=130.458s feedback_total=26.263s feedback_obs_pack=6.606s feedback_info_pack=1.331s effective_fps=425.17 +2026-09-27 05:19:35.404 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=934717 infer_calls=235025 feedback_calls=235024 infer_wait=2031.491s infer_obs_pack=4.092s env_step=132.388s feedback_total=26.639s feedback_obs_pack=6.700s feedback_info_pack=1.349s effective_fps=425.91 +2026-09-27 05:20:05.406 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=948711 infer_calls=238540 feedback_calls=238539 infer_wait=2058.833s infer_obs_pack=4.149s env_step=134.273s feedback_total=27.003s feedback_obs_pack=6.794s feedback_info_pack=1.368s effective_fps=426.53 +2026-09-27 05:20:35.862 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=961156 infer_calls=241666 feedback_calls=241665 infer_wait=2086.900s infer_obs_pack=4.202s env_step=135.963s feedback_total=27.332s feedback_obs_pack=6.877s feedback_info_pack=1.385s effective_fps=426.35 +2026-09-27 05:21:05.862 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=975324 infer_calls=245225 feedback_calls=245224 infer_wait=2114.158s infer_obs_pack=4.261s env_step=137.893s feedback_total=27.724s feedback_obs_pack=6.973s feedback_info_pack=1.404s effective_fps=427.02 +2026-09-27 05:21:35.863 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=989088 infer_calls=248671 feedback_calls=248670 infer_wait=2141.497s infer_obs_pack=4.318s env_step=139.784s feedback_total=28.089s feedback_obs_pack=7.063s feedback_info_pack=1.422s effective_fps=427.49 +2026-09-27 05:22:07.278 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1002011 infer_calls=251906 feedback_calls=251905 infer_wait=2170.398s infer_obs_pack=4.372s env_step=141.560s feedback_total=28.445s feedback_obs_pack=7.149s feedback_info_pack=1.439s effective_fps=427.34 +2026-09-27 05:22:37.279 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1016269 infer_calls=255476 feedback_calls=255475 infer_wait=2197.609s infer_obs_pack=4.436s env_step=143.532s feedback_total=28.831s feedback_obs_pack=7.244s feedback_info_pack=1.458s effective_fps=428.01 +2026-09-27 05:23:07.280 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1030330 infer_calls=258996 feedback_calls=258995 infer_wait=2224.916s infer_obs_pack=4.493s env_step=145.438s feedback_total=29.207s feedback_obs_pack=7.337s feedback_info_pack=1.476s effective_fps=428.58 +2026-09-27 05:23:38.342 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1042920 infer_calls=262146 feedback_calls=262145 infer_wait=2253.535s infer_obs_pack=4.547s env_step=147.153s feedback_total=29.554s feedback_obs_pack=7.422s feedback_info_pack=1.493s effective_fps=428.34 +2026-09-27 05:24:08.346 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1056198 infer_calls=265474 feedback_calls=265473 infer_wait=2280.904s infer_obs_pack=4.603s env_step=149.014s feedback_total=29.919s feedback_obs_pack=7.512s feedback_info_pack=1.511s effective_fps=428.58 +2026-09-27 05:24:38.347 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1071111 infer_calls=269216 feedback_calls=269215 infer_wait=2308.078s infer_obs_pack=4.664s env_step=151.002s feedback_total=30.322s feedback_obs_pack=7.607s feedback_info_pack=1.531s effective_fps=429.46 +2026-09-27 05:25:08.352 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1083052 infer_calls=272213 feedback_calls=272212 infer_wait=2335.473s infer_obs_pack=4.724s env_step=152.856s feedback_total=30.672s feedback_obs_pack=7.694s feedback_info_pack=1.547s effective_fps=429.15 +2026-09-27 05:25:38.354 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1093508 infer_calls=274835 feedback_calls=274834 infer_wait=2363.078s infer_obs_pack=4.776s env_step=154.533s feedback_total=31.026s feedback_obs_pack=7.778s feedback_info_pack=1.565s effective_fps=428.25 +2026-09-27 05:26:08.354 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1104189 infer_calls=277514 feedback_calls=277513 infer_wait=2390.638s infer_obs_pack=4.830s env_step=156.245s feedback_total=31.380s feedback_obs_pack=7.867s feedback_info_pack=1.581s effective_fps=427.47 +2026-09-27 05:26:40.540 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1116400 infer_calls=280578 feedback_calls=280577 infer_wait=2420.134s infer_obs_pack=4.892s env_step=158.126s feedback_total=31.775s feedback_obs_pack=7.959s feedback_info_pack=1.600s effective_fps=426.93 +2026-09-27 05:27:10.544 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1127388 infer_calls=283341 feedback_calls=283340 infer_wait=2447.566s infer_obs_pack=4.951s env_step=159.915s feedback_total=32.159s feedback_obs_pack=8.050s feedback_info_pack=1.621s effective_fps=426.30 +2026-09-27 05:27:42.226 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1136772 infer_calls=285698 feedback_calls=285697 infer_wait=2476.896s infer_obs_pack=5.004s env_step=161.528s feedback_total=32.521s feedback_obs_pack=8.130s feedback_info_pack=1.640s effective_fps=424.81 +2026-09-27 05:28:12.228 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1146878 infer_calls=288233 feedback_calls=288232 infer_wait=2504.423s infer_obs_pack=5.059s env_step=163.249s feedback_total=32.884s feedback_obs_pack=8.219s feedback_info_pack=1.656s effective_fps=423.89 +2026-09-27 05:28:42.235 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1157090 infer_calls=290792 feedback_calls=290791 infer_wait=2531.950s infer_obs_pack=5.115s env_step=164.987s feedback_total=33.249s feedback_obs_pack=8.312s feedback_info_pack=1.673s effective_fps=423.02 +2026-09-27 05:29:12.238 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1166180 infer_calls=293070 feedback_calls=293069 infer_wait=2559.743s infer_obs_pack=5.165s env_step=166.516s feedback_total=33.581s feedback_obs_pack=8.393s feedback_info_pack=1.687s effective_fps=421.76 +2026-09-27 05:29:42.240 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1176160 infer_calls=295574 feedback_calls=295573 infer_wait=2587.320s infer_obs_pack=5.218s env_step=168.191s feedback_total=33.949s feedback_obs_pack=8.477s feedback_info_pack=1.703s effective_fps=420.86 +2026-09-27 05:30:14.837 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1185782 infer_calls=297985 feedback_calls=297984 infer_wait=2617.534s infer_obs_pack=5.268s env_step=169.850s feedback_total=34.307s feedback_obs_pack=8.562s feedback_info_pack=1.719s effective_fps=419.46 +2026-09-27 05:30:44.838 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1197798 infer_calls=300992 feedback_calls=300991 infer_wait=2644.801s infer_obs_pack=5.328s env_step=171.757s feedback_total=34.716s feedback_obs_pack=8.661s feedback_info_pack=1.737s effective_fps=419.31 +2026-09-27 05:31:14.841 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1208968 infer_calls=303793 feedback_calls=303792 infer_wait=2672.376s infer_obs_pack=5.379s env_step=173.457s feedback_total=35.072s feedback_obs_pack=8.745s feedback_info_pack=1.757s effective_fps=418.87 +2026-09-27 05:31:44.845 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1219236 infer_calls=306368 feedback_calls=306367 infer_wait=2700.024s infer_obs_pack=5.431s env_step=175.109s feedback_total=35.410s feedback_obs_pack=8.826s feedback_info_pack=1.774s effective_fps=418.12 +2026-09-27 05:31:54.711 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=1222564 infer_calls=307201 feedback_calls=307201 infer_wait=2705.956s infer_obs_pack=5.447s env_step=175.635s feedback_total=35.517s feedback_obs_pack=8.852s feedback_info_pack=1.779s effective_fps=418.32 +2026-09-27 05:31:54.711 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 4481/1228800 episodes. diff --git a/experiments/e31-dppo-longer/results/dppo-walker/client-seed2.log b/experiments/e31-dppo-longer/results/dppo-walker/client-seed2.log new file mode 100644 index 0000000..47f2ac5 --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker/client-seed2.log @@ -0,0 +1,106 @@ +2026-09-27 04:42:41.460 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.classic.classic_env: pygame is not installed. Please install it with pip install "plugrl-env-client[classic]". +2026-09-27 04:42:41.460 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.atari.atari_env: Atari is not installed. Please install it with the 'atari' extra, e.g. 'pip install plugrl-env-client[atari]' +2026-09-27 04:42:41.460 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.d4rl.d4rl_env: d4rl is not installed. Please install it with pip install "plugrl-env-client[d4rl]". +2026-09-27 04:42:41.460 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.libero.libero_env: libero is not installed. Please install it with pip install "plugrl-env-client[libero]". +2026-09-27 04:42:41.461 | WARNING | plugrl_env_client.envs:_load_env_modules:27 - Skip loading env module plugrl_env_client.envs.robomimic.robomimic_env: Robomimic is not installed. Please install it with the 'robomimic' extra, e.g. 'pip install plugrl-env-client[robomimic]' +2026-09-27 04:42:41.504 | INFO | __main__:main:83 - Starting env client exp_name=mujoco-v1-nenv1-20260927-044241-d8681abd output_dir=runs/mujoco-v1-nenv1-20260927-044241-d8681abd recorder={'episode_freq': 0, 'thread0_only': True, 'record_video': False, 'video_fps': 30.0, 'record_full_rollout': False, 'record_obs_stats': True, 'record_episode_metrics': True, 'metric_window': 100} +2026-09-27 04:42:41.540 | INFO | plugrl_env_client.agent.websocket_env_client_agent:_wait_for_server:67 - Waiting for server at ws://127.0.0.1:9812... +2026-09-27 04:42:41.543 | INFO | plugrl_env_client.runner.run:report_server_metadata:56 - Server metadata: {'protocol_version': 1, 'server': 'plugrl-server', 'server_version': '0.1.0', 'algorithm': 'DPPOAlgorithm', 'policy': 'DPPOPolicy', 'action_dim': 6, 'action_horizon': 4} +2026-09-27 04:43:11.546 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=13614 infer_calls=3717 feedback_calls=3716 infer_wait=27.518s infer_obs_pack=0.054s env_step=1.630s feedback_total=0.360s feedback_obs_pack=0.087s feedback_info_pack=0.020s effective_fps=460.52 +2026-09-27 04:43:43.667 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=26658 infer_calls=7170 feedback_calls=7169 infer_wait=57.142s infer_obs_pack=0.110s env_step=3.303s feedback_total=0.730s feedback_obs_pack=0.173s feedback_info_pack=0.039s effective_fps=434.98 +2026-09-27 04:44:13.670 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=40329 infer_calls=10647 feedback_calls=10646 infer_wait=84.509s infer_obs_pack=0.167s env_step=5.111s feedback_total=1.126s feedback_obs_pack=0.264s feedback_info_pack=0.058s effective_fps=443.60 +2026-09-27 04:44:43.670 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=53902 infer_calls=14070 feedback_calls=14069 infer_wait=111.890s infer_obs_pack=0.236s env_step=6.919s feedback_total=1.509s feedback_obs_pack=0.354s feedback_info_pack=0.079s effective_fps=447.12 +2026-09-27 04:45:15.537 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=67116 infer_calls=17409 feedback_calls=17408 infer_wait=141.261s infer_obs_pack=0.290s env_step=8.650s feedback_total=1.867s feedback_obs_pack=0.438s feedback_info_pack=0.096s effective_fps=441.35 +2026-09-27 04:45:45.540 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=81647 infer_calls=21074 feedback_calls=21073 infer_wait=168.665s infer_obs_pack=0.346s env_step=10.461s feedback_total=2.238s feedback_obs_pack=0.526s feedback_info_pack=0.114s effective_fps=449.33 +2026-09-27 04:46:17.921 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=95567 infer_calls=24578 feedback_calls=24577 infer_wait=198.369s infer_obs_pack=0.405s env_step=12.328s feedback_total=2.624s feedback_obs_pack=0.618s feedback_info_pack=0.133s effective_fps=447.15 +2026-09-27 04:46:47.930 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=110313 infer_calls=28287 feedback_calls=28286 infer_wait=225.727s infer_obs_pack=0.467s env_step=14.165s feedback_total=3.009s feedback_obs_pack=0.709s feedback_info_pack=0.153s effective_fps=453.28 +2026-09-27 04:47:18.536 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=124046 infer_calls=31746 feedback_calls=31745 infer_wait=254.001s infer_obs_pack=0.517s env_step=15.798s feedback_total=3.328s feedback_obs_pack=0.789s feedback_info_pack=0.169s effective_fps=453.31 +2026-09-27 04:47:48.536 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=139350 infer_calls=35604 feedback_calls=35603 infer_wait=281.388s infer_obs_pack=0.576s env_step=17.631s feedback_total=3.685s feedback_obs_pack=0.878s feedback_info_pack=0.187s effective_fps=459.48 +2026-09-27 04:48:18.561 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=152492 infer_calls=38913 feedback_calls=38912 infer_wait=309.174s infer_obs_pack=0.624s env_step=19.207s feedback_total=3.990s feedback_obs_pack=0.953s feedback_info_pack=0.203s effective_fps=457.94 +2026-09-27 04:48:48.565 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=167764 infer_calls=42758 feedback_calls=42757 infer_wait=336.560s infer_obs_pack=0.681s env_step=21.046s feedback_total=4.346s feedback_obs_pack=1.044s feedback_info_pack=0.221s effective_fps=462.63 +2026-09-27 04:49:18.569 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=181165 infer_calls=46129 feedback_calls=46128 infer_wait=364.303s infer_obs_pack=0.730s env_step=22.629s feedback_total=4.659s feedback_obs_pack=1.121s feedback_info_pack=0.239s effective_fps=461.78 +2026-09-27 04:49:48.574 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=196017 infer_calls=49869 feedback_calls=49868 infer_wait=391.815s infer_obs_pack=0.784s env_step=24.408s feedback_total=4.980s feedback_obs_pack=1.204s feedback_info_pack=0.256s effective_fps=464.51 +2026-09-27 04:50:19.493 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=209452 infer_calls=53250 feedback_calls=53249 infer_wait=420.489s infer_obs_pack=0.832s env_step=25.990s feedback_total=5.284s feedback_obs_pack=1.278s feedback_info_pack=0.271s effective_fps=462.78 +2026-09-27 04:50:49.495 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=223856 infer_calls=56877 feedback_calls=56876 infer_wait=447.926s infer_obs_pack=0.888s env_step=27.800s feedback_total=5.627s feedback_obs_pack=1.364s feedback_info_pack=0.289s effective_fps=464.20 +2026-09-27 04:51:20.233 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=237921 infer_calls=60418 feedback_calls=60417 infer_wait=476.329s infer_obs_pack=0.938s env_step=29.445s feedback_total=5.938s feedback_obs_pack=1.442s feedback_info_pack=0.305s effective_fps=464.10 +2026-09-27 04:51:50.235 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=251914 infer_calls=63942 feedback_calls=63941 infer_wait=503.730s infer_obs_pack=0.994s env_step=31.269s feedback_total=6.305s feedback_obs_pack=1.530s feedback_info_pack=0.322s effective_fps=464.53 +2026-09-27 04:52:20.238 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=264488 infer_calls=67108 feedback_calls=67107 infer_wait=531.309s infer_obs_pack=1.050s env_step=32.977s feedback_total=6.640s feedback_obs_pack=1.612s feedback_info_pack=0.339s effective_fps=462.41 +2026-09-27 04:52:50.239 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=276589 infer_calls=70155 feedback_calls=70154 infer_wait=558.841s infer_obs_pack=1.103s env_step=34.711s feedback_total=6.988s feedback_obs_pack=1.699s feedback_info_pack=0.356s effective_fps=459.72 +2026-09-27 04:53:22.304 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=290798 infer_calls=73730 feedback_calls=73729 infer_wait=588.479s infer_obs_pack=1.156s env_step=36.426s feedback_total=7.314s feedback_obs_pack=1.780s feedback_info_pack=0.373s effective_fps=459.13 +2026-09-27 04:53:52.305 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=303224 infer_calls=76854 feedback_calls=76853 infer_wait=615.847s infer_obs_pack=1.213s env_step=38.283s feedback_total=7.678s feedback_obs_pack=1.874s feedback_info_pack=0.390s effective_fps=457.34 +2026-09-27 04:54:23.119 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=315234 infer_calls=79874 feedback_calls=79873 infer_wait=644.067s infer_obs_pack=1.274s env_step=40.101s feedback_total=8.053s feedback_obs_pack=1.970s feedback_info_pack=0.407s effective_fps=454.56 +2026-09-27 04:54:54.075 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=327445 infer_calls=82946 feedback_calls=82945 infer_wait=672.397s infer_obs_pack=1.333s env_step=41.932s feedback_total=8.438s feedback_obs_pack=2.066s feedback_info_pack=0.429s effective_fps=452.21 +2026-09-27 04:55:25.348 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=339677 infer_calls=86018 feedback_calls=86017 infer_wait=701.027s infer_obs_pack=1.389s env_step=43.786s feedback_total=8.824s feedback_obs_pack=2.159s feedback_info_pack=0.448s effective_fps=449.89 +2026-09-27 04:55:56.643 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=351889 infer_calls=89090 feedback_calls=89089 infer_wait=729.667s infer_obs_pack=1.444s env_step=45.635s feedback_total=9.220s feedback_obs_pack=2.260s feedback_info_pack=0.474s effective_fps=447.72 +2026-09-27 04:56:29.084 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=364116 infer_calls=92162 feedback_calls=92161 infer_wait=759.408s infer_obs_pack=1.510s env_step=47.510s feedback_total=9.610s feedback_obs_pack=2.359s feedback_info_pack=0.495s effective_fps=445.11 +2026-09-27 04:57:00.761 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=376341 infer_calls=95234 feedback_calls=95233 infer_wait=788.376s infer_obs_pack=1.565s env_step=49.409s feedback_total=10.007s feedback_obs_pack=2.458s feedback_info_pack=0.513s effective_fps=443.09 +2026-09-27 04:57:32.144 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=388570 infer_calls=98306 feedback_calls=98305 infer_wait=817.078s infer_obs_pack=1.623s env_step=51.270s feedback_total=10.405s feedback_obs_pack=2.557s feedback_info_pack=0.531s effective_fps=441.37 +2026-09-27 04:58:04.077 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=400813 infer_calls=101377 feedback_calls=101376 infer_wait=846.272s infer_obs_pack=1.687s env_step=53.172s feedback_total=10.811s feedback_obs_pack=2.660s feedback_info_pack=0.550s effective_fps=439.52 +2026-09-27 04:58:36.967 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=413055 infer_calls=104450 feedback_calls=104449 infer_wait=876.368s infer_obs_pack=1.750s env_step=55.130s feedback_total=11.201s feedback_obs_pack=2.761s feedback_info_pack=0.569s effective_fps=437.35 +2026-09-27 04:59:06.970 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=425278 infer_calls=107512 feedback_calls=107511 infer_wait=903.621s infer_obs_pack=1.814s env_step=57.039s feedback_total=11.607s feedback_obs_pack=2.862s feedback_info_pack=0.589s effective_fps=436.59 +2026-09-27 04:59:36.982 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=435355 infer_calls=110041 feedback_calls=110041 infer_wait=931.317s infer_obs_pack=1.865s env_step=58.640s feedback_total=11.959s feedback_obs_pack=2.953s feedback_info_pack=0.604s effective_fps=433.72 +2026-09-27 05:00:08.906 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=445728 infer_calls=112642 feedback_calls=112641 infer_wait=960.787s infer_obs_pack=1.918s env_step=60.328s feedback_total=12.350s feedback_obs_pack=3.054s feedback_info_pack=0.625s effective_fps=430.50 +2026-09-27 05:00:38.912 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=456539 infer_calls=115353 feedback_calls=115352 infer_wait=988.193s infer_obs_pack=1.981s env_step=62.122s feedback_total=12.747s feedback_obs_pack=3.153s feedback_info_pack=0.644s effective_fps=428.66 +2026-09-27 05:01:08.915 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=467475 infer_calls=118094 feedback_calls=118093 infer_wait=1015.768s infer_obs_pack=2.039s env_step=63.817s feedback_total=13.095s feedback_obs_pack=3.242s feedback_info_pack=0.664s effective_fps=427.03 +2026-09-27 05:01:38.923 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=478918 infer_calls=120962 feedback_calls=120961 infer_wait=1043.303s infer_obs_pack=2.091s env_step=65.569s feedback_total=13.442s feedback_obs_pack=3.332s feedback_info_pack=0.687s effective_fps=425.93 +2026-09-27 05:02:08.927 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=490925 infer_calls=123972 feedback_calls=123971 infer_wait=1070.768s infer_obs_pack=2.147s env_step=67.349s feedback_total=13.802s feedback_obs_pack=3.418s feedback_info_pack=0.704s effective_fps=425.39 +2026-09-27 05:02:38.928 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=503475 infer_calls=127121 feedback_calls=127120 infer_wait=1098.151s infer_obs_pack=2.202s env_step=69.180s feedback_total=14.178s feedback_obs_pack=3.506s feedback_info_pack=0.721s effective_fps=425.34 +2026-09-27 05:03:08.933 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=515606 infer_calls=130161 feedback_calls=130160 infer_wait=1125.614s infer_obs_pack=2.255s env_step=70.977s feedback_total=14.538s feedback_obs_pack=3.593s feedback_info_pack=0.740s effective_fps=424.93 +2026-09-27 05:03:38.938 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=528440 infer_calls=133379 feedback_calls=133378 infer_wait=1153.015s infer_obs_pack=2.313s env_step=72.804s feedback_total=14.914s feedback_obs_pack=3.683s feedback_info_pack=0.758s effective_fps=425.12 +2026-09-27 05:04:08.944 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=540773 infer_calls=136471 feedback_calls=136470 infer_wait=1180.462s infer_obs_pack=2.369s env_step=74.612s feedback_total=15.268s feedback_obs_pack=3.770s feedback_info_pack=0.775s effective_fps=424.90 +2026-09-27 05:04:38.944 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=553493 infer_calls=139658 feedback_calls=139657 infer_wait=1207.833s infer_obs_pack=2.423s env_step=76.483s feedback_total=15.624s feedback_obs_pack=3.858s feedback_info_pack=0.792s effective_fps=424.99 +2026-09-27 05:05:08.946 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=566029 infer_calls=142796 feedback_calls=142795 infer_wait=1235.257s infer_obs_pack=2.476s env_step=78.315s feedback_total=15.972s feedback_obs_pack=3.945s feedback_info_pack=0.810s effective_fps=424.94 +2026-09-27 05:05:38.950 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=578545 infer_calls=145933 feedback_calls=145932 infer_wait=1262.653s infer_obs_pack=2.533s env_step=80.153s feedback_total=16.335s feedback_obs_pack=4.033s feedback_info_pack=0.830s effective_fps=424.88 +2026-09-27 05:06:08.955 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=591203 infer_calls=149106 feedback_calls=149105 infer_wait=1290.056s infer_obs_pack=2.591s env_step=81.992s feedback_total=16.697s feedback_obs_pack=4.122s feedback_info_pack=0.848s effective_fps=424.92 +2026-09-27 05:06:38.955 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=604385 infer_calls=152409 feedback_calls=152408 infer_wait=1317.408s infer_obs_pack=2.648s env_step=83.863s feedback_total=17.068s feedback_obs_pack=4.212s feedback_info_pack=0.866s effective_fps=425.33 +2026-09-27 05:07:08.962 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=617017 infer_calls=155571 feedback_calls=155570 infer_wait=1344.824s infer_obs_pack=2.702s env_step=85.688s feedback_total=17.434s feedback_obs_pack=4.302s feedback_info_pack=0.883s effective_fps=425.34 +2026-09-27 05:07:38.967 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=629599 infer_calls=158720 feedback_calls=158719 infer_wait=1372.209s infer_obs_pack=2.760s env_step=87.533s feedback_total=17.808s feedback_obs_pack=4.393s feedback_info_pack=0.901s effective_fps=425.32 +2026-09-27 05:08:11.355 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=641875 infer_calls=161794 feedback_calls=161793 infer_wait=1402.031s infer_obs_pack=2.813s env_step=89.344s feedback_total=18.172s feedback_obs_pack=4.484s feedback_info_pack=0.922s effective_fps=424.42 +2026-09-27 05:08:41.356 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=654743 infer_calls=165015 feedback_calls=165014 infer_wait=1429.415s infer_obs_pack=2.872s env_step=91.193s feedback_total=18.539s feedback_obs_pack=4.575s feedback_info_pack=0.941s effective_fps=424.60 +2026-09-27 05:09:11.360 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=667462 infer_calls=168200 feedback_calls=168199 infer_wait=1456.790s infer_obs_pack=2.929s env_step=93.060s feedback_total=18.903s feedback_obs_pack=4.665s feedback_info_pack=0.959s effective_fps=424.68 +2026-09-27 05:09:41.364 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=680483 infer_calls=171465 feedback_calls=171464 infer_wait=1484.160s infer_obs_pack=2.985s env_step=94.918s feedback_total=19.276s feedback_obs_pack=4.758s feedback_info_pack=0.978s effective_fps=424.95 +2026-09-27 05:10:11.366 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=693326 infer_calls=174685 feedback_calls=174684 infer_wait=1511.549s infer_obs_pack=3.044s env_step=96.751s feedback_total=19.644s feedback_obs_pack=4.849s feedback_info_pack=0.996s effective_fps=425.10 +2026-09-27 05:10:41.368 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=705398 infer_calls=177710 feedback_calls=177709 infer_wait=1539.072s infer_obs_pack=3.097s env_step=98.490s feedback_total=19.996s feedback_obs_pack=4.937s feedback_info_pack=1.013s effective_fps=424.77 +2026-09-27 05:11:11.368 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=718054 infer_calls=180878 feedback_calls=180877 infer_wait=1566.515s infer_obs_pack=3.153s env_step=100.290s feedback_total=20.359s feedback_obs_pack=5.025s feedback_info_pack=1.030s effective_fps=424.80 +2026-09-27 05:11:41.368 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=731217 infer_calls=184178 feedback_calls=184177 infer_wait=1593.855s infer_obs_pack=3.209s env_step=102.155s feedback_total=20.743s feedback_obs_pack=5.117s feedback_info_pack=1.050s effective_fps=425.14 +2026-09-27 05:12:11.368 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=744015 infer_calls=187386 feedback_calls=187385 infer_wait=1621.286s infer_obs_pack=3.264s env_step=103.965s feedback_total=21.106s feedback_obs_pack=5.204s feedback_info_pack=1.072s effective_fps=425.24 +2026-09-27 05:12:43.019 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=756305 infer_calls=190466 feedback_calls=190465 infer_wait=1650.492s infer_obs_pack=3.317s env_step=105.682s feedback_total=21.459s feedback_obs_pack=5.294s feedback_info_pack=1.088s effective_fps=424.66 +2026-09-27 05:13:13.021 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=769043 infer_calls=193663 feedback_calls=193662 infer_wait=1677.863s infer_obs_pack=3.375s env_step=107.544s feedback_total=21.823s feedback_obs_pack=5.385s feedback_info_pack=1.107s effective_fps=424.74 +2026-09-27 05:13:43.026 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=781619 infer_calls=196822 feedback_calls=196821 infer_wait=1705.312s infer_obs_pack=3.428s env_step=109.354s feedback_total=22.181s feedback_obs_pack=5.472s feedback_info_pack=1.124s effective_fps=424.73 +2026-09-27 05:14:13.027 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=794139 infer_calls=199963 feedback_calls=199962 infer_wait=1732.704s infer_obs_pack=3.485s env_step=111.205s feedback_total=22.538s feedback_obs_pack=5.563s feedback_info_pack=1.142s effective_fps=424.69 +2026-09-27 05:14:43.030 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=806760 infer_calls=203126 feedback_calls=203125 infer_wait=1760.100s infer_obs_pack=3.542s env_step=113.043s feedback_total=22.907s feedback_obs_pack=5.654s feedback_info_pack=1.161s effective_fps=424.70 +2026-09-27 05:15:13.035 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=819179 infer_calls=206237 feedback_calls=206236 infer_wait=1787.542s infer_obs_pack=3.597s env_step=114.840s feedback_total=23.280s feedback_obs_pack=5.741s feedback_info_pack=1.179s effective_fps=424.61 +2026-09-27 05:15:43.038 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=831853 infer_calls=209412 feedback_calls=209411 infer_wait=1814.947s infer_obs_pack=3.652s env_step=116.666s feedback_total=23.650s feedback_obs_pack=5.832s feedback_info_pack=1.198s effective_fps=424.65 +2026-09-27 05:16:13.042 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=844266 infer_calls=212517 feedback_calls=212516 infer_wait=1842.344s infer_obs_pack=3.707s env_step=118.511s feedback_total=24.002s feedback_obs_pack=5.921s feedback_info_pack=1.217s effective_fps=424.56 +2026-09-27 05:16:43.046 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=857204 infer_calls=215753 feedback_calls=215752 infer_wait=1869.706s infer_obs_pack=3.762s env_step=120.369s feedback_total=24.382s feedback_obs_pack=6.012s feedback_info_pack=1.235s effective_fps=424.73 +2026-09-27 05:17:13.049 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=870216 infer_calls=219010 feedback_calls=219009 infer_wait=1897.052s infer_obs_pack=3.821s env_step=122.234s feedback_total=24.762s feedback_obs_pack=6.106s feedback_info_pack=1.253s effective_fps=424.94 +2026-09-27 05:17:43.975 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=882971 infer_calls=222210 feedback_calls=222209 infer_wait=1925.544s infer_obs_pack=3.873s env_step=123.942s feedback_total=25.114s feedback_obs_pack=6.188s feedback_info_pack=1.270s effective_fps=424.82 +2026-09-27 05:18:13.977 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=896958 infer_calls=225712 feedback_calls=225711 infer_wait=1952.862s infer_obs_pack=3.931s env_step=125.848s feedback_total=25.479s feedback_obs_pack=6.277s feedback_info_pack=1.288s effective_fps=425.48 +2026-09-27 05:18:43.978 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=910816 infer_calls=229181 feedback_calls=229180 infer_wait=1980.203s infer_obs_pack=3.989s env_step=127.731s feedback_total=25.848s feedback_obs_pack=6.369s feedback_info_pack=1.306s effective_fps=426.06 +2026-09-27 05:19:15.776 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=923852 infer_calls=232449 feedback_calls=232448 infer_wait=2009.534s infer_obs_pack=4.042s env_step=129.482s feedback_total=26.188s feedback_obs_pack=6.452s feedback_info_pack=1.323s effective_fps=425.89 +2026-09-27 05:19:45.781 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=937786 infer_calls=235942 feedback_calls=235941 infer_wait=2036.818s infer_obs_pack=4.100s env_step=131.411s feedback_total=26.564s feedback_obs_pack=6.544s feedback_info_pack=1.341s effective_fps=426.48 +2026-09-27 05:20:15.782 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=951956 infer_calls=239491 feedback_calls=239491 infer_wait=2064.140s infer_obs_pack=4.159s env_step=133.301s feedback_total=26.936s feedback_obs_pack=6.634s feedback_info_pack=1.361s effective_fps=427.17 +2026-09-27 05:20:46.862 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=964726 infer_calls=242690 feedback_calls=242689 infer_wait=2092.797s infer_obs_pack=4.211s env_step=135.005s feedback_total=27.278s feedback_obs_pack=6.716s feedback_info_pack=1.378s effective_fps=427.00 +2026-09-27 05:21:16.867 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=978666 infer_calls=246180 feedback_calls=246179 infer_wait=2120.110s infer_obs_pack=4.273s env_step=136.890s feedback_total=27.664s feedback_obs_pack=6.810s feedback_info_pack=1.396s effective_fps=427.56 +2026-09-27 05:21:46.870 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=992678 infer_calls=249691 feedback_calls=249690 infer_wait=2147.409s infer_obs_pack=4.333s env_step=138.798s feedback_total=28.044s feedback_obs_pack=6.901s feedback_info_pack=1.417s effective_fps=428.14 +2026-09-27 05:22:18.421 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1005610 infer_calls=252930 feedback_calls=252929 infer_wait=2176.478s infer_obs_pack=4.388s env_step=140.551s feedback_total=28.386s feedback_obs_pack=6.984s feedback_info_pack=1.434s effective_fps=427.96 +2026-09-27 05:22:48.422 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1019416 infer_calls=256390 feedback_calls=256389 infer_wait=2203.822s infer_obs_pack=4.445s env_step=142.425s feedback_total=28.755s feedback_obs_pack=7.074s feedback_info_pack=1.452s effective_fps=428.43 +2026-09-27 05:23:18.429 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1033608 infer_calls=259944 feedback_calls=259943 infer_wait=2231.137s infer_obs_pack=4.504s env_step=144.317s feedback_total=29.137s feedback_obs_pack=7.165s feedback_info_pack=1.472s effective_fps=429.04 +2026-09-27 05:23:50.345 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1046474 infer_calls=263170 feedback_calls=263169 infer_wait=2260.576s infer_obs_pack=4.557s env_step=146.058s feedback_total=29.488s feedback_obs_pack=7.252s feedback_info_pack=1.491s effective_fps=428.76 +2026-09-27 05:24:20.347 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1060296 infer_calls=266633 feedback_calls=266632 infer_wait=2287.927s infer_obs_pack=4.616s env_step=147.916s feedback_total=29.864s feedback_obs_pack=7.342s feedback_info_pack=1.510s effective_fps=429.21 +2026-09-27 05:24:50.352 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1075026 infer_calls=270320 feedback_calls=270319 infer_wait=2315.201s infer_obs_pack=4.676s env_step=149.839s feedback_total=30.246s feedback_obs_pack=7.436s feedback_info_pack=1.530s effective_fps=430.02 +2026-09-27 05:25:20.356 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1086034 infer_calls=273078 feedback_calls=273077 infer_wait=2342.733s infer_obs_pack=4.732s env_step=151.582s feedback_total=30.591s feedback_obs_pack=7.522s feedback_info_pack=1.546s effective_fps=429.32 +2026-09-27 05:25:50.362 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1096324 infer_calls=275657 feedback_calls=275656 infer_wait=2370.422s infer_obs_pack=4.782s env_step=153.200s feedback_total=30.935s feedback_obs_pack=7.602s feedback_info_pack=1.562s effective_fps=428.36 +2026-09-27 05:26:20.484 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1107779 infer_calls=278530 feedback_calls=278529 infer_wait=2398.057s infer_obs_pack=4.835s env_step=154.946s feedback_total=31.290s feedback_obs_pack=7.686s feedback_info_pack=1.580s effective_fps=427.86 +2026-09-27 05:26:53.417 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1120047 infer_calls=281602 feedback_calls=281601 infer_wait=2428.254s infer_obs_pack=4.896s env_step=156.861s feedback_total=31.687s feedback_obs_pack=7.783s feedback_info_pack=1.598s effective_fps=427.22 +2026-09-27 05:27:23.419 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1129634 infer_calls=284008 feedback_calls=284007 infer_wait=2455.917s infer_obs_pack=4.949s env_step=158.477s feedback_total=32.040s feedback_obs_pack=7.865s feedback_info_pack=1.617s effective_fps=426.05 +2026-09-27 05:27:53.422 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1139832 infer_calls=286568 feedback_calls=286567 infer_wait=2483.435s infer_obs_pack=5.007s env_step=160.188s feedback_total=32.422s feedback_obs_pack=7.958s feedback_info_pack=1.634s effective_fps=425.14 +2026-09-27 05:28:23.749 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1148618 infer_calls=288770 feedback_calls=288769 infer_wait=2511.654s infer_obs_pack=5.053s env_step=161.648s feedback_total=32.746s feedback_obs_pack=8.034s feedback_info_pack=1.648s effective_fps=423.67 +2026-09-27 05:28:53.751 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1158560 infer_calls=291259 feedback_calls=291258 infer_wait=2539.222s infer_obs_pack=5.105s env_step=163.335s feedback_total=33.114s feedback_obs_pack=8.119s feedback_info_pack=1.666s effective_fps=422.71 +2026-09-27 05:29:25.924 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1169037 infer_calls=293890 feedback_calls=293889 infer_wait=2568.969s infer_obs_pack=5.160s env_step=164.985s feedback_total=33.500s feedback_obs_pack=8.207s feedback_info_pack=1.684s effective_fps=421.64 +2026-09-27 05:29:55.932 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1179143 infer_calls=296423 feedback_calls=296422 infer_wait=2596.520s infer_obs_pack=5.212s env_step=166.685s feedback_total=33.882s feedback_obs_pack=8.296s feedback_info_pack=1.704s effective_fps=420.78 +2026-09-27 05:30:28.241 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1189464 infer_calls=299010 feedback_calls=299009 infer_wait=2626.382s infer_obs_pack=5.263s env_step=168.397s feedback_total=34.241s feedback_obs_pack=8.389s feedback_info_pack=1.721s effective_fps=419.67 +2026-09-27 05:30:58.243 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1201098 infer_calls=301927 feedback_calls=301926 infer_wait=2653.725s infer_obs_pack=5.320s env_step=170.263s feedback_total=34.620s feedback_obs_pack=8.483s feedback_info_pack=1.740s effective_fps=419.39 +2026-09-27 05:31:28.244 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Intermediate rollout timing summary: env_steps=1211614 infer_calls=304561 feedback_calls=304560 infer_wait=2681.423s infer_obs_pack=5.371s env_step=171.882s feedback_total=34.948s feedback_obs_pack=8.565s feedback_info_pack=1.757s effective_fps=418.72 +2026-09-27 05:31:58.590 | INFO | plugrl_env_client.runner.rollout:log_timing_summary:98 - Final rollout timing summary: env_steps=1222168 infer_calls=307201 feedback_calls=307201 infer_wait=2706.212s infer_obs_pack=5.422s env_step=173.561s feedback_total=35.296s feedback_obs_pack=8.650s feedback_info_pack=1.775s effective_fps=418.48 +2026-09-27 05:31:58.590 | INFO | plugrl_env_client.runner.run:run:156 - Server signalled the end of the run; collection stopped after 4786/1228800 episodes. diff --git a/experiments/e31-dppo-longer/results/dppo-walker/server-seed0.log b/experiments/e31-dppo-longer/results/dppo-walker/server-seed0.log new file mode 100644 index 0000000..d1cfaae --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker/server-seed0.log @@ -0,0 +1,36 @@ +04:42:30|INFO|plugrl_server version: 0.1.0 +04:42:30|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=307200, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, logprob_clamp_min=-5.0, logprob_clamp_max=2.0, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=300, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +04:42:30|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +04:42:30|INFO|Seeded python, numpy and torch with 0. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +04:42:30|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0 +04:42:30|INFO|Number of network parameters: 568392 +04:42:30|INFO|Policy created... +04:42:30|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +04:42:30|INFO|Algorithm created: + +04:42:30|INFO|Agent Server is listening on 0.0.0.0:9810 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +04:45:30|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/20481 +04:48:24|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/40961 +04:51:18|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/61441 +04:54:34|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/81921 +04:58:06|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/102401 +05:01:49|INFO|Checkpoint saved at step 122881 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/122881 +05:05:07|INFO|Checkpoint saved at step 143361 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/143361 +05:08:24|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/163841 +05:11:39|INFO|Checkpoint saved at step 184321 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/184321 +05:14:55|INFO|Checkpoint saved at step 204801 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/204801 +05:18:10|INFO|Checkpoint saved at step 225281 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/225281 +05:21:13|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/245761 +05:24:17|INFO|Checkpoint saved at step 266241 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/266241 +05:28:00|INFO|Checkpoint saved at step 286721 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/286721 +05:32:02|INFO|Checkpoint saved at step 307201 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed0/307201 +05:32:02|INFO|Stopping server as the algorithm signaled to stop. +05:32:02|INFO|Shutdown started: aborting pending infer requests. +05:32:02|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +05:32:02|INFO|Shutdown closing 1 websocket connection(s). +05:32:02|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 41276). +05:32:02|INFO|WebSocket server closed. +05:32:02|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e31-dppo-longer/results/dppo-walker/server-seed1.log b/experiments/e31-dppo-longer/results/dppo-walker/server-seed1.log new file mode 100644 index 0000000..9e9135a --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker/server-seed1.log @@ -0,0 +1,36 @@ +04:42:35|INFO|plugrl_server version: 0.1.0 +04:42:35|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=307200, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, logprob_clamp_min=-5.0, logprob_clamp_max=2.0, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=300, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +04:42:35|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +04:42:35|INFO|Seeded python, numpy and torch with 1. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +04:42:35|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1 +04:42:35|INFO|Number of network parameters: 568392 +04:42:35|INFO|Policy created... +04:42:35|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +04:42:35|INFO|Algorithm created: + +04:42:35|INFO|Agent Server is listening on 0.0.0.0:9811 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +04:45:35|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/20481 +04:48:29|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/40961 +04:51:22|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/61441 +04:54:37|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/81921 +04:58:14|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/102401 +05:01:57|INFO|Checkpoint saved at step 122881 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/122881 +05:05:14|INFO|Checkpoint saved at step 143361 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/143361 +05:08:28|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/163841 +05:11:45|INFO|Checkpoint saved at step 184321 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/184321 +05:15:00|INFO|Checkpoint saved at step 204801 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/204801 +05:18:11|INFO|Checkpoint saved at step 225281 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/225281 +05:21:11|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/245761 +05:24:15|INFO|Checkpoint saved at step 266241 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/266241 +05:27:55|INFO|Checkpoint saved at step 286721 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/286721 +05:31:54|INFO|Checkpoint saved at step 307201 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed1/307201 +05:31:54|INFO|Stopping server as the algorithm signaled to stop. +05:31:54|INFO|Shutdown started: aborting pending infer requests. +05:31:54|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +05:31:54|INFO|Shutdown closing 1 websocket connection(s). +05:31:54|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 48988). +05:31:54|INFO|WebSocket server closed. +05:31:54|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e31-dppo-longer/results/dppo-walker/server-seed2.log b/experiments/e31-dppo-longer/results/dppo-walker/server-seed2.log new file mode 100644 index 0000000..e157eee --- /dev/null +++ b/experiments/e31-dppo-longer/results/dppo-walker/server-seed2.log @@ -0,0 +1,36 @@ +04:42:40|INFO|plugrl_server version: 0.1.0 +04:42:40|INFO|Algorithm: dppo, Config: DPPOAlgoConfigWalker(global_steps=307200, gamma=0.99, gamma_denoising=0.99, actor_lr=0.0001, critic_lr=0.001, actor_weight_decay=0.0, critic_weight_decay=0.0, actor_lr_scheduler=SchedulerConfig(min_lr=0.0001, warmup_steps=10), critic_lr_scheduler=SchedulerConfig(min_lr=0.001, warmup_steps=10), buffer_size=1024, gae_lambda=0.95, update_epochs=5, norm_adv=True, clip_ploss_coef=0.01, clip_ploss_coef_base=0.01, clip_ploss_coef_rate=3, clip_vloss_coef=inf, ent_coef=0.0, vf_coef=0.5, max_grad_norm=inf, target_kl=inf, logprob_noise_level=0.1, sampling_noise_level=0.1, logprob_clamp_min=-5.0, logprob_clamp_max=2.0, clip_advantage_lower_quantile=0, clip_advantage_upper_quantile=1, n_critic_warmup_itrs=0, use_normalized_rewards=True, batch_size=512, train_itrs=300, save_interval=20, grad_accum_steps=8, policy_checkpoint_path=None, restore='all') +04:42:40|INFO|Policy: dppo-policy, Config: DPPOPolicyConfigWalker(algo='dppo', device=device(type='cpu'), env_type='gym', env_name='walker2d-medium-v2', checkpoint_path=None, critic=DPPOCriticObsConfig(mlp_dims=[256, 256, 256], activation='Mish', residual_style=True)) +04:42:40|INFO|Seeded python, numpy and torch with 2. One env client is then reproducible; several are not, because batch composition depends on when their requests arrive. +04:42:40|INFO|Checkpoint Manager created: + at /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2 +04:42:40|INFO|Number of network parameters: 568392 +04:42:40|INFO|Policy created... +04:42:40|INFO|Initialized RolloutBuffer buffer_size=1024 action_shape=(1024, 20, 4, 6) value_shape=(1024,) +04:42:40|INFO|Algorithm created: + +04:42:40|INFO|Agent Server is listening on 0.0.0.0:9812 +/home/guangzhao/zuogou/plugrl/plugrl-server/.venv/lib/python3.11/site-packages/torch/utils/data/dataloader.py:665: UserWarning: 'pin_memory' argument is set as true but no accelerator is found, then device pinned memory won't be used. + warnings.warn(warn_msg) +04:45:42|INFO|Checkpoint saved at step 20481 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/20481 +04:48:35|INFO|Checkpoint saved at step 40961 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/40961 +04:51:29|INFO|Checkpoint saved at step 61441 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/61441 +04:54:43|INFO|Checkpoint saved at step 81921 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/81921 +04:58:14|INFO|Checkpoint saved at step 102401 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/102401 +05:01:57|INFO|Checkpoint saved at step 122881 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/122881 +05:05:15|INFO|Checkpoint saved at step 143361 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/143361 +05:08:30|INFO|Checkpoint saved at step 163841 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/163841 +05:11:45|INFO|Checkpoint saved at step 184321 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/184321 +05:15:00|INFO|Checkpoint saved at step 204801 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/204801 +05:18:11|INFO|Checkpoint saved at step 225281 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/225281 +05:21:14|INFO|Checkpoint saved at step 245761 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/245761 +05:24:18|INFO|Checkpoint saved at step 266241 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/266241 +05:27:59|INFO|Checkpoint saved at step 286721 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/286721 +05:31:58|INFO|Checkpoint saved at step 307201 to /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/results/dppo-walker/dppo/dppo-policy/dppo-walker-seed2/307201 +05:31:58|INFO|Stopping server as the algorithm signaled to stop. +05:31:58|INFO|Shutdown started: aborting pending infer requests. +05:31:58|INFO|Shutdown cleanup finished: pending_futures=1, drained_requests=1 +05:31:58|INFO|Shutdown closing 1 websocket connection(s). +05:31:58|INFO|Shutdown interrupted an in-flight inference request from ('127.0.0.1', 42320). +05:31:58|INFO|WebSocket server closed. +05:31:58|INFO|Scheduler task cancelled and cleaned up. diff --git a/experiments/e31-dppo-longer/results/run.out b/experiments/e31-dppo-longer/results/run.out new file mode 100644 index 0000000..51f7056 --- /dev/null +++ b/experiments/e31-dppo-longer/results/run.out @@ -0,0 +1,8 @@ +start 2026-09-27 04:42:08 +code: f8ffec0 (src /home/guangzhao/zuogou/plugrl/e31/src) +client: 931ab56 +iters: 300 +end 2026-09-27 05:32:02 +dppo-hopper failed seeds: 0 +dppo-walker failed seeds: 0 +E31_DONE diff --git a/experiments/e31-dppo-longer/results/verdicts.txt b/experiments/e31-dppo-longer/results/verdicts.txt new file mode 100644 index 0000000..6b2bc29 --- /dev/null +++ b/experiments/e31-dppo-longer/results/verdicts.txt @@ -0,0 +1,31 @@ +P1 every cell runs end to end + dppo-hopper dppo-policy Hopper-v5 HOLDS (48 min) + dppo-walker dppo-policy Walker2d-v5 HOLDS (50 min) + +status learns: mean of iterations 291-300 >= 500 on 2 of 3 + dppo-hopper 872.8, 961.0, 812.1 3 of 3 -> learns + dppo-walker 528.0, 622.4, 584.9 3 of 3 -> learns + +P2 dppo-policy learns Hopper + HOLDS + +P3 dppo-policy learns Walker2d + HOLDS + +reported: mean return at E28's length (91-100), and at 191-200, 291-300 + dppo-hopper seed 0 336.1 685.3 872.8 + dppo-hopper seed 1 405.5 830.5 961.0 + dppo-hopper seed 2 350.1 752.6 812.1 + dppo-walker seed 0 447.9 602.5 528.0 + dppo-walker seed 1 435.6 540.3 622.4 + dppo-walker seed 2 392.6 476.6 584.9 + +reported: first iteration and mean of the last ten, return and episode length + dppo-hopper seed 0 return 11.2 -> 872.8 length 18.7 -> 456.3 + dppo-hopper seed 1 return 12.4 -> 961.0 length 16.8 -> 628.2 + dppo-hopper seed 2 return 15.8 -> 812.1 length 21.0 -> 652.2 + dppo-walker seed 0 return -2.2 -> 528.0 length 17.6 -> 422.2 + dppo-walker seed 1 return -1.6 -> 622.4 length 17.6 -> 540.8 + dppo-walker seed 2 return -2.6 -> 584.9 length 15.1 -> 517.3 + +wrote /home/guangzhao/zuogou/plugrl/e31/experiments/e31-dppo-longer/summary.tsv diff --git a/experiments/e31-dppo-longer/run.sh b/experiments/e31-dppo-longer/run.sh new file mode 100644 index 0000000..0d44bb0 --- /dev/null +++ b/experiments/e31-dppo-longer/run.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# E31: E28's two rising dppo-policy cells, run to 300 iterations. See PROTOCOL.md. +# +# bash run.sh +# +# Both cells are E24's run_cell.sh, unchanged, as E28 ran them; only the +# number of iterations differs. The server runs this checkout's code through +# PYTHONPATH; the Pythons are the venvs beside it on guangzhao. +set -uo pipefail + +HERE="$(cd "$(dirname "$0")" && pwd)" +ROOT="$(cd "$HERE/../.." && pwd)" +R="$HERE/results" +CELL="$ROOT/experiments/e24-coverage/run_cell.sh" +ITERS="${ITERS:-300}" +mkdir -p "$R" + +export OMP_NUM_THREADS=1 +export PYTHONPATH="$ROOT/src" +export PLUGRL_ENV_CLIENT="${PLUGRL_ENV_CLIENT:-$ROOT/../plugrl-env-client}" +export PLUGRL_SERVER_PYTHON="${PLUGRL_SERVER_PYTHON:-$ROOT/../plugrl-server/.venv/bin/python}" +export PLUGRL_CLIENT_PYTHON="${PLUGRL_CLIENT_PYTHON:-$PLUGRL_ENV_CLIENT/.venv/bin/python}" +for py in "$PLUGRL_SERVER_PYTHON" "$PLUGRL_CLIENT_PYTHON"; do + [ -x "$py" ] || { echo "error: no Python at $py" >&2; exit 1; } +done + +date '+start %F %T' +echo "code: $(git -C "$ROOT" rev-parse --short HEAD) (src $PYTHONPATH)" +echo "client: $(git -C "$PLUGRL_ENV_CLIENT" rev-parse --short HEAD)" +echo "iters: $ITERS" + +BUFFER=1024 BATCH=512 PORT_BASE=9800 bash "$CELL" dppo-hopper dppo-policy hopper dppo hopper Hopper-v5 "$ITERS" "$R/dppo-hopper" > "$R/dppo-hopper.out" 2>&1 & +A=$! +sleep 20 +BUFFER=1024 BATCH=512 PORT_BASE=9810 bash "$CELL" dppo-walker dppo-policy walker dppo walker Walker2d-v5 "$ITERS" "$R/dppo-walker" > "$R/dppo-walker.out" 2>&1 & +B=$! +wait "$A" "$B" + +date '+end %F %T' +for c in dppo-hopper dppo-walker; do + printf '%-12s %s\n' "$c" "$(grep -E 'failed seeds' "$R/$c.out" 2>/dev/null)" +done +echo E31_DONE diff --git a/experiments/e31-dppo-longer/summarise.py b/experiments/e31-dppo-longer/summarise.py new file mode 100644 index 0000000..28981bc --- /dev/null +++ b/experiments/e31-dppo-longer/summarise.py @@ -0,0 +1,228 @@ +"""Read E31's verdicts from each cell's logs and tensorboards. + + python summarise.py + +P1 cell by cell, then the status rule, then P2 and P3 - PROTOCOL.md's +reading order - and the return at E28's length (iterations 91-100) beside +the end. E28's summarise.py with its two dppo-policy cells at 300 +iterations. Writes `summary.tsv` in E24's columns. +""" + +import datetime +import pathlib +import re +import sys + +from tensorboard.backend.event_processing.event_accumulator import EventAccumulator + +HERE = pathlib.Path(__file__).resolve().parent +RESULTS = HERE / "results" +SEEDS = (0, 1, 2) +ITERS = 300 + +# cell: (policy, algorithm, task) +CELLS = { + "dppo-hopper": ("dppo-policy", "dppo", "Hopper-v5"), + "dppo-walker": ("dppo-policy", "dppo", "Walker2d-v5"), +} +ABSOLUTE = 500.0 # Hopper, Walker2d: mean of iterations 291-300 +GAIN = 200.0 # HalfCheetah (not run here): the last ten minus the first + + +def run_dir(cell: str, seed: int) -> pathlib.Path | None: + hits = sorted((RESULTS / cell).glob(f"*/*/{cell}-seed{seed}")) + return hits[0] if hits else None + + +def curve(run: pathlib.Path, tag: str) -> list[float]: + events = sorted((run / "tensorboard").glob("events.*")) + if not events: + return [] + acc = EventAccumulator(str(events[0]), size_guidance={"scalars": 0}) + acc.Reload() + if tag not in acc.Tags()["scalars"]: + return [] + return [e.value for e in acc.Scalars(tag)] + + +def wall_minutes(out: str) -> float | None: + stamps = dict(re.findall(r"^(start|end)\s+(\S+ \S+)$", out, flags=re.M)) + if "start" not in stamps or "end" not in stamps: + return None + fmt = "%Y-%m-%d %H:%M:%S" + delta = datetime.datetime.strptime(stamps["end"], fmt) - datetime.datetime.strptime( + stamps["start"], fmt + ) + return delta.total_seconds() / 60 + + +def mean(xs: list[float]) -> float: + return sum(xs) / len(xs) if xs else float("nan") + + +def seed_passes(task: str, reward: list[float]) -> bool: + if len(reward) < ITERS: + return False + last10 = mean(reward[ITERS - 10 : ITERS]) + if task == "HalfCheetah-v5": + return last10 - reward[0] >= GAIN + return last10 >= ABSOLUTE + + +def main() -> int: + rows = {} + print("P1 every cell runs end to end") + runs_ok = {} + for cell, (policy, algo, task) in CELLS.items(): + out_path = RESULTS / f"{cell}.out" + out = ( + out_path.read_text(encoding="utf-8", errors="replace") + if out_path.exists() + else "" + ) + exits = { + int(s): int(rc) for s, rc in re.findall(r"seed (\d) finished rc=(\d+)", out) + } + ok_cell = True + for seed in SEEDS: + run = run_dir(cell, seed) + reward = curve(run, "rollout/reward") if run else [] + length = curve(run, "rollout/length") if run else [] + saves = ( + [int(p.name) for p in run.iterdir() if p.name.isdigit()] if run else [] + ) + log_path = RESULTS / cell / f"server-seed{seed}.log" + log = ( + log_path.read_text(encoding="utf-8", errors="replace") + if log_path.exists() + else "" + ) + # The checkpoint directory's step can be off by one (E16), so + # the last save is found by count, not by name. + ok = ( + len(reward) >= ITERS + and len(saves) >= ITERS // 20 + and bool(log) + and "Traceback" not in log + and exits.get(seed) == 0 + ) + ok_cell &= ok + rows[cell, seed] = dict( + ok=ok, reward=reward, length=length, passes=seed_passes(task, reward) + ) + runs_ok[cell] = ok_cell + minutes = wall_minutes(out) + print( + f" {cell:15s} {policy:12s} {task:15s} " + f"{'HOLDS' if ok_cell else 'FAILS'}" + + (f" ({minutes:.0f} min)" if minutes is not None else "") + ) + for seed in SEEDS: + r = rows[cell, seed] + if not r["ok"]: + print( + f" seed {seed}: {len(r['reward'])} of {ITERS} iterations logged" + ) + + print(f"\nstatus learns: mean of iterations 291-300 >= {ABSOLUTE:.0f} on 2 of 3") + status = {} + for cell, (policy, algo, task) in CELLS.items(): + passed = sum(rows[cell, s]["passes"] for s in SEEDS) + if not runs_ok[cell]: + status[cell] = "did not run" + elif passed >= 2: + status[cell] = "learns" + else: + status[cell] = "did not learn in 1,228,800 steps" + figures = [] + for s in SEEDS: + reward = rows[cell, s]["reward"] + last10 = ( + mean(reward[ITERS - 10 : ITERS]) + if len(reward) >= ITERS + else float("nan") + ) + if task == "HalfCheetah-v5" and reward: + figures.append(f"{last10 - reward[0]:+.1f}") + else: + figures.append(f"{last10:.1f}") + print( + f" {cell:15s} {', '.join(figures):28s} {passed} of 3 -> {status[cell]}" + ) + + def verdict(name: str, text: str, holds: bool | None) -> None: + mark = ( + "not read - a cell did not run" + if holds is None + else ("HOLDS" if holds else "FALSIFIED") + ) + print(f"\n{name} {text}\n {mark}") + + def learns(cell: str) -> bool | None: + return None if status[cell] == "did not run" else status[cell] == "learns" + + verdict("P2", "dppo-policy learns Hopper", learns("dppo-hopper")) + verdict("P3", "dppo-policy learns Walker2d", learns("dppo-walker")) + + print("\nreported: mean return at E28's length (91-100), and at 191-200, 291-300") + for cell in CELLS: + for s in SEEDS: + r = rows[cell, s]["reward"] + spans = [ + mean(r[a:b]) if len(r) >= b else float("nan") + for a, b in ((90, 100), (190, 200), (290, 300)) + ] + print(f" {cell:15s} seed {s} " + " ".join(f"{v:7.1f}" for v in spans)) + + print( + "\nreported: first iteration and mean of the last ten, return and episode length" + ) + lines = [ + "cell\tpolicy\talgorithm\ttask\tclaim\tseed\titerations\truns\tfirst_return\tlast10_return\tfirst_length\tlast10_length" + ] + for cell, (policy, algo, task) in CELLS.items(): + for s in SEEDS: + r = rows[cell, s] + reward, length = r["reward"], r["length"] + first = reward[0] if reward else float("nan") + last10 = ( + mean(reward[ITERS - 10 : ITERS]) + if len(reward) >= ITERS + else float("nan") + ) + first_len = length[0] if length else float("nan") + last10_len = ( + mean(length[ITERS - 10 : ITERS]) + if len(length) >= ITERS + else float("nan") + ) + print( + f" {cell:15s} seed {s} return {first:8.1f} -> {last10:8.1f} " + f"length {first_len:6.1f} -> {last10_len:6.1f}" + ) + claim = "learns" if status[cell] == "learns" else "runs" + lines.append( + "\t".join( + [ + cell, + policy, + algo, + task, + claim, + str(s), + str(len(reward)), + str(r["ok"]).lower(), + f"{first:.1f}", + f"{last10:.1f}", + f"{first_len:.1f}", + f"{last10_len:.1f}", + ] + ) + ) + (HERE / "summary.tsv").write_text("\n".join(lines) + "\n", encoding="utf-8") + print(f"\nwrote {HERE / 'summary.tsv'}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/experiments/e31-dppo-longer/summary.tsv b/experiments/e31-dppo-longer/summary.tsv new file mode 100644 index 0000000..356a30c --- /dev/null +++ b/experiments/e31-dppo-longer/summary.tsv @@ -0,0 +1,7 @@ +cell policy algorithm task claim seed iterations runs first_return last10_return first_length last10_length +dppo-hopper dppo-policy dppo Hopper-v5 learns 0 300 true 11.2 872.8 18.7 456.3 +dppo-hopper dppo-policy dppo Hopper-v5 learns 1 300 true 12.4 961.0 16.8 628.2 +dppo-hopper dppo-policy dppo Hopper-v5 learns 2 300 true 15.8 812.1 21.0 652.2 +dppo-walker dppo-policy dppo Walker2d-v5 learns 0 300 true -2.2 528.0 17.6 422.2 +dppo-walker dppo-policy dppo Walker2d-v5 learns 1 300 true -1.6 622.4 17.6 540.8 +dppo-walker dppo-policy dppo Walker2d-v5 learns 2 300 true -2.6 584.9 15.1 517.3