deps: base install drops pandas and ffmpeg (video extra) - 201.6 MB to 72.5 MB

README.md

@@ -8,17 +8,18 @@
 ## ✨ Features
 
   * **No deep learning dependencies**: The policy stays on the server. The base
-    install declares nine runtime dependencies - `gymnasium`, `websockets` and
+    install declares eight runtime dependencies - `gymnasium`, `websockets` and
     `msgpack` do the environment and wire work, next to `loguru`, `dm-tree`,
-    `tyro`, `plugrl-protocol`, `pandas` and `imageio[ffmpeg]` - and none of them
-    is a deep learning framework, so environments pinned to old `mujoco-py` or
-    `cython<3` can be used with a modern training stack. E1 measures that bare
-    install at 30 packages and 224M on Linux with no CUDA wheels
-    (`plugrl-server/experiments/e1-dependency-conflict/`, round 4). Correction:
-    this line previously said the client needs "only `gymnasium`, `websockets`
-    and `msgpack`", which named three of the nine declared dependencies.
-    `pandas` is declared but is not imported anywhere in `src/` today - it is
-    weight in the install, not a requirement of the runtime.
+    `tyro`, `plugrl-protocol` and `imageio` - and none of them is a deep
+    learning framework, so environments pinned to old `mujoco-py` or
+    `cython<3` can be used with a modern training stack. Recording videos
+    needs the `video` extra, which adds ffmpeg.
+    Correction: this line previously said the client needs "only `gymnasium`,
+    `websockets` and `msgpack`", which named three of the nine then declared.
+    Two of those nine have since left the base install. `pandas` was never
+    imported, and `imageio[ffmpeg]` became `imageio`, with ffmpeg moved to
+    `video`. E45 measured them at 45 MB and 77 MB of the 222 MB install
+    (`plugrl-server/experiments/e45-env-side-footprint/`).
   * **Distributed communication**: `websockets` plus `msgpack` for asynchronous transfer between the server and any number of env clients, across machines.
   * **Modular design**: Separates the environment-side runtime (`plugrl-env-client`) from the shared protocol layer (`plugrl-protocol`).
   * **Command-line interface**: A `tyro`-powered CLI for starting and managing env clients.
@@ -61,26 +62,41 @@ The easiest way to get started is by cloning the repository and using `uv` to ma
     uv sync --extra mujoco
     ```
 
-    All six extras, with `uv`:
+    The six extras do not all go in one environment, because they need
+    different MuJoCo versions:
 
-    ```bash
-    uv sync --extra mujoco --extra robomimic --extra atari --extra classic --extra libero --extra d4rl
-    ```
+    - `mujoco`, `classic`, `atari` and `d4rl` share one, on gymnasium's
+      MuJoCo 3:
 
-    ```
-    pip install -e ".[mujoco, robomimic, atari, classic, libero, d4rl]"
-    ```
+      ```bash
+      uv sync --extra mujoco --extra classic --extra atari --extra d4rl
+      ```
+
+    - `robomimic` and `libero` each need robosuite 1.4.1 with MuJoCo 2.3.7,
+      which is what the LIBERO experiments ran on. Give each an environment
+      of its own, in a separate checkout:
 
-    Correction: these two lines were labelled "Everything" but listed only five
-    of the six extras `pyproject.toml` declares - `d4rl` was missing, while the
-    environment table below names it as the extra for `d4rl-v1`. Against the
-    committed `uv.lock`, all six resolve to 100 packages and the old five to 97
-    (`uv sync --frozen --dry-run`, uv 0.9.8): `robomimic` already carries `d4rl`
-    and `mujoco-py`, so the `d4rl` extra itself adds `gymnasium-robotics`,
-    `mujoco-py-cython3` and `pettingzoo`. Note that "all six" is not a light
-    install - E1 measures the env client with the `robomimic` extra at 7.2G with
+      ```bash
+      uv sync --extra robomimic   # or: uv sync --extra libero
+      ```
+
+    `pyproject.toml` declares `mujoco` in conflict with the other two, so `uv`
+    refuses to combine them rather than resolving one MuJoCo for both. With
+    `pip`, keep the two sets in separate virtual environments.
+
+    Correction: this section used to give one command for all six extras. It
+    could not produce a working environment: robosuite 1.4.1 rejects MuJoCo 3,
+    and the `mujoco` extra needs it. Neither was the lock resolving them
+    correctly. It still held robomimic 0.3.0 after the extra moved to 0.4.0,
+    and resolving it again would have put MuJoCo 2.3.7 under `mujoco` too.
+
+    The `robomimic` extra is not a light install. E1 measures it at 7.2G with
     16 CUDA wheels, because robomimic ships its own policy learning code.
 
+    `video` goes with any of them. The recorder needs it for mp4 output
+    (`--recorder.record-video`, `--recorder.record-full-rollout`); its PNG
+    snapshots need only the base install.
+
     Note: `robomimic` and `libero` pull in `egl-probe`, whose legacy CMake build
     needs `CMAKE_POLICY_VERSION_MINIMUM=3.5` when using CMake 4+. `uv` is
     configured in this repository to apply that automatically. If you install

pyproject.toml

@@ -18,15 +18,22 @@ dependencies = [
     "msgpack>=1.1.1,<2.0.0",
     "tyro>=0.9.31,<0.10.0",
     "plugrl-protocol @ git+https://github.com/PlugRL/plugrl-protocol.git@840296ecbe39dddc6fb9f3ac92479decac0797bd",
-    "pandas>=2.3.3,<3.0.0",
-    "imageio[ffmpeg]>=2.37.2,<3.0.0",
+    # The recorder's PNG snapshots. Its mp4 videos need ffmpeg: the `video`
+    # extra.
+    "imageio>=2.37.2,<3.0.0",
 ]
 
 [project.scripts]
 plugrl-run-env-client = "plugrl_env_client.cli:main"
 plugrl-run-worker = "plugrl_env_client.cli:main"
 
 [project.optional-dependencies]
+# The recorder's videos (--recorder.record-video, --recorder.record-full-rollout).
+# imageio-ffmpeg carries its own ffmpeg binary, 77 MB of an install that
+# otherwise does not need it (E45).
+video = [
+    "imageio[ffmpeg]>=2.37.2,<3.0.0",
+]
 # robomimic 0.3.0, the last on PyPI, imports mujoco_py in env_robosuite, and
 # mujoco_py needs MuJoCo 2.1 binaries and system GL headers; v0.4.0 does not,
 # and supports robosuite 1.2 onwards. robosuite 1.4.1 with mujoco 2.3.7
@@ -56,8 +63,12 @@ d4rl = [
     "patchelf>=0.17.2.4,<0.18.0",
     "gymnasium-robotics[mujoco-py]>=1.4.1,<2.0.0",
 ]
+# LIBERO runs on the same stack as robomimic: robosuite 1.4.1 with MuJoCo
+# 2.3.7, as in the client environment E42 recorded. Unpinned, it resolved
+# MuJoCo 3.6, which robosuite 1.4.1 rejects.
 libero = [
     "libero @ git+https://github.com/CTP314/LIBERO.git@f3bf9428c0d7113b0b72d1462d9552fca35fcf92",
+    "mujoco==2.3.7",
 ]
 
 [dependency-groups]
@@ -82,6 +93,12 @@ testpaths = ["tests"]
 
 [tool.uv]
 extra-build-variables = { "egl-probe" = { CMAKE_POLICY_VERSION_MINIMUM = "3.5" } }
+# robomimic and LIBERO need MuJoCo 2.3.7 and `mujoco` needs MuJoCo 3, so the
+# lock resolves them separately instead of forcing one MuJoCo on all of them.
+conflicts = [
+    [{ extra = "robomimic" }, { extra = "mujoco" }],
+    [{ extra = "libero" }, { extra = "mujoco" }],
+]
 
 [build-system]
 requires = ["setuptools>=77", "wheel"]

src/plugrl_env_client/recorder/writers.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import importlib.util
 from pathlib import Path
 from typing import Any
 
@@ -18,6 +19,26 @@
 )
 
 
+def _ffmpeg_available() -> bool:
+    return importlib.util.find_spec("imageio_ffmpeg") is not None
+
+
+def _video_writer(path: Path, *, fps: float):
+    """An mp4 writer, or an error that says which extra provides one.
+
+    The base install keeps imageio for the PNG snapshots and leaves out
+    imageio-ffmpeg, which carries its own 77 MB ffmpeg binary; videos need
+    the `video` extra.
+    """
+    if not _ffmpeg_available():
+        raise RuntimeError(
+            f"Writing {path.name} needs ffmpeg, which the base install leaves "
+            "out. Install the `video` extra: `uv sync --extra video`, or "
+            "`pip install 'plugrl-env-client[video]'`."
+        )
+    return imageio.get_writer(path, fps=fps)
+
+
 class ObservationArtifactWriter:
     def __init__(self, *, obs_stats_path: Path) -> None:
         self.obs_stats_path = obs_stats_path
@@ -134,7 +155,7 @@ def write_episode_videos(
                 continue
             path = images_dir / f"{key}.mp4"
             path.parent.mkdir(parents=True, exist_ok=True)
-            writer = imageio.get_writer(path, fps=self.video_fps)
+            writer = _video_writer(path, fps=self.video_fps)
             try:
                 for frame in frames:
                     writer.append_data(frame)
@@ -147,7 +168,7 @@ def write_full_rollout_frame(self, event: FullRolloutFrameEvent) -> None:
             if writer is None:
                 path = self.full_videos_dir / f"{key}.mp4"
                 path.parent.mkdir(parents=True, exist_ok=True)
-                writer = imageio.get_writer(path, fps=self.video_fps)
+                writer = _video_writer(path, fps=self.video_fps)
                 self._full_video_writers[key] = writer
             frames = np.asarray(value)
             grid = make_grid(np.asarray([ensure_rgb(frame) for frame in frames]))

tests/test_cli.py

@@ -626,10 +626,27 @@ def fake_get_writer(path, *, fps):
     monkeypatch.setattr(
         "plugrl_env_client.recorder.writers.imageio.get_writer", fake_get_writer
     )
+    monkeypatch.setattr(
+        "plugrl_env_client.recorder.writers._ffmpeg_available", lambda: True
+    )
     writer = VideoArtifactWriter(full_videos_dir=tmp_path / "full", video_fps=12)
     writer.write_episode_videos(
         images_dir=tmp_path / "sampled" / "images",
         image_frames={"cam": [np.zeros((2, 2, 3), dtype=np.uint8)]},
     )
 
     assert calls == [(tmp_path / "sampled" / "images" / "cam.mp4", 12.0)]
+
+
+def test_a_video_without_ffmpeg_names_the_extra(monkeypatch, tmp_path):
+    """The base install leaves imageio-ffmpeg out; the error says what to add."""
+    monkeypatch.setattr(
+        "plugrl_env_client.recorder.writers._ffmpeg_available", lambda: False
+    )
+    writer = VideoArtifactWriter(full_videos_dir=tmp_path / "full", video_fps=12)
+
+    with pytest.raises(RuntimeError, match=r"`video` extra"):
+        writer.write_episode_videos(
+            images_dir=tmp_path / "sampled" / "images",
+            image_frames={"cam": [np.zeros((2, 2, 3), dtype=np.uint8)]},
+        )