Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ src/unilab/assets/motions/x2/*.npz
src/unilab/assets/motions/x2/*.csv

# Robot mesh assets (downloaded from HF at runtime)
src/unilab/assets/bots/
src/unilab/assets/robots/x2/meshes/*.STL
src/unilab/assets/robots/g1/assets/
src/unilab/assets/robots/g1/textures/
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ clean:
find . -type d -name ".ruff_cache" -exec rm -rf {} +
find . -type d -name "htmlcov" -exec rm -rf {} +
find . -type f -name ".coverage" -delete
rm -f train_appo.log train_sac.log train_flashsac.log train_rsl_rl.log MUJOCO_LOG.TXT
rm -f train_appo.log train_td3.log train_sac.log train_flashsac.log train_rsl_rl.log MUJOCO_LOG.TXT
find src/unilab/assets/.cache -type f ! -name '.gitkeep' -delete 2>/dev/null || true
find src/unilab/assets/caches -type f ! -name '.gitkeep' -delete 2>/dev/null || true
find src/unilab/assets/checkpoints -type f ! -name '.gitkeep' -delete 2>/dev/null || true
Expand Down
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,13 @@ and [support matrix](https://unilabsim.github.io/UniLab-doc/en/5-reference/5-sup
## Ecosystem

UniLab is designed to be a shared task and training surface for robot-specific
repositories. Current downstream examples include
[MicroDuck RL](https://github.com/unilabsim/microduck_rl_unilab) and
[EngineAI RL](https://github.com/unilabsim/engineai_rl_unilab). They can ship
robot recipes independently while consuming the same task, backend, and RL
contracts.
repositories. They can ship robot recipes independently while consuming the
same task, backend, and RL contracts. Current downstream examples:

- [MicroDuck RL](https://github.com/unilabsim/microduck_rl_unilab)
- [EngineAI RL](https://github.com/unilabsim/engineai_rl_unilab)
- [Wuji](https://github.com/unilabsim/wuji_unilab)
- [Legged Manipulation](https://github.com/unilabsim/legged-manipulation_unilab)

## Documentation

Expand Down
11 changes: 7 additions & 4 deletions README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,13 @@ uv run train --algo sac --task g1_walk_flat --sim mujoco

## 生态

UniLab 被设计为机器人专属仓库共享的任务与训练界面。目前的下游示例包括
[MicroDuck RL](https://github.com/unilabsim/microduck_rl_unilab) 和
[EngineAI RL](https://github.com/unilabsim/engineai_rl_unilab)。它们可以独立发布机器人
recipe,同时消费同一套 task、backend 和 RL contract。
UniLab 被设计为机器人专属仓库共享的任务与训练界面。下游仓库可以独立发布机器人
recipe,同时消费同一套 task、backend 和 RL contract。目前的下游示例:

- [MicroDuck RL](https://github.com/unilabsim/microduck_rl_unilab)
- [EngineAI RL](https://github.com/unilabsim/engineai_rl_unilab)
- [Wuji](https://github.com/unilabsim/wuji_unilab)
- [Legged Manipulation](https://github.com/unilabsim/legged-manipulation_unilab)

## 文档

Expand Down
41 changes: 41 additions & 0 deletions docs/sphinx/source/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,47 @@ UniLab 遵循[语义化版本](https://semver.org/)。本共享页面以中英

## Unreleased / 未发布

- Update the required `unisim-core` release to `>=1.2.0` and the pinned
`unilab-rl` release to `==1.2.0`, including the ROCm profile.
将必需的 `unisim-core` 版本更新为 `>=1.2.0`,并将钉定的 `unilab-rl` 版本更新为
`==1.2.0`,ROCm 配置档同步更新。

- Host the FR3 SuperDex native bot assets (collision SDF, render GLB, license)
on the Hugging Face dataset
[unilabsim/unilab-robots](https://huggingface.co/datasets/unilabsim/unilab-robots),
consistent with the other robot mesh assets. The asset hub downloads the
`bots/arms/fr3_v2` snapshot into `src/unilab/assets/` on first use, and
`uv run unilab-pull-assets --robot fr3_v2` pre-fetches it;
`SUPERDEX_ASSETS_PATH` / `env.superdex_assets_root` remain as overrides for
auditing a local `project_superdex` checkout and take precedence without
downloading.
FR3 SuperDex 原生 bot 资产(collision SDF、render GLB、许可证)改为托管在
Hugging Face 数据集
[unilabsim/unilab-robots](https://huggingface.co/datasets/unilabsim/unilab-robots),
与其他机器人 mesh 资产的处理方式一致。asset hub 首次使用时自动把
`bots/arms/fr3_v2` 快照下载到 `src/unilab/assets/`,也可用
`uv run unilab-pull-assets --robot fr3_v2` 预拉取;
`SUPERDEX_ASSETS_PATH` / `env.superdex_assets_root` 保留为审计本地
`project_superdex` checkout 的覆盖方式,优先级更高且不会触发下载。

- Add a `superdex` optional extra (`uv sync --extra superdex` or
`pip install "unilab[superdex]"`) that pulls the published SuperDex
Physics/Robotics 1.0.0 wheels (CPython 3.12/3.13, Linux x86_64) through
`unisim-core[superdex]`, replacing the temporary source-build setup. The
`sim=mujoco` runtime check now guards on the `mujoco_uni` binding, since
plain `mujoco` also arrives with the superdex extra.
SuperDex native interactive playback crashing on the first rendered frame
(`eval --sim superdex --render-mode interactive`) was first fixed in
`unisim-core` 1.1.6; the required release line is now `>=1.2.0`.
新增 `superdex` optional extra(`uv sync --extra superdex` 或
`pip install "unilab[superdex]"`),通过 `unisim-core[superdex]`
安装已发布的 SuperDex Physics/Robotics 1.0.0 wheel(CPython 3.12/3.13、
Linux x86_64),取代临时的源码编译安装方式。`sim=mujoco` 的运行时检查
改为检查 `mujoco_uni` 绑定,因为普通 `mujoco` 包也会随 superdex extra
装入。SuperDex 原生 interactive 回放首帧渲染崩溃的问题
(`eval --sim superdex --render-mode interactive`)最早在 `unisim-core`
1.1.6 修复;当前必需的版本线为 `>=1.2.0`。

- Go2 arm manipulation/locomotion, its legacy helpers, assets, tools and HIM-PPO
owners moved to [legged-manipulation_unilab](https://github.com/unilabsim/legged-manipulation_unilab)
under [#1528](https://github.com/unilabsim/UniLab/issues/1528). Dedicated source
Expand Down
2 changes: 2 additions & 0 deletions docs/sphinx/source/en/1-getting_started/2-installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,7 @@ uv sync --extra mujoco --extra mjwarp --extra newton
| MJWarp | `uv sync --extra mujoco --extra mjwarp` | NVIDIA CUDA and an explicit CUDA process device |
| Genesis | `uv sync --extra genesis` | The validated path uses Linux x86_64, an NVIDIA GPU, and the pinned torch/Genesis versions |
| Newton | `uv sync --extra newton` | NVIDIA CUDA; can be combined with the `mujoco` / `mjwarp` extras in one environment |
| SuperDex | `uv sync --extra superdex` | Published wheels support Linux x86_64 with CPython 3.12/3.13 only; FR3 assets download from Hugging Face on first use (`SUPERDEX_ASSETS_PATH` overrides with a local checkout) |
| Drake | `make setup-drake` | C++20, Eigen/fmt/spdlog, and an existing Drake prefix or the script's download path |
| IsaacGym | `bash scripts/tools/setup_isaacgym_env.sh` | Linux x86_64, NVIDIA driver, and a separate Python 3.8 worker environment |
| IsaacSim | `bash scripts/tools/setup_isaacsim_env.sh` | Linux x86_64, NVIDIA CUDA, a separate Python 3.11 worker, and Kit EULA acceptance |
Expand All @@ -198,6 +199,7 @@ runtime variables, renderer requirements, and verification commands:
- {doc}`Genesis <../2-user_guide/3-backends/5-genesis>`
- {doc}`Drake <../2-user_guide/3-backends/6-drake>`
- {doc}`Newton <../2-user_guide/3-backends/7-newton>`
- {doc}`SuperDex <../2-user_guide/3-backends/8-superdex>`
- {doc}`IsaacGym <../2-user_guide/3-backends/3-isaacgym>`
- {doc}`IsaacSim <../2-user_guide/3-backends/4-isaacsim>`

Expand Down
118 changes: 80 additions & 38 deletions docs/sphinx/source/en/2-user_guide/3-backends/8-superdex.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,52 +9,56 @@ training checks do not establish full-training performance or platform support.
The implementation is tracked in [#1534](https://github.com/Motphys/UniLab/issues/1534)
under [roadmap #1533](https://github.com/Motphys/UniLab/issues/1533).

## Local Development Setup
## Installation

This development profile uses locally linked UniSim and UniLab checkouts; it
does not require a new published version. SuperDex Physics/Robotics 1.0.0 requires
Python 3.12. CPU physics does not require CUDA. Native rendering and video are
not part of the FR3 owner; training defaults to `no_play=true`.

In the UniLab checkout, use an existing Python 3.12 virtual environment or create
one, then install the local packages:

For the temporary local SuperDex source build used by this roadmap, first run:
SuperDex Physics/Robotics 1.0.0 is published as Python wheels and is an
optional UniLab extra; no native source build is required. The wheels carry
the native batch executor and support CPython 3.12/3.13 on Linux x86_64 only;
on other platforms the extra is empty and the CLI reports a targeted runtime
diagnostic. CPU physics does not require CUDA. The FR3 owner has no record
(video) playback — the `.superdex_bot` asset carries no MJCF visual model — so
play defaults to the native interactive viewer (`play_render_mode=interactive`
with `play_env_num=1`); use `training.play_render_mode=none` for headless runs.

```bash
bash scripts/tools/setup_superdex_env.sh
source ~/.cache/unisim/superdex/env.sh
# Source checkout (default Python 3.13; wheels support CPython 3.12/3.13):
uv sync --extra superdex

# From PyPI:
pip install "unilab[superdex]"
```

With no arguments the script clones the modified
`unilabsim/project_superdex` integration branch, installs the public SuperDex
Python facade, builds `mochi_physics_pybind` and `superdex_robotics_pybind` in
Release mode, and installs the local UniSim/UniRL/UniLab checkouts editable. It
does not publish or install a SuperDex wheel from PyPI. The generated
`env.sh` exports the assets and native-extension paths. Re-run the script after
changing the SuperDex checkout; CMake reuses the existing build directory.
The extra delegates version pins to UniSim through `unisim-core[superdex]`.
UniSim's superdex extra already carries the plain `mujoco` package (used for
MJCF conversion and the offline playback renderer); `mujoco-uni-runtime` is
not required — only the MuJoCo physics backend needs it. The current wheels
are a temporary unilabsim build (`superdex-physics-uni` /
`superdex-robotics-uni`); once the upstream project_superdex release publishes
the `superdex-physics` / `superdex-robotics` wheels, UniSim switches the
package names and UniLab needs no change.

Native FR3 assets are hosted on Hugging Face
([unilabsim/unilab-robots](https://huggingface.co/datasets/unilabsim/unilab-robots)),
like the other robot mesh assets; the wheels do not carry robot binaries. The
asset hub registers `bots/arms/fr3_v2/fr3_v2.superdex_bot` and downloads the
snapshot into `src/unilab/assets/` on first use, checking its collision SDF,
render files, `LICENSE` and `NOTICE` before constructing physics. To pre-fetch
the assets (e.g. for CI or offline prep):

```bash
uv venv --python 3.12
export UNILAB_LOCAL_UNISIM=/absolute/path/to/unisim
uv pip install -e "${UNILAB_LOCAL_UNISIM}[superdex,mujoco]" -e . --group pyproject.toml:dev
export UV_NO_SYNC=1
export SUPERDEX_ASSETS_PATH=/absolute/path/to/project_superdex/assets
uv run unilab-pull-assets --robot fr3_v2
```

`UNILAB_LOCAL_UNISIM` enables the repository's local dependency validation:
tests require an editable installation and check that its metadata and imported
module point to exactly that checkout. Without this variable, the normal
indexed-release requirement remains in force. `UV_NO_SYNC=1` preserves the local
links when running existing Make targets; `uv sync` would re-resolve the locked
release profile.
To audit a local `project_superdex` checkout instead, set
`SUPERDEX_ASSETS_PATH=/absolute/path/to/project_superdex/assets`, or set
`env.superdex_assets_root=/absolute/path/to/project_superdex/assets` to
override it for a specific owner invocation. An explicit root takes precedence
over the Hugging Face download and never downloads.

Native FR3 assets remain in the upstream SuperDex checkout. The asset hub
registers `bots/arms/fr3_v2/fr3_v2.superdex_bot`, checking its collision SDF,
render files, `LICENSE` and `NOTICE` before constructing physics. No native
robot binaries are bundled or downloaded by UniLab. Set
`env.superdex_assets_root=/absolute/path/to/project_superdex/assets` to override
`SUPERDEX_ASSETS_PATH` for a specific owner invocation.
A local source build is only needed when changing the SuperDex engine itself:
`bash scripts/tools/setup_superdex_env.sh` clones the integration branch,
builds the native extensions and links the local UniSim/UniLab checkouts
editable. Regular use does not need it.

## Run the FR3 Task

Expand All @@ -70,9 +74,46 @@ research profile, not rated hardware limits. `superdex_effort_limits` declares
the same bounds at the native backend boundary. The SDK remains single-threaded;
the native scene executor below owns all supported CPU parallelism.

## Run the Go2 Task

The `go2_joystick_flat/superdex` owner trains and evaluates the Go2 quadruped
on SuperDex. It inherits the MuJoCo owner's policy I/O (49 actor observations,
52 critic observations, 12 position-target actions) and control timing,
declares its own command ranges and reward weights, and explicitly opts into
contact approximation; see "Validation and Ownership" below for the contract
details.

Train directly on SuperDex (CPU physics, automatic native workers):

```bash
uv run train --algo ppo --task go2_joystick_flat --sim superdex
```

The owner defaults to 1024 environments and 400 iterations. Logs and
checkpoints land in `logs/rsl_rl_ppo/Go2JoystickFlat/<timestamp>_superdex/`.

Evaluate a trained run with `--load-run`. Record playback is the default: it
steps 200 frames across 16 environments and writes `play_video.mp4` into the
run directory through the offline MuJoCo renderer while SuperDex remains the
physics backend:

```bash
uv run eval --algo ppo --task go2_joystick_flat --sim superdex \
--load-run 2026-09-10_10-50-00_superdex
```

For the native SuperDex (Polyscope) viewer instead of a video, add
`--render-mode interactive`; the CLI forces `training.play_env_num=1` and the
owner layer switches the env to the serial executor for that run.

A checkpoint trained on MuJoCo can be evaluated on SuperDex directly
(sim2sim): pass its run directory to `--load-run`. The play entrypoint
validates the source `run_config.json` against the sim2sim contract before
constructing the environment and rejects incompatible policy I/O.

## Default CPU Environment Parallelism

The backend uses SuperDex's source-built `SceneBatchExecutor`, a persistent C++
The backend uses the `SceneBatchExecutor` shipped in the SuperDex wheel, a persistent C++
thread pool that owns the barrier across independent scenes. Each substep writes
batched generalized forces, advances scenes, and returns articulated/link state,
contact sensors and solver status without per-environment Python binding calls.
Expand Down Expand Up @@ -172,7 +213,8 @@ uv run --no-sync pytest tests/assets/test_superdex_assets.py \
tests/envs/test_fr3_superdex.py tests/test_cli_runtime_requirements.py -q
```

The optional native tests require the SDK and `SUPERDEX_ASSETS_PATH`; they cover
The optional native tests require the SDK and the FR3 assets (downloaded from
Hugging Face on demand, or provided through `SUPERDEX_ASSETS_PATH`); they cover
finite rollout data, selected reset isolation, immediate observation refresh
and a spawned `EnvFactory`. Missing runtime/assets produce an explicit skip;
such a run is not native validation. The base asset/config tests need no native
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,6 +165,7 @@ uv sync --extra mujoco --extra mjwarp --extra newton
| MJWarp | `uv sync --extra mujoco --extra mjwarp` | NVIDIA CUDA 和显式 CUDA process device |
| Genesis | `uv sync --extra genesis` | 已验证路径使用 Linux x86_64、NVIDIA GPU 及固定版本 torch/Genesis |
| Newton | `uv sync --extra newton` | NVIDIA CUDA;可与 `mujoco` / `mjwarp` extra 组合进同一环境 |
| SuperDex | `uv sync --extra superdex` | 已发布 wheel 仅支持 Linux x86_64、CPython 3.12/3.13;FR3 资产首次使用时自动从 Hugging Face 下载(`SUPERDEX_ASSETS_PATH` 可指定本地 checkout 覆盖) |
| Drake | `make setup-drake` | C++20、Eigen/fmt/spdlog,以及已有 Drake prefix 或脚本下载路径 |
| IsaacGym | `bash scripts/tools/setup_isaacgym_env.sh` | Linux x86_64、NVIDIA driver 和独立 Python 3.8 worker 环境 |
| IsaacSim | `bash scripts/tools/setup_isaacsim_env.sh` | Linux x86_64、NVIDIA CUDA、独立 Python 3.11 worker 和 Kit EULA 接受 |
Expand All @@ -179,6 +180,7 @@ Drake、IsaacGym 和 IsaacSim 的 setup 脚本会将外部 runtime 安装到仓
- {doc}`Genesis <../2-user_guide/3-backends/5-genesis>`
- {doc}`Drake <../2-user_guide/3-backends/6-drake>`
- {doc}`Newton <../2-user_guide/3-backends/7-newton>`
- {doc}`SuperDex <../2-user_guide/3-backends/8-superdex>`
- {doc}`IsaacGym <../2-user_guide/3-backends/3-isaacgym>`
- {doc}`IsaacSim <../2-user_guide/3-backends/4-isaacsim>`

Expand Down
Loading
Loading