diff --git a/docs/index.rst b/docs/index.rst index d3218514afb7..885bc7640cc7 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -142,7 +142,6 @@ Table of Contents :caption: Experimental Features source/experimental-features/bleeding-edge - source/experimental-features/newton-physics-integration/index source/experimental-features/visuo_tactile_sensor .. toctree:: diff --git a/docs/source/experimental-features/newton-physics-integration/index.rst b/docs/source/experimental-features/newton-physics-integration/index.rst deleted file mode 100644 index b93c5a3fd2c0..000000000000 --- a/docs/source/experimental-features/newton-physics-integration/index.rst +++ /dev/null @@ -1,45 +0,0 @@ -Newton Physics Integration -=========================== - -`Newton `_ is a GPU-accelerated, extensible, and differentiable physics simulation engine designed for robotics, research, -and advanced simulation workflows. Built on top of `NVIDIA Warp `_ and integrating MuJoCo Warp, Newton provides high-performance -simulation, modern Python APIs, and a flexible architecture for both users and developers. - -Newton is an Open Source community-driven project with contributions from NVIDIA, Google Deep Mind, and Disney Research, -managed through the Linux Foundation. - -This integration is available on the `develop branch `_ of Isaac Lab as part of Isaac Lab 3.0 Beta, and is -under active development. Many features are not yet supported, and only a limited set of classic RL and flat terrain locomotion -reinforcement learning examples are included at the moment. - -Both this Isaac Lab integration and Newton itself are under heavy development. We intend to support additional -features for other reinforcement learning and imitation learning workflows in the future, but the above tasks should be -a good lens through which to understand how Newton integration works in Isaac Lab. - -We have validated Newton simulation against PhysX by transferring learned policies from Newton to PhysX and vice versa -Furthermore, we have also successfully deployed a Newton-trained locomotion policy to a G1 robot. - -Newton can support `multiple solvers `_ for handling different types of physics simulation, but for the moment, the Isaac -Lab integration focuses primarily on the MuJoCo-Warp solver. - -Future updates of Isaac Lab and Newton should include both ongoing improvements in performance as well as integration -with additional solvers. - -During the development phase of both Newton and this Isaac Lab integration, you are likely to encounter breaking -changes as well as limited documentation. We do not expect to be able to provide official support or debugging assistance -until the framework has reached an official release. We appreciate your understanding and patience as we work to deliver a robust and polished framework! - -For an overview of how the multi-backend architecture works, including how to add a new backend, see -:doc:`/source/overview/core-concepts/multi_backend_architecture`. - - -.. toctree:: - :maxdepth: 2 - :titlesonly: - - installation - warp-environments - warp-env-migration - limitations-and-known-bugs - solver-transitioning - using-kamino diff --git a/docs/source/experimental-features/newton-physics-integration/limitations-and-known-bugs.rst b/docs/source/experimental-features/newton-physics-integration/limitations-and-known-bugs.rst deleted file mode 100644 index e5eab3996d8a..000000000000 --- a/docs/source/experimental-features/newton-physics-integration/limitations-and-known-bugs.rst +++ /dev/null @@ -1,55 +0,0 @@ -Limitations -=========== - -During the early development phase of both Newton and this Isaac Lab integration, -you are likely to encounter breaking changes as well as limited documentation. - -We do not expect to be able to provide support or debugging assistance until the framework has reached an official release. - -Here is a non-exhaustive list of capabilities currently supported in the Newton experimental feature branch grouped by extension: - -* isaaclab: - * Articulation API (supports both articulations and single-body articulations as rigid bodies) - * Contact Sensor - * Direct & Manager single agent workflows - * Omniverse Kit visualizer - * Newton visualizer -* isaaclab_assets: - * Quadrupeds - * Anymal-B, Anymal-C, Anymal-D - * Unitree A1, Go1, Go2 - * Spot - * Humanoids - * Unitree H1 & G1 - * Cassie - * Arms and Hands - * Franka - * UR10 - * Allegro Hand - * Toy examples - * Cartpole - * Ant - * Humanoid -* isaaclab_tasks: - * Direct: - * Cartpole (State, RGB, Depth) - * Ant - * Humanoid - * Allegro Hand Repose Cube - * Manager based: - * Cartpole (State) - * Ant - * Humanoid - * Locomotion (velocity flat terrain) - * Anymal-B - * Anymal-C - * Anymal-D - * Cassie - * A1 - * Go1 - * Go2 - * Unitree G1 - * Unitree H1 - * Manipulation reach - * Franka - * UR10 diff --git a/docs/source/features/visualization.rst b/docs/source/features/visualization.rst index 26e9bec98414..c37fb6d9be96 100644 --- a/docs/source/features/visualization.rst +++ b/docs/source/features/visualization.rst @@ -549,5 +549,5 @@ See Also - :doc:`/source/overview/core-concepts/renderers` — renderer backends (RTX, Newton Warp, OVRTX) - :doc:`/source/overview/core-concepts/scene_data_providers` — how scene data flows from physics to visualizers -- :doc:`/source/experimental-features/newton-physics-integration/index` — Newton physics integration guide +- :doc:`/source/overview/core-concepts/physical-backends/newton/index` — Newton backend guide - :doc:`/source/migration/migrating_to_isaaclab_3-0` — migration guide with ``--headless`` deprecation details diff --git a/docs/source/overview/core-concepts/index.rst b/docs/source/overview/core-concepts/index.rst index 052d7080eb67..9e29cc22bed2 100644 --- a/docs/source/overview/core-concepts/index.rst +++ b/docs/source/overview/core-concepts/index.rst @@ -8,6 +8,7 @@ This section we introduce core concepts in Isaac Lab. multi_backend_architecture + physical-backends/index schema_cfgs task_workflows actuators diff --git a/docs/source/overview/core-concepts/multi_backend_architecture.rst b/docs/source/overview/core-concepts/multi_backend_architecture.rst index 352245c76ac7..5d894c6820f8 100644 --- a/docs/source/overview/core-concepts/multi_backend_architecture.rst +++ b/docs/source/overview/core-concepts/multi_backend_architecture.rst @@ -382,6 +382,6 @@ See Also - :doc:`/source/migration/migrating_to_isaaclab_3-0` — migration guide from Isaac Lab 2.x to the multi-backend architecture - :doc:`/source/features/hydra` — preset system for multi-backend environment configurations -- :doc:`/source/experimental-features/newton-physics-integration/index` — Newton physics integration - guide +- :doc:`physical-backends/index` — feature matrix and per-backend guides (PhysX, Newton, OvPhysX) +- :doc:`physical-backends/newton/index` — Newton backend guide - :doc:`renderers` — renderer backend architecture diff --git a/docs/source/overview/core-concepts/physical-backends/index.rst b/docs/source/overview/core-concepts/physical-backends/index.rst new file mode 100644 index 000000000000..baf224a8c4ac --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/index.rst @@ -0,0 +1,138 @@ +Physics Backends +================ + +Isaac Lab 3.0 supports multiple physics backends through a unified API. Each backend +exposes the same :class:`~isaaclab.assets.Articulation`, +:class:`~isaaclab.assets.RigidObject`, sensor and renderer surfaces, while differing +in solver characteristics, maturity, and feature coverage. See +:doc:`../multi_backend_architecture` for how the dispatch and factory machinery work +under the hood. + +This page summarizes what each backend supports today; the sub-pages document +backend-specific configuration, installation, and limitations. + +.. toctree:: + :maxdepth: 2 + + physx/index + newton/index + ovphysx/index + solver-comparison + + +Choosing a Backend +------------------ + +* **PhysX** — the historical default. Production-ready, broad coverage of Isaac Lab + features, and the reference for behavior parity. Selected via + :class:`~isaaclab_physx.physics.PhysxCfg`. +* **Newton** — GPU-accelerated, Warp-native, and differentiable. The Newton + integration ships with the MuJoCo-Warp solver and beta support for the Kamino + solver. Selected via :class:`~isaaclab_newton.physics.NewtonCfg`. +* **OvPhysX** — a **highly experimental** kit-less PhysX backend that reads + scene-level parameters from the USD ``PhysicsScene`` prim. Selected via + :class:`~isaaclab_ovphysx.physics.OvPhysxCfg`. Not recommended for general use yet. + +The active backend is selected at simulation construction time and applies to every +asset, sensor, and renderer instantiated thereafter: + +.. code-block:: python + + from isaaclab.sim import SimulationCfg + from isaaclab_physx.physics import PhysxCfg + from isaaclab_newton.physics import NewtonCfg, MJWarpSolverCfg + + # PhysX (default) + sim_cfg = SimulationCfg(physics=PhysxCfg()) + + # Newton with MuJoCo-Warp + sim_cfg = SimulationCfg(physics=NewtonCfg(solver_cfg=MJWarpSolverCfg())) + + +Feature Support Matrix +---------------------- + +The matrix below is intentionally coarse-grained. For exhaustive per-asset and +per-task support, see each backend's own ``limitations`` page. + +.. list-table:: + :header-rows: 1 + :widths: 30 20 30 20 + + * - Feature + - PhysX + - Newton + - OvPhysX + * - Maturity + - Stable + - Beta + - Highly experimental + * - Default solver + - TGS (rigid body) + - MuJoCo-Warp + - PhysX (TGS / PGS via USD) + * - Alternative solvers + - PGS + - Kamino (beta), additional Newton solvers planned + - — + * - Differentiable + - No + - Yes (via Warp) + - No + * - Articulation API + - Yes + - Yes + - In-flight (PR #5459) + * - Rigid Object API + - Yes + - Yes + - Yes + * - Contact Sensor + - Yes + - Yes + - In-flight (PR #5422) + * - IMU + - Yes + - Yes + - In-flight (PR #5421) + * - Frame Transformer / Ray Caster / PVA / Joint-Wrench Sensor + - Yes + - Yes + - Not yet + * - Camera / Tiled Rendering + - Yes (RTX) + - Yes (Newton-Warp renderer) + - Not yet + * - Requires Isaac Sim + - Yes + - Optional (only for the Omniverse visualizer) + - Yes + * - Solver configuration source + - :class:`~isaaclab_physx.physics.PhysxCfg` + - :class:`~isaaclab_newton.physics.NewtonCfg` + solver config + - USD ``PhysicsScene`` + :class:`~isaaclab_ovphysx.physics.OvPhysxCfg` + + +Selecting Backends per Task +--------------------------- + +Tasks that support more than one backend define a Hydra preset on +``SimulationCfg.physics``. The example below shows the cartpole task config which +declares all three backends side by side: + +.. code-block:: python + + from isaaclab_physx.physics import PhysxCfg + from isaaclab_newton.physics import MJWarpSolverCfg, NewtonCfg + from isaaclab_ovphysx.physics import OvPhysxCfg + + @configclass + class CartpolePhysicsCfg(PresetCfg): + default: PhysxCfg = PhysxCfg() + physx: PhysxCfg = PhysxCfg() + newton_mjwarp: NewtonCfg = NewtonCfg(solver_cfg=MJWarpSolverCfg()) + ovphysx: OvPhysxCfg = OvPhysxCfg() + +Users then select the backend at the command line via ``presets=`` or by +overriding the physics field directly. See :ref:`hydra-backend-solver-presets` for +the full Hydra interaction. diff --git a/docs/source/overview/core-concepts/physical-backends/newton/index.rst b/docs/source/overview/core-concepts/physical-backends/newton/index.rst new file mode 100644 index 000000000000..087adda9cb23 --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/newton/index.rst @@ -0,0 +1,45 @@ +Newton Backend +============== + +`Newton `_ is a +GPU-accelerated, extensible, and differentiable physics simulation engine designed +for robotics, research, and advanced simulation workflows. Built on top of +`NVIDIA Warp `_ and integrating MuJoCo Warp, Newton +provides high-performance simulation, modern Python APIs, and a flexible +architecture for both users and developers. + +Newton is an Open Source community-driven project with contributions from NVIDIA, +Google Deep Mind, and Disney Research, managed through the Linux Foundation. + +Newton support in Isaac Lab is in beta and under active development. Many features +are still maturing, and the Isaac Lab integration ships a focused, validated set of +classic RL and flat-terrain locomotion environments. We have validated Newton +simulation against PhysX by transferring learned policies in both directions and +have successfully deployed a Newton-trained locomotion policy to a G1 robot. + +Newton can support `multiple solvers +`_ for +handling different types of physics simulation. The Isaac Lab integration ships +two solver pages: + +* :doc:`mjwarp-solver` — the primary, validated solver path. +* :doc:`kamino-solver` — beta support on selected classic tasks. + +During the beta phase, breaking changes and incomplete documentation are still +expected. Official support and debugging assistance will follow once the framework +reaches an official release. + +For an overview of how the multi-backend architecture works, including how to add a +new backend, see :doc:`../../multi_backend_architecture`. + + +.. toctree:: + :maxdepth: 2 + :titlesonly: + + installation + supported-features + mjwarp-solver + kamino-solver + warp-environments + warp-env-migration diff --git a/docs/source/experimental-features/newton-physics-integration/installation.rst b/docs/source/overview/core-concepts/physical-backends/newton/installation.rst similarity index 97% rename from docs/source/experimental-features/newton-physics-integration/installation.rst rename to docs/source/overview/core-concepts/physical-backends/newton/installation.rst index 330ade886c3e..846472b6dbfe 100644 --- a/docs/source/experimental-features/newton-physics-integration/installation.rst +++ b/docs/source/overview/core-concepts/physical-backends/newton/installation.rst @@ -1,7 +1,7 @@ Installation ============ -Installing the Newton physics integration requires three things: +Installing the Newton backend requires three things: 1) The ``develop`` branch of Isaac Lab 2) Ubuntu 22.04 or 24.04 diff --git a/docs/source/experimental-features/newton-physics-integration/using-kamino.rst b/docs/source/overview/core-concepts/physical-backends/newton/kamino-solver.rst similarity index 98% rename from docs/source/experimental-features/newton-physics-integration/using-kamino.rst rename to docs/source/overview/core-concepts/physical-backends/newton/kamino-solver.rst index 7c8b6f2d564c..b6d428d13d3c 100644 --- a/docs/source/experimental-features/newton-physics-integration/using-kamino.rst +++ b/docs/source/overview/core-concepts/physical-backends/newton/kamino-solver.rst @@ -1,7 +1,7 @@ -.. _newton-using-kamino: +.. _newton-kamino-solver: -Using the Kamino Solver -======================= +Kamino Solver +============= Kamino is a Newton solver, not a separate Isaac Lab physics backend. In Isaac Lab, Kamino is enabled by selecting a :class:`~isaaclab_newton.physics.NewtonCfg` whose @@ -49,7 +49,7 @@ solver config types used by the presets: Then add a ``newton_kamino`` entry beside the existing ``default``, ``physx``, and ``newton_mjwarp`` entries: -.. literalinclude:: ../../../../source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_env_cfg.py +.. literalinclude:: ../../../../../../source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_env_cfg.py :language: python :start-at: class CartpolePhysicsCfg :end-at: ovphysx: OvPhysxCfg = OvPhysxCfg() diff --git a/docs/source/experimental-features/newton-physics-integration/solver-transitioning.rst b/docs/source/overview/core-concepts/physical-backends/newton/mjwarp-solver.rst similarity index 88% rename from docs/source/experimental-features/newton-physics-integration/solver-transitioning.rst rename to docs/source/overview/core-concepts/physical-backends/newton/mjwarp-solver.rst index 5d78f44de503..805f51a7ec2d 100644 --- a/docs/source/experimental-features/newton-physics-integration/solver-transitioning.rst +++ b/docs/source/overview/core-concepts/physical-backends/newton/mjwarp-solver.rst @@ -1,11 +1,13 @@ -Solver Transitioning -==================== - -Transitioning to the Newton physics engine introduces new physics solvers that handle simulation using different numerical approaches. -While Newton supports several different solvers, our initial focus for Isaac Lab is on using the -MuJoCo-Warp solver from Google DeepMind. Isaac Lab also includes beta support for the Kamino -solver on selected classic tasks. Kamino is selected through a physics preset rather than as a -separate backend; see :ref:`hydra-backend-solver-presets` and :ref:`newton-using-kamino`. +MJWarp Solver +============= + +The MuJoCo-Warp solver from Google DeepMind is the primary, validated solver +for the Newton backend in Isaac Lab. It is enabled by setting +:attr:`~isaaclab_newton.physics.NewtonCfg.solver_cfg` to a +:class:`~isaaclab_newton.physics.MJWarpSolverCfg`, usually exposed as the +``newton_mjwarp`` physics preset on a task configuration. Newton ships with +beta support for an alternative Kamino solver — see :doc:`kamino-solver` and +:ref:`hydra-backend-solver-presets` for how presets are selected. .. note:: diff --git a/docs/source/overview/core-concepts/physical-backends/newton/supported-features.rst b/docs/source/overview/core-concepts/physical-backends/newton/supported-features.rst new file mode 100644 index 000000000000..d7fd6aabf0d7 --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/newton/supported-features.rst @@ -0,0 +1,94 @@ +Supported Features +================== + +The Newton backend is in beta. Breaking changes and incomplete documentation are +still expected, and official support or debugging assistance will only be +available once the integration reaches an official release. + + +Discovering Newton-Supported Tasks +---------------------------------- + +A task supports the Newton backend when its physics ``PresetCfg`` declares a +``newton_mjwarp`` (or ``newton_kamino``) entry. To list every task that +currently supports Newton: + +.. code-block:: bash + + grep -rln "newton_mjwarp" source/isaaclab_tasks/ + +Passing ``presets=newton_mjwarp`` to a task without that preset will raise an +error at launch. The :doc:`mjwarp-solver` page covers how to add a Newton +preset to your own task. + + +Supported APIs +-------------- + +The following capabilities are covered by the Newton backend on ``develop`` at +the time of writing. The list is non-exhaustive and continues to grow. + +isaaclab +^^^^^^^^ + +* Articulation API (multi-link and single-body articulations) +* Rigid Object and Rigid Object Collection APIs +* Sensors: Contact Sensor, IMU, Frame Transformer, Joint Wrench, PVA +* Direct and Manager-based single-agent workflows +* Omniverse Kit visualizer (when Isaac Sim is installed) +* Newton-Warp visualizer (kit-less) +* Tiled rendering via the Newton-Warp renderer + +The following sensors are backend-agnostic (implemented in ``isaaclab`` core) +and work transparently with Newton: + +* Ray Caster +* Camera — see :doc:`../../sensors/camera` + +isaaclab_assets +^^^^^^^^^^^^^^^ + +* Quadrupeds: Anymal-B, Anymal-C, Anymal-D, Unitree A1, Unitree Go1, Unitree + Go2, Spot +* Humanoids: Unitree H1, Unitree G1, Cassie +* Arms and hands: Franka, UR10, Allegro Hand, Shadow Hand +* Toy examples: Cartpole, Ant, Humanoid + +isaaclab_tasks +^^^^^^^^^^^^^^ + +Direct workflows: + +* Cartpole (state, RGB, depth) +* Ant, Humanoid +* Allegro Hand Repose Cube, Shadow Hand, Shadow Hand Over +* Locomotion (shared base env) + +Manager-based workflows: + +* Classic: Cartpole, Ant, Humanoid +* Locomotion velocity, flat terrain: A1, Anymal-B, Anymal-C, Anymal-D, Cassie, + Unitree G1, Go1, Go2, Unitree H1, Spot +* Locomotion velocity, rough terrain: Anymal-C, Cassie, Go1, Go2 +* Manipulation: reach (Franka, UR10), cabinet, dexsuite + + +Solver Coverage +--------------- + +* **MuJoCo-Warp solver**: the primary, validated path for every supported task. +* **Kamino solver**: beta. Currently validated on ``Isaac-Cartpole-Direct-v0``, + ``Isaac-Ant-Direct-v0``, ``Isaac-Cartpole-v0``, and ``Isaac-Ant-v0``. See + :doc:`kamino-solver`. + +Other Newton solvers (e.g. VBD) are not yet exposed through Isaac Lab. + + +Known Gaps +---------- + +* Soft bodies, particles, and other non-rigid PhysX features are not yet + available through Newton. +* Behaviour on stiff contact stacks can diverge from PhysX; expect to retune + contact and substep parameters when porting tasks across backends. +* Multi-agent and self-play workflows are not yet wired up for Newton. diff --git a/docs/source/experimental-features/newton-physics-integration/warp-env-migration.rst b/docs/source/overview/core-concepts/physical-backends/newton/warp-env-migration.rst similarity index 100% rename from docs/source/experimental-features/newton-physics-integration/warp-env-migration.rst rename to docs/source/overview/core-concepts/physical-backends/newton/warp-env-migration.rst diff --git a/docs/source/experimental-features/newton-physics-integration/warp-environments.rst b/docs/source/overview/core-concepts/physical-backends/newton/warp-environments.rst similarity index 99% rename from docs/source/experimental-features/newton-physics-integration/warp-environments.rst rename to docs/source/overview/core-concepts/physical-backends/newton/warp-environments.rst index 6af7d934dafe..a5dcbd648724 100644 --- a/docs/source/experimental-features/newton-physics-integration/warp-environments.rst +++ b/docs/source/overview/core-concepts/physical-backends/newton/warp-environments.rst @@ -220,7 +220,7 @@ Limitations ~~~~~~~~~~~ The warp env path is experimental and has the following known constraints. These are -specific to warp envs; for Newton physics limitations see :doc:`limitations-and-known-bugs`. +specific to warp envs; for Newton physics limitations see :doc:`supported-features`. **Physics backend** diff --git a/docs/source/overview/core-concepts/physical-backends/ovphysx/index.rst b/docs/source/overview/core-concepts/physical-backends/ovphysx/index.rst new file mode 100644 index 000000000000..b136dafd681f --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/ovphysx/index.rst @@ -0,0 +1,68 @@ +OvPhysX Backend +=============== + +.. warning:: + + OvPhysX is **highly experimental** and is not recommended for general use yet. + The public surface is changing rapidly while the backend is under active + development. This page is a placeholder and will be expanded once the + in-flight integration work lands on ``develop``. + +OvPhysX is a kit-less variant of the PhysX backend. It drives PhysX directly +(without the Omniverse Kit runtime) and reads scene-level solver parameters +from the USD ``PhysicsScene`` prim rather than from a Python config. The Python +config :class:`~isaaclab_ovphysx.physics.OvPhysxCfg` only exposes the handful of +GPU buffer sizes that are not represented on the USD schema. + +OvPhysX is selected through :class:`~isaaclab_ovphysx.physics.OvPhysxCfg`: + +.. code-block:: python + + from isaaclab.sim import SimulationCfg + from isaaclab_ovphysx.physics import OvPhysxCfg + + sim_cfg = SimulationCfg(physics=OvPhysxCfg()) + +Why use OvPhysX? +---------------- + +* **Kit-less execution.** OvPhysX avoids Omniverse Kit, which makes it a useful + experimental path for headless deployments and for backends that don't need + the Kit runtime stack. +* **USD-as-source-of-truth.** Solver parameters are taken from the + ``PhysicsScene`` USD prim, so authoring tools that already manage USD scenes + do not need a parallel Python config. + +What works today +---------------- + +The asset and sensor surface tracks PhysX, but only a subset is implemented and +validated at the time of writing. Rigid Object support is merged on +``develop``; the remaining assets and sensors are landing through a series of +stacked pull requests: + +* RigidObject — merged via + `PR #5426 `_. +* Articulation — open in + `PR #5459 `_. +* Contact Sensor — open in + `PR #5422 `_. +* IMU — open in + `PR #5421 `_. +* RigidObjectCollection — open in + `PR #5570 `_. +* SceneDataProvider — open in + `PR #5589 `_. + +Other sensors (Frame Transformer, Joint Wrench, PVA, Ray Caster) and the +rendering surface are not yet wired up for OvPhysX. + +Status and follow-up +-------------------- + +This page is intentionally a stub. Once the in-flight OvPhysX work merges, this +section will be expanded with full installation, configuration, and supported +feature lists matching the other backends. The expansion is tracked in +`issue #5634 `_. + +For architectural context, see :doc:`../../multi_backend_architecture`. diff --git a/docs/source/overview/core-concepts/physical-backends/physx/configuration.rst b/docs/source/overview/core-concepts/physical-backends/physx/configuration.rst new file mode 100644 index 000000000000..1fc743d376b4 --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/physx/configuration.rst @@ -0,0 +1,108 @@ +PhysX Configuration +=================== + +PhysX scene-level settings live on :class:`~isaaclab_physx.physics.PhysxCfg`, +which replaces the legacy ``PhysxCfg`` from Isaac Lab 2.x. Per-actor settings +(such as per-articulation iteration counts, contact filtering, or material +properties) continue to be authored on the USD schema; see :doc:`../../schema_cfgs` +for the schema-level configuration helpers. + +The example below shows a representative ``PhysxCfg`` for a contact-rich +manipulation task: + +.. code-block:: python + + from isaaclab.sim import SimulationCfg + from isaaclab_physx.physics import PhysxCfg + + physx_cfg = PhysxCfg( + solver_type=1, # TGS + min_position_iteration_count=8, + max_position_iteration_count=64, + min_velocity_iteration_count=1, + max_velocity_iteration_count=4, + enable_ccd=False, + enable_stabilization=True, + bounce_threshold_velocity=0.2, + friction_offset_threshold=0.04, + friction_correlation_distance=0.025, + ) + + sim_cfg = SimulationCfg(dt=1 / 120, physics=physx_cfg) + + +Common Parameters +----------------- + +The following list highlights the parameters that most often need tuning. The +full reference lives on :class:`~isaaclab_physx.physics.PhysxCfg`. + +Solver Selection +^^^^^^^^^^^^^^^^ + +* ``solver_type``: ``1`` for **TGS** (Temporal Gauss-Seidel, default) or ``0`` + for **PGS** (Projective Gauss-Seidel). TGS is the recommended default for + articulated robots; PGS can be more forgiving on stiff legacy assets. +* ``solve_articulation_contact_last``: alters the articulation solver order so + that dynamic contact is resolved at the end of the solve. Useful for stiff + gripping scenarios. Requires Isaac Sim 5.1+. + + +Solver Iterations +^^^^^^^^^^^^^^^^^ + +* ``min_position_iteration_count`` / ``max_position_iteration_count``: clamp + range applied to every actor's individual position-iteration count. +* ``min_velocity_iteration_count`` / ``max_velocity_iteration_count``: clamp + range for velocity iterations. +* ``enable_external_forces_every_iteration``: applies external forces on every + TGS position iteration. Reduces noisy velocity updates at additional + compute cost; ignored with the PGS solver. + + +Contact and Stability +^^^^^^^^^^^^^^^^^^^^^ + +* ``enable_ccd``: continuous-collision detection for fast-moving bodies. +* ``enable_stabilization``: extra solver stabilization pass; recommended only + when ``dt`` is large (< 30 Hz). Corrupts contact-sensor force-magnitude + readings — disable it if you rely on the contact sensor for force + observations. +* ``bounce_threshold_velocity``: relative velocity threshold [m/s] above which + contacts bounce. +* ``friction_offset_threshold``: contact point distance [m] at which friction + forces are applied. +* ``friction_correlation_distance``: distance [m] used to merge nearby + contacts into a single friction anchor. + + +GPU Buffer Sizing +^^^^^^^^^^^^^^^^^ + +PhysX on the GPU does **not** dynamically grow scene buffers, so undersized +buffers crash or silently drop contacts. The ``gpu_*`` fields on +:class:`~isaaclab_physx.physics.PhysxCfg` control these capacities. The +defaults are tuned for a few-hundred-environment locomotion task; large +multi-thousand-environment manipulation tasks usually need to raise: + +* ``gpu_max_rigid_contact_count`` +* ``gpu_max_rigid_patch_count`` +* ``gpu_found_lost_pairs_capacity`` +* ``gpu_found_lost_aggregate_pairs_capacity`` +* ``gpu_total_aggregate_pairs_capacity`` +* ``gpu_collision_stack_size`` +* ``gpu_heap_capacity`` + +PhysX prints ``[PhysX]`` warnings when a buffer is exhausted; treat those as +hard failures and re-tune. + + +See Also +-------- + +* :class:`~isaaclab_physx.physics.PhysxCfg` — full parameter reference. +* :doc:`../../schema_cfgs` — schema-level configuration helpers. +* :doc:`../../multi_backend_architecture` — how PhysX plugs into the backend + factory pattern. +* PhysX 5 SDK documentation: + https://nvidia-omniverse.github.io/PhysX/physx/5.4.1/_api_build/classPxSceneDesc.html diff --git a/docs/source/overview/core-concepts/physical-backends/physx/index.rst b/docs/source/overview/core-concepts/physical-backends/physx/index.rst new file mode 100644 index 000000000000..3daedd4116b1 --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/physx/index.rst @@ -0,0 +1,37 @@ +PhysX Backend +============= + +`NVIDIA PhysX `_ +is the historical default physics backend in Isaac Lab. It runs through +`NVIDIA Isaac Sim `_'s Omniverse Kit +runtime and supports GPU-accelerated rigid-body, articulation, soft-body, and +particle simulation. PhysX is the reference backend for behaviour parity in Isaac +Lab — assets, sensors, and tasks have all been validated against it first, and the +other backends are measured against PhysX behaviour. + +PhysX is selected via :class:`~isaaclab_physx.physics.PhysxCfg`: + +.. code-block:: python + + from isaaclab.sim import SimulationCfg + from isaaclab_physx.physics import PhysxCfg + + sim_cfg = SimulationCfg(physics=PhysxCfg()) + +The PhysX backend uses the Temporal Gauss-Seidel (TGS) solver by default and also +exposes a Projective Gauss-Seidel (PGS) variant. Scene-level solver tuning, GPU +buffer sizing, and contact-handling parameters live on +:class:`~isaaclab_physx.physics.PhysxCfg`; per-actor settings remain on the USD +schema. See :doc:`configuration` for the most common knobs. + +For an overview of how the multi-backend architecture works, see +:doc:`../../multi_backend_architecture`. + + +.. toctree:: + :maxdepth: 2 + :titlesonly: + + installation + configuration + supported-features diff --git a/docs/source/overview/core-concepts/physical-backends/physx/installation.rst b/docs/source/overview/core-concepts/physical-backends/physx/installation.rst new file mode 100644 index 000000000000..60b3af80e1ed --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/physx/installation.rst @@ -0,0 +1,31 @@ +Installation +============ + +PhysX is installed as part of the standard Isaac Lab installation. It runs through +`NVIDIA Isaac Sim `_'s Omniverse Kit +runtime, so Isaac Sim is a required dependency for the PhysX backend. + +Follow the :ref:`isaaclab-installation-root` guide for the full installation +procedure. The short version: + +1. Install Isaac Sim 6.0 (binary install or pip install — see the Isaac Sim + documentation for system requirements). +2. Clone Isaac Lab and run ``./isaaclab.sh -i`` to install the Isaac Lab + extensions on top of Isaac Sim. + +No extra packages are required for the PhysX backend specifically — the PhysX +runtime ships with Isaac Sim. + + +Testing the Installation +------------------------ + +To verify the PhysX backend is working, run any classic Isaac Lab task with the +default preset: + +.. code-block:: bash + + ./isaaclab.sh -p scripts/environments/zero_agent.py --task Isaac-Cartpole-v0 --num_envs 128 + +The ``default`` preset on most tasks resolves to PhysX. You can also pass +``presets=physx`` explicitly on tasks that declare multi-backend physics presets. diff --git a/docs/source/overview/core-concepts/physical-backends/physx/supported-features.rst b/docs/source/overview/core-concepts/physical-backends/physx/supported-features.rst new file mode 100644 index 000000000000..faf620e963a9 --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/physx/supported-features.rst @@ -0,0 +1,70 @@ +Supported Features +================== + +PhysX is the broadest backend in Isaac Lab. It is the reference for behaviour +parity and supports every public asset, sensor, and renderer surface in the +framework. Tasks built before Isaac Lab 3.0 ran on PhysX, and the bulk of the +shipped tasks still default to the PhysX preset. + +The summary below is intentionally coarse; consult each component's API +documentation for fine-grained capability details. + +Core Simulation +--------------- + +* Articulation API (multi-link articulations, fixed-base and floating-base + articulations, single-body articulations modeled as rigid bodies) +* Rigid Object and Rigid Object Collection APIs +* Soft-body and particle simulation (legacy — not exposed through the + Isaac Lab asset surface but available through PhysX schemas) +* CPU and GPU pipelines; GPU is the default for the vectorized RL workloads + + +Sensors +------- + +PhysX implements the following sensors directly under +``isaaclab_physx/sensors/``: + +* Contact Sensor +* IMU +* Frame Transformer +* Joint Wrench Sensor +* PVA + +The following sensors are backend-agnostic (implemented in ``isaaclab`` core) +and work transparently with PhysX: + +* Ray Caster +* Camera — see :doc:`../../sensors/camera` + + +Rendering +--------- + +* RTX renderer (real-time rasterized; path tracing available through the + underlying Omniverse RTX pipeline) +* Tiled rendering for vectorized RGB / depth / segmentation + + +Tasks and Workflows +------------------- + +* Direct and Manager-based workflows +* All ``isaaclab_tasks`` environments default to the PhysX preset unless the + task explicitly opts in to a different backend +* Imitation learning and motion-generation pipelines (Mimic, motion generators) + + +Known Caveats +------------- + +* PhysX requires Isaac Sim and Omniverse Kit to be installed. +* GPU buffer sizes are static and must be tuned per task — see + :doc:`configuration`. +* ``enable_stabilization`` can corrupt contact-force readings reported through + the contact sensor; disable it if you rely on the contact sensor for + force-magnitude observations. +* The PhysX TGS solver behaviour can differ from Newton's MJWarp solver on + stiff contact stacks; if you are porting a task to Newton, expect to retune + contact-handling parameters. diff --git a/docs/source/overview/core-concepts/physical-backends/solver-comparison.rst b/docs/source/overview/core-concepts/physical-backends/solver-comparison.rst new file mode 100644 index 000000000000..d272d0f2f673 --- /dev/null +++ b/docs/source/overview/core-concepts/physical-backends/solver-comparison.rst @@ -0,0 +1,279 @@ +Solver Comparison +================= + +This page summarizes the user-visible behavioural differences between the +solvers shipped with the Isaac Lab physical backends. It's intended as a +porting reference: most tasks reuse the same USD asset across backends, but +contact, friction, and stabilization behave differently enough that retuning +is usually required when moving from one solver to another. + +The solvers covered are: + +* **PhysX TGS** — default solver for the :doc:`PhysX backend ` + (Temporal Gauss-Seidel). PhysX also exposes a Projective Gauss-Seidel + (PGS) variant via :attr:`~isaaclab_physx.physics.PhysxCfg.solver_type`, + which behaves similarly for the purposes of this comparison. +* **Newton MuJoCo-Warp (MJWarp)** — primary :doc:`Newton solver `, + configured by :class:`~isaaclab_newton.physics.MJWarpSolverCfg`. +* **Newton Kamino** — beta P-ADMM :doc:`Newton solver `, + configured by :class:`~isaaclab_newton.physics.KaminoSolverCfg`. + +Newton additionally ships ``FeatherstoneSolverCfg`` and ``XPBDSolverCfg``; +neither is wired into an Isaac Lab task at the time of writing and they +are omitted from this comparison. + + +Friction Model +-------------- + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Friction handling + * - PhysX TGS + - Coulomb friction with **patch-based** anchors. Tangential forces are + merged across nearby contacts via + :attr:`~isaaclab_physx.physics.PhysxCfg.friction_correlation_distance` + and applied above + :attr:`~isaaclab_physx.physics.PhysxCfg.friction_offset_threshold`. + The friction cone is always pyramidal. + * - MJWarp + - MuJoCo's friction model. The friction cone shape is selectable via + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.cone` + (``"pyramidal"`` or ``"elliptic"``). The tangential-to-normal + impedance ratio is exposed as + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.impratio`. + * - Kamino + - Per-contact friction resolved inside the P-ADMM solve. Contact + warm-starting is selectable via + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.padmm_contact_warmstart_method`; + the validated presets use ``"geom_pair_net_force"``. + +**Porting implication.** Tasks tuned for PhysX's patch friction can feel +"grippier" than MJWarp's per-contact friction at the same friction +coefficient. When moving a manipulation task from PhysX to MJWarp, expect to +raise friction coefficients and consider switching ``cone`` to ``"elliptic"`` +for stiffer contact stacks. + + +Contact Detection and Resolution +-------------------------------- + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Collision pipeline + * - PhysX TGS + - PhysX's built-in broadphase + narrowphase, with optional continuous + collision detection via + :attr:`~isaaclab_physx.physics.PhysxCfg.enable_ccd`. Pre-sized GPU + buffers (``gpu_max_rigid_contact_count`` etc.) cap the number of + contacts per step; oversubscription is a hard error. + * - MJWarp + - Two modes selected by + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.use_mujoco_contacts`: + MuJoCo's internal contact pipeline (default) or Newton's + :class:`~isaaclab_newton.physics.NewtonCollisionPipelineCfg`. The two + are mutually exclusive. GJK/EPA iteration count is exposed via + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.ccd_iterations`. + * - Kamino + - Optionally uses Kamino's internal collision detector (``"primitive"`` + or ``"unified"``) via + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.use_collision_detector`, + otherwise falls back to Newton's :class:`CollisionPipeline`. + Contact penetration margin is set by + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.constraints_delta`. + +**Porting implication.** A task that runs with ``--enable_ccd`` on PhysX +won't get the same protection on MJWarp/Kamino at large ``dt`` — Newton's +CCD is convex GJK/EPA, not the swept-shape CCD PhysX uses. The mitigation +on Newton is shorter ``dt`` or higher +:attr:`~isaaclab_newton.physics.NewtonCfg.num_substeps`. + + +Restitution and Bounce +---------------------- + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Restitution + * - PhysX TGS + - Restitution coefficient is per-material on the USD shape. A global + velocity threshold + :attr:`~isaaclab_physx.physics.PhysxCfg.bounce_threshold_velocity` + suppresses restitution below ~0.5 m/s by default. + * - MJWarp + - Restitution follows the MJCF model translated from USD. There is no + bounce-threshold gate — small-velocity contacts can still bounce + unless you reduce the per-material restitution. + * - Kamino + - Restitution is contained in the contact constraint set; behaviour is + similar to MJWarp. + +**Porting implication.** Tasks that rely on PhysX's bounce-threshold to +suppress jitter (e.g. footstep contact on flat ground) may show contact +chatter on Newton until restitution coefficients are reduced. + + +Constraint Stabilization +------------------------ + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Stabilization + * - PhysX TGS + - Implicit, via the TGS solver. An optional extra pass is enabled by + :attr:`~isaaclab_physx.physics.PhysxCfg.enable_stabilization` (note: + corrupts contact-sensor force magnitudes). + * - MJWarp + - Implicit, set by the MuJoCo solver's ``solref``/``solimp`` per + constraint. No top-level toggle. + * - Kamino + - Explicit Baumgarte stabilization with separate gains for joint + bilaterals (:attr:`~isaaclab_newton.physics.KaminoSolverCfg.constraints_alpha`), + joint-limit unilaterals + (:attr:`~isaaclab_newton.physics.KaminoSolverCfg.constraints_beta`), + and contact unilaterals + (:attr:`~isaaclab_newton.physics.KaminoSolverCfg.constraints_gamma`). + + +Solver Convergence +------------------ + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Iteration model + * - PhysX TGS + - Two iteration counts, per actor: position + (``min/max_position_iteration_count``) and velocity + (``min/max_velocity_iteration_count``). The solver takes the largest + actor's count clamped to the scene range. No global convergence + tolerance. + * - MJWarp + - :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.iterations` outer + Newton/CG iterations and + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.ls_iterations` line + searches per outer iteration. Convergence gate: + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.tolerance` (default + ``1e-6``). + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.ls_parallel` switches + line search to parallel execution at a small accuracy cost. + * - Kamino + - P-ADMM with separate + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.padmm_primal_tolerance`, + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.padmm_dual_tolerance`, + and + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.padmm_compl_tolerance` + gates, capped at + :attr:`~isaaclab_newton.physics.KaminoSolverCfg.padmm_max_iterations`. + Acceleration and warm-starting are tunable. + + +Articulation Coordinates +------------------------ + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Coordinate representation + * - PhysX TGS + - **Reduced-coordinate** articulations (joint-space, Featherstone-like). + Joint state is the canonical truth; body transforms are derived via + forward kinematics. + * - MJWarp + - Reduced-coordinate, computed by the MuJoCo solver. + * - Kamino + - **Maximal-coordinate**: each body has its own free-body state, + constraints are enforced via Baumgarte stabilization. Resets go + through a dedicated FK pass (:attr:`~isaaclab_newton.physics.KaminoSolverCfg.use_fk_solver`) + so maximal body poses match the reduced joint state after a state + write. + +**Porting implication.** Kamino is more sensitive to inconsistent reset +state — joint positions and body poses must agree, which Isaac Lab's asset +write paths handle but custom reset code can break. + + +Substepping and Timestep +------------------------ + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Substep model + * - PhysX TGS + - PhysX runs at the simulation ``dt``. No external substep counter; + internal substepping is per-actor. + * - MJWarp + - Top-level :attr:`~isaaclab_newton.physics.NewtonCfg.num_substeps` + controls how many solver substeps run per Isaac Lab step. Effective + solver ``dt`` is ``SimulationCfg.dt / num_substeps``. + * - Kamino + - Same :attr:`~isaaclab_newton.physics.NewtonCfg.num_substeps` knob. + Validated Kamino task presets typically use 1–2 substeps; expect to + raise this for contact-heavy tasks. + + +GPU Buffers and Throughput +-------------------------- + +.. list-table:: + :header-rows: 1 + :widths: 25 75 + + * - Solver + - Memory model + * - PhysX TGS + - Static GPU buffers sized at construction. Undersized buffers cause + crashes or dropped contacts at scale. See the + :doc:`PhysX configuration page ` for the + ``gpu_*`` knobs. + * - MJWarp + - Pre-allocated per-environment limits: + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.njmax` + (constraint rows) and + :attr:`~isaaclab_newton.physics.MJWarpSolverCfg.nconmax` (contact + points). The remainder of state lives in dynamically-sized Warp + arrays. + * - Kamino + - Inherits MJWarp's pre-allocation pattern via Newton, plus its own + contact-pair allocator + (:attr:`~isaaclab_newton.physics.KaminoSolverCfg.collision_detector_max_contacts_per_pair`) + when using the internal collision detector. + + +Porting Checklist +----------------- + +When moving a task between solvers: + +1. **Re-validate contact behavior.** Run the task at the smallest + ``num_envs`` with a visualizer; watch for new penetration or jitter + before scaling up. +2. **Retune friction.** PhysX patch friction and MJWarp per-contact friction + are not interchangeable at the same coefficient. +3. **Retune restitution.** MJWarp/Kamino have no bounce-threshold gate; + reduce per-material restitution to suppress jitter. +4. **Choose substeps.** PhysX → Newton typically needs at least 1–2 + substeps for contact-heavy tasks; manipulation tasks may need more. +5. **Watch for CCD differences.** Tasks that relied on PhysX swept CCD + should either reduce ``dt`` or raise ``num_substeps`` on Newton. +6. **For Kamino specifically**, also validate reset behaviour and consider + Baumgarte gains if you see drift on joint or contact constraints. diff --git a/docs/source/overview/reinforcement-learning/rl_existing_scripts.rst b/docs/source/overview/reinforcement-learning/rl_existing_scripts.rst index 56d6482f4676..d69faa9f6d01 100644 --- a/docs/source/overview/reinforcement-learning/rl_existing_scripts.rst +++ b/docs/source/overview/reinforcement-learning/rl_existing_scripts.rst @@ -29,9 +29,11 @@ See :doc:`/source/features/hydra` for all available names and how the selectors .. note:: - **Not all environments support every backend.** Using a preset with an environment - that has not been configured for that backend will raise an error at launch. See - :doc:`/source/experimental-features/newton-physics-integration/index` for details. + **Not all environments support the Newton backend yet.** Using ``presets=newton_mjwarp`` with an + environment that has not been configured for Newton will raise an error at launch. See + :doc:`/source/overview/core-concepts/physical-backends/newton/index` + for more details, and the :ref:`migrating-to-isaaclab-3-0` + guide for how to add Newton support to your own environments. Newton does not require Isaac Sim (kit-less mode). See :ref:`kitless-installation` for setup.