[Newton] Add GPU H2 tablecloth speed-comparison example by maxkra15 · Pull Request #7185 · isaac-sim/IsaacLab

maxkra15 · 2026-08-19T20:18:19Z

Description

Add an experimental Isaac Lab example: five fixed-base H2 Plus robots with native SHARPA hands perform contact-only tablecloth pulls at 4, 2, 1.5, 0.75 and 0.5 m/s. The three faster trials keep the dishes on their tables; both slow trials drop them.

uv run isaaclab example tablecloth --device cuda:0
# One trial without rendering:
uv run isaaclab example tablecloth --device cuda:0 --speeds 4.0 --viz none

InteractiveScene owns the side-by-side worlds, default ground plane, table visuals and primitive tableware colliders. Warp and Newton IK script approach, insertion, finger closure, withdrawal, lift and pull. Physics, actuators, IK, interpolation and grasp monitoring are GPU graph-captured. No cloth attachments or added fingertip pads are used.

Addressed the remaining review comments by keeping the scenario in one examples/tablecloth.py: standard articulation/cloth configs, private asset preparation and controller, and public articulation selection/native views instead of custom index resolution. The Contrib modules and dependency are removed. The seven-file feature diff has one 1,384-line Python example, down from 1,563 lines across three files. No RL task, custom recorder, new dependency, backend tests or generic solver changes remain. Shared VBD fixes landed in #8360. This branch includes upstream develop at 32fea6be38e3dafd3a157883780198e0bb3fc1a3.

The manufacturer USD is pinned to b6a8942b0803b6c137e58cef12beb4b03e4a2fa7 and retrieved through Isaac Lab's asset cache. Documentation is one compact example card, not a new page.

Screenshot and videos

Retained headless Kit recordings at 1080p/60 fps. Playback speed is not simulation throughput; Catmull–Clark only smooths rendering. These show the previously qualified physical configuration, retained unchanged by the refactor.

Five-speed overview:

tablecloth_h2_overview_1080p60.mp4

Wider side view of the fastest robot:

tablecloth_h2_wider_side_1080p60.mp4

Validation

Head: 37e1a5bd8f1da6dc1cf648f842348530fe975284. Pinned Newton 1.7.0.dev0 and Warp 1.17.0, NVIDIA L40.

Fresh complete five-world production run: 344 physics frames plus initialization, all states finite, full physics/actuator graph captured, all ten physical grips retained through the full pull. Minimum material travel 44.89 cm; worst sampled gap 1.35 mm. All three dishes remain on each of the three faster tables; all three fall from each slow table. Recorded source hashes match this head. Independent raw-trajectory checks confirm the outcomes. The run took 18.8 minutes including startup and diagnostic recording.
Actual five-world initialization, controller outputs for all 345 recorded observations, and all 64 interpolation ticks match the previous implementation exactly. Native control views alias the real strided target storage.
Captured two-world/two-frame CUDA startup completed with finite states. The temporary GPU diagnostic recorder is outside this PR; it adds no per-frame host state reads.
16 existing launcher contract checks passed; no new test file. CI retains the existing bounded startup matrix entry, not a complete manipulation test.
Full repository hooks, changelog validation, git diff --check, and a clean warning-as-error Sphinx HTML build passed.

Limitations

Calibrated fixed-base manipulation, not humanoid balance or a general grasp planner. The 64 physics ticks per 60 Hz control frame favor contact accuracy over real-time throughput. Direct compliant-ALM drives do not enforce native hardware torque limits or armature; whole-robot self-collision avoidance is not provided. Asset/contact changes require revalidation. One deterministic full run is not a robustness guarantee under perturbations. Existing upstream asset/runtime warnings remain.

Checklist

Followed contribution, coding-style and test-audit guidance.
Addressed the scenario-locality and public-index-selection review comments.
Validated GPU motion; retained bounded launch coverage without additional tests.
Ran hooks, changelog validation and a clean docs build.
Added the core package fragment/minor marker and a small actual JPEG.
Existing entry in CONTRIBUTORS.md.

kellyguo11 · 2026-09-01T01:39:17Z

do you want to land this on develop?

…n-tablecloth-demos

…n-tablecloth-demos # Conflicts: # source/isaaclab_newton/isaaclab_newton/physics/vbd_manager.py # source/isaaclab_newton/test/physics/test_vbd_core.py

maxkra15 · 2026-09-28T09:00:45Z

run-ci

…oth-h1-task

maxkra15 · 2026-09-30T13:10:50Z

run-ci

maxkra15 · 2026-10-12T13:33:58Z

run-ci

maxkra15 · 2026-10-12T14:40:57Z

run-ci

changed scope

## Summary Extract the reusable Newton fixes from #7185, as requested in review. No tablecloth environment, asset, example, or video recorder is included. - Include active cloth bending constraints in VBD graph coloring. - Expose structural `rigid_joint_linear_ke` and `rigid_joint_angular_ke`, independently of actuator gains; defaults retain Newton's `1e5` values. - Expose `friction_epsilon` with the native default `1e-2`, documented in m/s. - Publish generalized joint state after each standalone VBD physics tick. `update_joint_state=False` opts out. Coupled solvers and externally integrated rigid bodies retain their existing state ownership. - Respect a configured nested articulation root when constructing the Newton IK prototype view. Publication is a device-side Warp operation included in the physics CUDA graph, without CPU state reads. ## Validation Merged upstream `develop` at `cfa9e35c2a756849f3fea4bde9eac947f85a8300`. Current-head tests use pinned Newton `1.7.0.dev0` (`5ea82eef6f7911388025a8df6ccdb172e2fed286`) and Warp `1.17.0` on an NVIDIA L40: - `test_vbd_core.py`: **9 passed** — bending, native defaults, explicit stiffness/friction settings, and analytic gravity/torque equilibrium. - `test_rigid_object.py -k vbd_root`: **3 passed** — public-state publication, opt-out, and externally owned state. - `test_newton_ik_solver.py`: **5 passed**. - `test_coupler_runtime.py`: **4 passed** — includes comparison against native coupled stepping to protect generalized-state ownership. - `test_coupler.py`: **73 passed**. - Each file ran in its own process with `uv run --frozen --no-sync python -m pytest -q`. Existing upstream deprecation warnings remain. - Local regression mutations fail without the coupled-state guard, linear/angular gain forwarding, friction forwarding, or native default preservation. The duplicated stiffness simulation was consolidated while retaining independent default/configuration and physical-equilibrium checks. - `ISAACLAB_CHANGELOG_BASE_REF=cfa9e35c2a756849f3fea4bde9eac947f85a8300 UV_FROZEN=1 uv run --no-sync isaaclab -f`: passed. - `uv run --frozen --no-sync make -C docs current-docs`: clean warning-as-error build passed. ## Type of change - Bug fix - Non-breaking configuration additions - Documentation update ## Checklist - [x] Read the contribution guidelines and used the existing package conventions. - [x] Added focused regression coverage and ran formatting/lint checks. - [x] Added package changelog fragments and a minor-version marker for configuration additions. - [x] Updated the VBD tuning guide and built the documentation.

maxkra15 · 2026-10-12T20:38:04Z

run-ci

maxkra15 · 2026-10-12T21:18:44Z

run-ci

…oth-demos

maxkra15 · 2026-10-12T23:20:29Z

run-ci

Add Newton tablecloth demos

cd17461

github-actions Bot added the isaac-lab Related to Isaac Lab team label Aug 19, 2026

kellyguo11 approved these changes Sep 1, 2026

View reviewed changes

kellyguo11 added this to Isaac Lab Sep 1, 2026

kellyguo11 moved this to In progress in Isaac Lab Sep 1, 2026

maxkra15 added 4 commits September 10, 2026 13:52

Merge remote-tracking branch 'origin/develop' into maxkra15/max/newto…

0b12dfa

…n-tablecloth-demos

Refine Newton tablecloth demos

61c663b

Merge remote-tracking branch 'origin/develop' into maxkra15/max/newto…

77cf583

…n-tablecloth-demos

Clean up tablecloth changelog fragments

98fa9ad

github-actions Bot added the documentation Improvements or additions to documentation label Sep 14, 2026

maxkra15 added 5 commits September 14, 2026 10:41

Focus PR on tablecloth demos

5177546

Refactor Newton tablecloth demos

cad38fc

Focus VBD example on tablecloth comparison

a4ad7d7

Render configured Newton cloth appearance

e305ef8

Fix deformable appearance changelog fragment

560c87c

maxkra15 changed the title ~~Add Newton VBD tablecloth demos~~ Add Newton VBD tablecloth comparison Sep 14, 2026

Merge remote-tracking branch 'origin/develop' into maxkra15/max/newto…

07ce4d3

…n-tablecloth-demos # Conflicts: # source/isaaclab_newton/isaaclab_newton/physics/vbd_manager.py # source/isaaclab_newton/test/physics/test_vbd_core.py

ooctipus mentioned this pull request Sep 25, 2026

[5C] Move Newton deformable ownership out of contrib #8021

Merged

4 tasks done

maxkra15 added 2 commits September 28, 2026 10:59

Merge upstream develop

f31988d

Keep upstream lockfile metadata

7ffd441

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 28, 2026

maxkra15 added 2 commits September 30, 2026 14:41

Merge remote-tracking branch 'origin/develop' into max/newton-tablecl…

94df308

…oth-h1-task

Combine tablecloth comparison and H1 task examples

186f7f2

maxkra15 changed the title ~~Add Newton VBD tablecloth comparison~~ Add Newton VBD tablecloth examples Sep 30, 2026

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 30, 2026

maxkra15 added 2 commits October 12, 2026 14:45

Add compact tablecloth preview card and video links

d17c81a

Simplify H2 tablecloth control and asset setup

acb24f2

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 12, 2026

Write H2 tablecloth trajectories directly to GPU targets

903c610

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 12, 2026

Remove general VBD articulation claims from the guide

5c047a2

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 12, 2026

Merge upstream develop after VBD support fixes

c77969f

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 12, 2026

ooctipus reviewed Oct 12, 2026

View reviewed changes

Comment thread source/isaaclab_contrib/isaaclab_contrib/assets/tablecloth.py Outdated

ooctipus reviewed Oct 12, 2026

View reviewed changes

Comment thread source/isaaclab_contrib/isaaclab_contrib/controllers/tablecloth.py Outdated

ooctipus reviewed Oct 12, 2026

View reviewed changes

Comment thread source/isaaclab_contrib/isaaclab_contrib/controllers/tablecloth.py Outdated

maxkra15 added 2 commits October 13, 2026 01:12

Inline and simplify the H2 tablecloth example

462ecbc

Merge remote-tracking branch 'origin/develop' into max/newton-tablecl…

37e1a5b

…oth-demos

isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 12, 2026

AntoineRichard approved these changes Oct 13, 2026

View reviewed changes

maxkra15 merged commit 9fb7d05 into isaac-sim:develop Oct 13, 2026
59 checks passed

github-project-automation Bot moved this from In progress to Done in Isaac Lab Oct 13, 2026

Navigation Menu

Repository navigation

[Newton] Add GPU H2 tablecloth speed-comparison example - #7185

[Newton] Add GPU H2 tablecloth speed-comparison example#7185
maxkra15 merged 53 commits into
isaac-sim:developfrom
maxkra15:max/newton-tablecloth-demos

maxkra15 commented Aug 19, 2026 •

edited

Loading

Uh oh!

kellyguo11 commented Sep 1, 2026

Uh oh!

maxkra15 commented Sep 28, 2026

Uh oh!

maxkra15 commented Sep 30, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

maxkra15 commented Aug 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Screenshot and videos

Validation

Limitations

Checklist

Uh oh!

kellyguo11 commented Sep 1, 2026

Uh oh!

maxkra15 commented Sep 28, 2026

Uh oh!

maxkra15 commented Sep 30, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

maxkra15 commented Oct 12, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

maxkra15 commented Aug 19, 2026 •

edited

Loading