Skip to content

Fix Newton VBD articulation state and cloth bending integration - #8360

Merged
AntoineRichard merged 6 commits into
isaac-sim:developfrom
maxkra15:max/newton-vbd-articulation-fixes
Oct 12, 2026
Merged

AntoineRichard merged 6 commits into
isaac-sim:developfrom
maxkra15:max/newton-vbd-articulation-fixes

Conversation

@maxkra15

@maxkra15 maxkra15 commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Reusable Newton fixes extracted from #7185. No environment, robot asset, example or video recorder is included.

  • Include active cloth bending constraints in VBD coloring.
  • Expose structural linear/angular joint stiffness independently of actuator gains, preserving native 1e5 defaults.
  • Expose friction_epsilon, preserving 1e-2 m/s.
  • Publish generalized state after standalone VBD physics ticks inside the CUDA graph. update_joint_state=False opts out; coupled and externally integrated rigid bodies retain their state ownership.
  • Resolve configured nested USD articulation roots when constructing a Newton IK action prototype.
  • Document structural versus actuator stiffness and NewtonCfg collision scheduling at their proper owners.

Related: #6733. Prerequisite for the refreshed #7185 example; independent of #8107. Migration and new runtime dependencies: none.

Type of change

  • Bug fix
  • Non-breaking configuration additions
  • Documentation update

Validation

Merged upstream develop at dd2f470fcad5873980e9bc4acffb043238aca41c. Validated source committed as f2a6caef2e342d511c2b15677f24fe862fcfef66 on an NVIDIA L40, pinned Newton 1.7.0.dev0 (5ea82eef6f7911388025a8df6ccdb172e2fed286) and Warp 1.17.0. Each file ran separately with uv run --frozen --no-sync python -m pytest -q:

Test selection Passed
source/isaaclab_newton/test/physics/test_vbd_core.py 9
source/isaaclab_newton/test/assets/test_articulation.py 38
source/isaaclab_newton/test/assets/test_rigid_object.py -k vbd_root 3
source/isaaclab_newton/test/controllers/test_newton_ik_solver.py 5
source/isaaclab_contrib/test/coupling/test_coupler_runtime.py 4
source/isaaclab_contrib/test/coupling/test_coupler.py 73

132 passed, zero failed or skipped. The new real-asset nested-root action test checks an independently derived 0.4-radian joint target; removing the root suffix fails at native articulation selection. Existing upstream deprecation/fixture warnings remain.

ISAACLAB_CHANGELOG_BASE_REF=origin/develop UV_FROZEN=1 uv run --no-sync isaaclab -f passed, including the changelog gate. A fresh warning-as-error documentation build passed using python -m sphinx -b html -W --keep-going -E -a -j auto docs docs/_build/support-review-guide-final.

The retained tests distinguish native defaults/configuration, analytic stiffness equilibrium, public state ownership and coupled/native parity. Newton owns offset/moving-parent joint reconstruction; no duplicate kinematics fixture or tablecloth rollout is added here.

Publication overhead was measured on the existing headless block-and-tackle demo on the same L40: four separate processes in off/on/on/off order, each with 40 warm-up frames and five 120-frame batches. The average process medians were 18.2855 ms/frame off and 18.3147 ms/frame on (difference 0.0292 ms, 0.16%). All runs captured a graph. This is a small-workload observation, not a significant difference or general throughput guarantee; GPU 1 was concurrently used for Kit rendering during later runs. The comparison changes only update_joint_state and synchronizes between batches, not per frame.

New-head CI is requested separately.

Checklist

  • Read the contribution guide and followed package conventions.
  • Ran formatting/lint, focused tests and the test-audit gate.
  • Updated documentation and per-package fragments/minor marker.
  • Existing entry in CONTRIBUTORS.md.

@maxkra15
maxkra15 requested a review from a team October 7, 2026 09:37
@github-actions github-actions Bot added bug Something isn't working documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team labels Oct 7, 2026
@maxkra15

maxkra15 commented Oct 7, 2026

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 7, 2026
@greptile-apps

greptile-apps Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[High risk] Modifies VBD physics solver behavior and joint state synchronization.

Fix coupled managers' state-ownership policy before merging.

Findings

  1. P1 Coupled solvers overwrite owned state ▶
  2. P2 Articulated joint motion goes untested ▶

Summary

This PR adds cloth bending to VBD coloring, exposes structural joint stiffness, publishes joint state after body integration, and uses configured nested roots for Newton IK.

  • Coupled managers inherit publication without applying their state-ownership settings.
  • The publication regression covers a free root, but not joints between moving bodies.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Physics tick] --> B[Run solver substeps]
    B --> C[Final state_0]
    C --> D{_update_joint_state?}
    D -->|Yes| E[eval_ik writes joint_q and joint_qd]
    D -->|No| F[Keep solver-owned joint state]
    G[Plain VBD build] --> H[Set flag from configuration]
    H --> D
    I[Coupled manager build] --> J[Inherit flag without setting ownership policy]
    J --> D
Loading

Reviews (1) · Last reviewed commit: "Fix VBD articulation state and cloth ben..." · Reviewed by Greptile

Comment thread source/isaaclab_newton/isaaclab_newton/physics/vbd_manager.py
Comment thread source/isaaclab_newton/test/assets/test_rigid_object.py

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

Needs attention: Style consistency (2). Other reviewed categories look good.

  • PR description: Description matches the diff and reports workload, warmup, and timings; add the benchmark command, hardware, driver, environment count, revisions, and pinned Newton version for reproducibility.
  • Design and architecture: Architecture looks good.
  • Scope and over-engineering: No over-engineering issues.
  • API: API changes look good.
  • Compatibility and deprecation: No breaking changes identified.
  • Implementation: Implementation looks good.
  • Style consistency: The added own-scene VBD test makes the module-level scene-ownership description stale; update that description to include this case.
  • Test quality: Test quality looks good.

Minor fixes needed. Posted 2 inline findings.

Automated review; human maintainers own approval decisions.

Comment thread source/isaaclab_newton/test/assets/test_rigid_object.py
Comment thread source/isaaclab_newton/test/physics/test_vbd_core.py Outdated
Comment thread source/isaaclab_newton/test/physics/test_vbd_core.py Outdated

@mmichelis mmichelis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the option for eval_ik!

@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 11, 2026
@AntoineRichard
AntoineRichard enabled auto-merge (squash) October 12, 2026 11:24
@maxkra15

Copy link
Copy Markdown
Contributor Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 12, 2026
@AntoineRichard
AntoineRichard merged commit 3c02cf7 into isaac-sim:develop Oct 12, 2026
91 of 92 checks passed
maxkra15 added a commit that referenced this pull request Oct 13, 2026
## Description

Add an experimental Isaac Lab example: five fixed-base H2 Plus robots
with native SHARPA hands perform contact-only tablecloth pulls at 4, 2,
1.5, 0.75 and 0.5 m/s. The three faster trials keep the dishes on their
tables; both slow trials drop them.

```bash
uv run isaaclab example tablecloth --device cuda:0
# One trial without rendering:
uv run isaaclab example tablecloth --device cuda:0 --speeds 4.0 --viz none
```

`InteractiveScene` owns the side-by-side worlds, default ground plane,
table visuals and primitive tableware colliders. Warp and Newton IK
script approach, insertion, finger closure, withdrawal, lift and pull.
Physics, actuators, IK, interpolation and grasp monitoring are GPU
graph-captured. No cloth attachments or added fingertip pads are used.

Addressed the remaining review comments by keeping the scenario in one
`examples/tablecloth.py`: standard articulation/cloth configs, private
asset preparation and controller, and public articulation
selection/native views instead of custom index resolution. The Contrib
modules and dependency are removed. The seven-file feature diff has one
1,384-line Python example, down from 1,563 lines across three files. No
RL task, custom recorder, new dependency, backend tests or generic
solver changes remain. Shared VBD fixes landed in #8360. This branch
includes upstream `develop` at
`32fea6be38e3dafd3a157883780198e0bb3fc1a3`.

The manufacturer USD is pinned to
`b6a8942b0803b6c137e58cef12beb4b03e4a2fa7` and retrieved through Isaac
Lab's asset cache. Documentation is one compact example card, not a new
page.

## Screenshot and videos

![Five H2 Plus / SHARPA tablecloth
trials](https://raw.githubusercontent.com/maxkra15/IsaacLab/71c9d480dc5185c111b83b851faa8dd460165690/docs/source/_static/examples/tablecloth_h2.jpg)

Retained headless Kit recordings at 1080p/60 fps. Playback speed is not
simulation throughput; Catmull–Clark only smooths rendering. These show
the previously qualified physical configuration, retained unchanged by
the refactor.

Five-speed overview:


https://github.com/user-attachments/assets/6a22e791-ed3c-4488-9853-6e8afdbcbcc2

Wider side view of the fastest robot:


https://github.com/user-attachments/assets/27ac0365-e862-4d60-bd85-51f585a580d5

## Validation

Head: `37e1a5bd8f1da6dc1cf648f842348530fe975284`. Pinned Newton
`1.7.0.dev0` and Warp `1.17.0`, NVIDIA L40.

- Fresh complete five-world production run: 344 physics frames plus
initialization, all states finite, full physics/actuator graph captured,
all ten physical grips retained through the full pull. Minimum material
travel 44.89 cm; worst sampled gap 1.35 mm. All three dishes remain on
each of the three faster tables; all three fall from each slow table.
Recorded source hashes match this head. Independent raw-trajectory
checks confirm the outcomes. The run took 18.8 minutes including startup
and diagnostic recording.
- Actual five-world initialization, controller outputs for all 345
recorded observations, and all 64 interpolation ticks match the previous
implementation exactly. Native control views alias the real strided
target storage.
- Captured two-world/two-frame CUDA startup completed with finite
states. The temporary GPU diagnostic recorder is outside this PR; it
adds no per-frame host state reads.
- 16 existing launcher contract checks passed; no new test file. CI
retains the existing bounded startup matrix entry, not a complete
manipulation test.
- Full repository hooks, changelog validation, `git diff --check`, and a
clean warning-as-error Sphinx HTML build passed.

## Limitations

Calibrated fixed-base manipulation, not humanoid balance or a general
grasp planner. The 64 physics ticks per 60 Hz control frame favor
contact accuracy over real-time throughput. Direct compliant-ALM drives
do not enforce native hardware torque limits or armature; whole-robot
self-collision avoidance is not provided. Asset/contact changes require
revalidation. One deterministic full run is not a robustness guarantee
under perturbations. Existing upstream asset/runtime warnings remain.

## Checklist

- [x] Followed contribution, coding-style and test-audit guidance.
- [x] Addressed the scenario-locality and public-index-selection review
comments.
- [x] Validated GPU motion; retained bounded launch coverage without
additional tests.
- [x] Ran hooks, changelog validation and a clean docs build.
- [x] Added the core package fragment/minor marker and a small actual
JPEG.
- [x] Existing entry in `CONTRIBUTORS.md`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants