Skip to content

Delegate native learned-dynamics primitives to PhysicsNeMo - #8835

Draft
ooctipus wants to merge 22 commits into
isaac-sim:mainfrom
ooctipus:feat/physicsnemo-native-integration
Draft

ooctipus wants to merge 22 commits into
isaac-sim:mainfrom
ooctipus:feat/physicsnemo-native-integration

Conversation

@ooctipus

@ooctipus ooctipus commented Oct 26, 2026 •

Copy link
Copy Markdown
Collaborator

Replace local learned-dynamics implementations with these PhysicsNeMo Robotics components:

  • JointStateCodec + GravityEncoder: replace Isaac Lab's heading conversion and gravity-feature math.
  • FieldNormalizer: replaces separate offline NeRD training/deployment normalization.
  • RunningNormalizer: replaces SuperMimic's local RunningNorm in its world model, policy and critic.
  • LearnedDynamicsSolver + DynamicsInferenceSession: replace local history/reset buffers, prediction orchestration and state-writeback/FK code; add resident temporal caching.

NeRD networks and contact encoders were already reused before this PR. Isaac Lab retains tasks, capture, physical-model checks, dataset loading, replay, optimization and checkpoint orchestration.

Actual code reduction: 59 production Python lines. 13 existing files change; including tests/docs, the net change is +20 lines. The main diff includes unmerged prerequisites.

Prototype inference: 4.0× faster at 128 worlds (1.30 → 0.324 ms); 12.2× at 4,096 (19.52 → 1.60 ms). RTX 5090, FP32, CUDA graphs; excludes collision, FK, sensors and policy/training. End-to-end throughput remains unmeasured.

Validated: 15 integration tests, MLP/NeRD training smokes, normalization parity, formatting and docs passed.

Migration: NeRD v2 checkpoints need refitting; SuperMimic checkpoints remain compatible. Both SuperMimic learners require PhysicsNeMo here.

Isaac Lab dependencies: #8825/#8826 and shared #8822. They do not require this follow-up's new support commit.

External dependency: PhysicsNeMo Robotics (internal repository, package physicsnemo_robotics), local branch feat/resident-dynamics-ownership, required commit e597e95bedb1f3f90637f178c045bc70c944e5d2. This support is unpublished; no dependency MR exists yet because push access was denied. This PR remains draft until publication and package compatibility are resolved.

Contribution checklist
  • Read contribution guidelines and applied test-audit criteria
  • Updated owner tests, documentation, migration notes and changelog fragments
  • Audited dependency directions and avoided new integration modules
  • Ran repository formatting/changelog checks
  • Combined-branch validation complete
  • Publish compatible PhysicsNeMo support and resolve installation pins
  • Merge prerequisite task/recorder PRs

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infrastructure isaac-mimic Related to Isaac Mimic team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant