Skip to content

[Proposal] Support Differential IK on the Newton backend #5451

Closed

Description

@aravindskumar-nvidia

Proposal

Add a PhysX-compatible get_jacobians() to isaaclab_newton's ArticulationView so DifferentialInverseKinematicsAction works on Newton, and drop the PhysX pin that's been used as a workaround in IK env configs (e.g. the Franka reach IK envs).

Motivation

DifferentialInverseKinematicsAction reads the Jacobian via self._asset.root_view.get_jacobians(). PhysX has it; Newton's ArticulationView (from newton.selection) only exposes eval_jacobian(), which returns the spatial Jacobian — so the existing call site doesn't work as-is on Newton, regardless of which robot is being used.

The current workaround is to pin affected envs to PhysX. The Franka reach IK envs do this explicitly:

# IK control is not supported with Newton physics; use PhysX only.
self.sim.physics = PhysxCfg(bounce_threshold_velocity=0.2)

It would be nice to remove that carve-out so IK envs run on either backend.

Alternatives

A clear and concise description of any alternative solutions or features you've considered, if any.

Build Info

Describe the versions where you are observing the missing feature in:

  • Isaac Lab Version: 4.6.22
  • isaaclab_newton Version: 0.5.25

Additional context

  • Subclass newton.selection.ArticulationView in isaaclab_newton.assets.articulation and add get_jacobians().
  • Inside it, call eval_jacobian(state), then convert spatial to geometric per body
  • Return a wp.array so the existing wp.to_torch(...) path in task_space_actions.py keeps working unchanged.
  • Re-export the adapter from isaaclab_newton.assets.articulation so Articulation picks it up instead of Newton's class directly.

Checklist

  • I have checked that there is no similar issue in the repo (required)

Acceptance Criteria

(May be refined)

  • isaaclab_newton's ArticulationView.get_jacobians() matches PhysX's shape and convention for fixed- and floating-base articulations.
  • DifferentialInverseKinematicsAction runs on Newton without changes to its call site.
  • The PhysX pin in the Franka reach IK envs can be removed.
  • Test added comparing the Newton Jacobian against PhysX on a representative articulation.

Activity

  1. aravindskumar-nvidia commented on Apr 30, 2026

    @aravindskumar-nvidia
    ContributorAuthor

    @kellyguo11 please let me know if this makes sense. I can open a PR accordingly

  2. kellyguo11 commented on Apr 30, 2026

    @kellyguo11
    Contributor

    Thanks, we've started looking into this. PR is here - #5400

  3. aravindskumar-nvidia commented on Apr 30, 2026

    @aravindskumar-nvidia
    ContributorAuthor

    Thank you! Exactly what I was looking for. Will try out the branch

  4. AntoineRichard commented on Aug 27, 2026

    @AntoineRichard
    Collaborator

    Thanks @aravindskumar-nvidia. This was implemented by #5400 ([Newton] Backend-agnostic task-space accessors for IK/OSC, merged 2026-05-13). Newton articulations now expose the Jacobian contract DifferentialInverseKinematicsAction relies on — see the test_get_jacobians_* cases in source/isaaclab_newton/test/assets/test_articulation.py — and the PhysX-only carve-out in the Franka reach IK configs is gone, so diffik resolves on the Newton MJWarp preset.

    Closing as implemented — please reopen with a repro if an IK/OSC case on Newton still misbehaves.


    🤖 This issue was reviewed as part of an AI-assisted triage of the Isaac Lab backlog, with automated verification against the current codebase. If we got this wrong, please reopen — we'd rather hear about it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions