Skip to content

Fix operational-space feedback and task-frame consistency - #7868

Merged
kellyguo11 merged 4 commits into
isaac-sim:developfrom
ooctipus:fix/osc-test-feedback
Sep 17, 2026
Merged

kellyguo11 merged 4 commits into
isaac-sim:developfrom
ooctipus:fix/osc-test-feedback

Conversation

@ooctipus

@ooctipus ooctipus commented Sep 17, 2026 •

Copy link
Copy Markdown
Collaborator

OperationalSpaceControllerAction and the OSC integration-test helper used link-origin poses and Jacobians together with center-of-mass velocity feedback. When a link rotates with an offset center of mass, these quantities describe different points, producing incorrect velocity feedback. The task-frame test also copied rounded command quaternions directly into reference frames, producing inconsistent composed targets.

Use body_link_vel_w and root_link_vel_w in both OSC callers, and normalize the test's cloned task-frame quaternion before coordinate conversion. Controller gains, raw command fixtures, convergence tolerances, and step budgets match the base branch; the nullspace damping ratio remains 1.0.

Two direct regressions guard these contracts: a moving, fixed-base Franka checks each caller's velocity against its link Jacobian multiplied by measured joint velocity, and a numerical check verifies that the same rounded command resolves to the same absolute target through root and task frames without mutating its inputs.

Provenance:

  • #5400 migrated both OSC callers to body_link_jacobian_w while retaining the COM velocity accessors.
  • The rounded fixtures and unnormalized task-frame copy originated in #913. #7624 later normalized absolute pose commands, while the separately supplied frame remained unnormalized. The exact change that first exposed the convergence failures has not been bisected; these references establish the input inconsistencies, not the first failing CI run.

Validation:

  • Restoring COM feedback makes both velocity regression cases fail, with a maximum linear-velocity discrepancy of 0.01469 m/s versus the 1e-4 assertion tolerance. Both pass with link-origin feedback.
  • Removing frame normalization makes the target-equivalence regression fail with a quaternion-component discrepancy of 0.00010675 versus the 1e-6 tolerance. It passes with the correction.
  • The previously failing test_franka_taskframe_pose_abs and original test_franka_pose_abs_with_nullspace_centering both pass with damping 1.0.
  • Full OSC integration file: 21 passed in 113 seconds, including all three new regression cases.
  • uv run isaaclab -f passes.

The earlier RL, legacy-rendering, and contrib failures have matching failures and passing corresponding jobs in separate fixes: #7870, #7871, and #7866, respectively. Those fixes remain separate from these OSC corrections.

@ooctipus
ooctipus requested a review from a team September 17, 2026 11:06
@github-actions github-actions Bot added bug Something isn't working isaac-lab Related to Isaac Lab team labels Sep 17, 2026

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The patch makes a focused test-only correction: it normalizes task-frame quaternion fixtures, increases damping for the nullspace-centering pose test, and switches velocity feedback to link-origin quantities consistent with the pose and Jacobian used by _update_states.

  • Design and architecture: The feedback path now consistently uses link-origin pose, Jacobian, and velocity data. The damping adjustment remains scoped to one test configuration and does not alter the operational-space controller implementation.
  • API: No public APIs, defaults, exports, or documented runtime behavior are changed. The edits are confined to test fixtures and a private test helper, with an appropriate .skip changelog fragment.
  • Implementation: The quaternion fixtures are normalized before use, and the changed velocity producer matches the existing link-origin consumers. Increasing the damping ratio from 1.0 to 2.0 reduces coverage of the less-damped convergence regime, but this is a non-blocking tradeoff for stabilizing the large-step nullspace-centering test.

No blocking issues. No inline issue met the actionable-evidence threshold; the assessment above records the review feedback.

Automated review; human maintainers own approval decisions.

@greptile-apps

greptile-apps Bot commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with the test feedback and fixtures now using consistent reference points and valid task-frame orientations.

Summary

This PR corrects operational-space controller integration-test fixtures and feedback without changing the controller implementation, convergence tolerances, or simulation budgets.

  • Normalizes absolute-pose quaternion fixtures before they are reused as task-frame orientations.
  • Uses link-origin end-effector and root velocities consistently with the link-origin pose and Jacobian.
  • Increases task damping for the large nullspace-centering pose steps.
  • Adds a changelog fragment describing the test stabilization.

Reviews (1) · Last reviewed commit: "Correct operational-space controller tes..."

@maxkra15

Copy link
Copy Markdown
Contributor

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 17, 2026
@ooctipus ooctipus changed the title Fix operational-space controller test feedback and convergence Use consistent link-origin feedback for operational-space control Sep 17, 2026
@ooctipus ooctipus changed the title Use consistent link-origin feedback for operational-space control Fix operational-space feedback and task-frame consistency Sep 17, 2026
@ooctipus

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 17, 2026
@kellyguo11

Copy link
Copy Markdown
Contributor

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 17, 2026
@kellyguo11
kellyguo11 merged commit e836331 into isaac-sim:develop Sep 17, 2026
79 of 82 checks passed
kellyguo11 added a commit to kellyguo11/IsaacLab-public that referenced this pull request Sep 17, 2026
…7868)

`OperationalSpaceControllerAction` and the OSC integration-test helper
used link-origin poses and Jacobians together with center-of-mass
velocity feedback. When a link rotates with an offset center of mass,
these quantities describe different points, producing incorrect velocity
feedback. The task-frame test also copied rounded command quaternions
directly into reference frames, producing inconsistent composed targets.

Use `body_link_vel_w` and `root_link_vel_w` in both OSC callers, and
normalize the test's cloned task-frame quaternion before coordinate
conversion. Controller gains, raw command fixtures, convergence
tolerances, and step budgets match the base branch; the nullspace
damping ratio remains 1.0.

Two direct regressions guard these contracts: a moving, fixed-base
Franka checks each caller's velocity against its link Jacobian
multiplied by measured joint velocity, and a numerical check verifies
that the same rounded command resolves to the same absolute target
through root and task frames without mutating its inputs.

Provenance:

- [isaac-sim#5400](isaac-sim#5400) migrated both
OSC callers to `body_link_jacobian_w` while retaining the COM velocity
accessors.
- The rounded fixtures and unnormalized task-frame copy originated in
[isaac-sim#913](isaac-sim#913).
[isaac-sim#7624](isaac-sim#7624) later
normalized absolute pose commands, while the separately supplied frame
remained unnormalized. The exact change that first exposed the
convergence failures has not been bisected; these references establish
the input inconsistencies, not the first failing CI run.

Validation:

- Restoring COM feedback makes both velocity regression cases fail, with
a maximum linear-velocity discrepancy of 0.01469 m/s versus the 1e-4
assertion tolerance. Both pass with link-origin feedback.
- Removing frame normalization makes the target-equivalence regression
fail with a quaternion-component discrepancy of 0.00010675 versus the
1e-6 tolerance. It passes with the correction.
- The previously failing `test_franka_taskframe_pose_abs` and original
`test_franka_pose_abs_with_nullspace_centering` both pass with damping
1.0.
- Full OSC integration file: **21 passed** in 113 seconds, including all
three new regression cases.
- `uv run isaaclab -f` passes.

The earlier RL, legacy-rendering, and contrib failures have matching
failures and passing corresponding jobs in separate fixes: isaac-sim#7870, isaac-sim#7871,
and isaac-sim#7866, respectively. Those fixes remain separate from these OSC
corrections.

---------

Co-authored-by: Kelly Guo <kellyg@nvidia.com>
(cherry picked from commit e836331)
kellyguo11 added a commit to kellyguo11/IsaacLab-public that referenced this pull request Sep 17, 2026
…7868)

`OperationalSpaceControllerAction` and the OSC integration-test helper
used link-origin poses and Jacobians together with center-of-mass
velocity feedback. When a link rotates with an offset center of mass,
these quantities describe different points, producing incorrect velocity
feedback. The task-frame test also copied rounded command quaternions
directly into reference frames, producing inconsistent composed targets.

Use `body_link_vel_w` and `root_link_vel_w` in both OSC callers, and
normalize the test's cloned task-frame quaternion before coordinate
conversion. Controller gains, raw command fixtures, convergence
tolerances, and step budgets match the base branch; the nullspace
damping ratio remains 1.0.

Two direct regressions guard these contracts: a moving, fixed-base
Franka checks each caller's velocity against its link Jacobian
multiplied by measured joint velocity, and a numerical check verifies
that the same rounded command resolves to the same absolute target
through root and task frames without mutating its inputs.

Provenance:

- [isaac-sim#5400](isaac-sim#5400) migrated both
OSC callers to `body_link_jacobian_w` while retaining the COM velocity
accessors.
- The rounded fixtures and unnormalized task-frame copy originated in
[isaac-sim#913](isaac-sim#913).
[isaac-sim#7624](isaac-sim#7624) later
normalized absolute pose commands, while the separately supplied frame
remained unnormalized. The exact change that first exposed the
convergence failures has not been bisected; these references establish
the input inconsistencies, not the first failing CI run.

Validation:

- Restoring COM feedback makes both velocity regression cases fail, with
a maximum linear-velocity discrepancy of 0.01469 m/s versus the 1e-4
assertion tolerance. Both pass with link-origin feedback.
- Removing frame normalization makes the target-equivalence regression
fail with a quaternion-component discrepancy of 0.00010675 versus the
1e-6 tolerance. It passes with the correction.
- The previously failing `test_franka_taskframe_pose_abs` and original
`test_franka_pose_abs_with_nullspace_centering` both pass with damping
1.0.
- Full OSC integration file: **21 passed** in 113 seconds, including all
three new regression cases.
- `uv run isaaclab -f` passes.

The earlier RL, legacy-rendering, and contrib failures have matching
failures and passing corresponding jobs in separate fixes: isaac-sim#7870, isaac-sim#7871,
and isaac-sim#7866, respectively. Those fixes remain separate from these OSC
corrections.

---------

Co-authored-by: Kelly Guo <kellyg@nvidia.com>
(cherry picked from commit e836331)
kellyguo11 added a commit to kellyguo11/IsaacLab-public that referenced this pull request Sep 17, 2026
…7868)

`OperationalSpaceControllerAction` and the OSC integration-test helper
used link-origin poses and Jacobians together with center-of-mass
velocity feedback. When a link rotates with an offset center of mass,
these quantities describe different points, producing incorrect velocity
feedback. The task-frame test also copied rounded command quaternions
directly into reference frames, producing inconsistent composed targets.

Use `body_link_vel_w` and `root_link_vel_w` in both OSC callers, and
normalize the test's cloned task-frame quaternion before coordinate
conversion. Controller gains, raw command fixtures, convergence
tolerances, and step budgets match the base branch; the nullspace
damping ratio remains 1.0.

Two direct regressions guard these contracts: a moving, fixed-base
Franka checks each caller's velocity against its link Jacobian
multiplied by measured joint velocity, and a numerical check verifies
that the same rounded command resolves to the same absolute target
through root and task frames without mutating its inputs.

Provenance:

- [isaac-sim#5400](isaac-sim#5400) migrated both
OSC callers to `body_link_jacobian_w` while retaining the COM velocity
accessors.
- The rounded fixtures and unnormalized task-frame copy originated in
[isaac-sim#913](isaac-sim#913).
[isaac-sim#7624](isaac-sim#7624) later
normalized absolute pose commands, while the separately supplied frame
remained unnormalized. The exact change that first exposed the
convergence failures has not been bisected; these references establish
the input inconsistencies, not the first failing CI run.

Validation:

- Restoring COM feedback makes both velocity regression cases fail, with
a maximum linear-velocity discrepancy of 0.01469 m/s versus the 1e-4
assertion tolerance. Both pass with link-origin feedback.
- Removing frame normalization makes the target-equivalence regression
fail with a quaternion-component discrepancy of 0.00010675 versus the
1e-6 tolerance. It passes with the correction.
- The previously failing `test_franka_taskframe_pose_abs` and original
`test_franka_pose_abs_with_nullspace_centering` both pass with damping
1.0.
- Full OSC integration file: **21 passed** in 113 seconds, including all
three new regression cases.
- `uv run isaaclab -f` passes.

The earlier RL, legacy-rendering, and contrib failures have matching
failures and passing corresponding jobs in separate fixes: isaac-sim#7870, isaac-sim#7871,
and isaac-sim#7866, respectively. Those fixes remain separate from these OSC
corrections.

---------

Co-authored-by: Kelly Guo <kellyg@nvidia.com>
(cherry picked from commit e836331)
kellyguo11 added a commit that referenced this pull request Sep 17, 2026
# Description

Consolidates the complete `develop` to `release/3.0.0` backport set
previously split across #7851, #7852, and #7853. Keeping the changes
together lets CI validate the runtime, task, tooling, dependency, and
renderer prerequisites as one coherent release tree.

The aggregate includes all audited develop changes through `e8363313c`,
including the latest #7866, #7867, #7868, #7869, and #7870 fixes. It
also includes the exact net patch from #7871 (`412cac434` / patch ID
`0f69afa6`) because the externally updated Franka asset already breaks
the release legacy-rendering matrix; #7871 is still pending merge to
`develop` at the time of this update.

Release CI now uses the same immutable Isaac Sim `latest-develop` digest
as `develop`:

```yaml
isaacsim_image_tag: latest-develop@sha256:223adbb0a6f1897ed78d40597dac2efc740843e8051779ffdcbb021a9edc1217
```

This supersedes #7852 and #7853.

## Type of change

- Bug fix (non-breaking change which fixes an issue)
- New feature (non-breaking change which adds functionality)
- Documentation update
- Release integration/backport

## Release backport

- [ ] <!-- backport-active-release --> Not applicable: this PR targets
`release/3.0.0` directly.

## Screenshots

Not applicable.

## Validation

- Latest five develop commits are patch-equivalent in the aggregate
tree.
- #7871 renderer fix matches its source patch ID exactly.
- `.github/workflows/config.yaml` matches `develop`.
- 33 OVRTX USD/export tests passed; 19 optional backend tests skipped
locally.
- 3 focused Franka Pour/Lift/Reorient collider regressions passed.
- 45 standalone supervisor tests passed; 386 simulator launch cases
skipped locally.
- All 23 repository skills validated.
- Ruff check and formatting check passed on all manually
merged/latest-fix Python files.
- Workflow YAML parsing, `uv lock --check`, and aggregate `git diff
--check` passed.
- Full `uv run isaaclab -f` and local docs build cannot run on macOS
ARM64 because the lock supports Linux x86_64/aarch64 and Windows AMD64;
Linux CI is authoritative for those checks.

## Checklist

- [x] I have read and understood the contribution guidelines.
- [ ] I have run the full pre-commit command locally
(platform-incompatible; focused Ruff/format checks passed and CI is
running).
- [x] I have made corresponding documentation changes.
- [x] My focused local checks generate no new warnings.
- [x] I have added/backported tests that prove the fixes are effective.
- [x] I have included changelog fragments for every touched source
package.
- [x] Contributor attribution is preserved from the source changes.

---------

Signed-off-by: Kelly Guo <kellyg@nvidia.com>
Co-authored-by: Rebecca Zhang <168459200+rebeccazhang0707@users.noreply.github.com>
Co-authored-by: Kai Pei <2047767028@qq.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: ooctipus <zhengyuz@nvidia.com>
Co-authored-by: Mustafa H <34825877+StafaH@users.noreply.github.com>
Co-authored-by: Maximilian Krause <99733341+maxkra15@users.noreply.github.com>
Co-authored-by: Ege Sekkin <72572910+nvsekkin@users.noreply.github.com>
Co-authored-by: Antoine RICHARD <antoiner@nvidia.com>
Co-authored-by: Piotr Barejko <pbarejko@nvidia.com>
Co-authored-by: nvsekkin <esekkin@nvidia.com>
Co-authored-by: camevor <camevor@nvidia.com>
Co-authored-by: hujc <jichuanh@nvidia.com>
Co-authored-by: marcodiiga <1969828+marcodiiga@users.noreply.github.com>
Co-authored-by: Alesiani Marco <marcodiiga@users.noreply.github.com>
Co-authored-by: Diego Ferigo <dferigo@rai-inst.com>
Co-authored-by: Matthew Taylor <mataylor@nvidia.com>
Co-authored-by: Yuchen Deng <yuchendeng@nvidia.com>
Co-authored-by: Daniela Hase <116915287+daniela-hase@users.noreply.github.com>
Co-authored-by: Henry Hu <yukanghu7@163.com>
Co-authored-by: r-schmitt <139814266+r-schmitt@users.noreply.github.com>
ooctipus added a commit to vidurv-nvidia/IsaacLab that referenced this pull request Sep 19, 2026
Update develop through fa3687d. Remove the raw-target quaternion normalization and extra nullspace damping left by 20ea23a, restoring the test behavior retained by the merged fix in isaac-sim#7868. Preserve the schema fragment migration and remove the superseded OSC test changelog fragment.

Validation: 21 OSC tests, 17 mesh-fragment tests, five CUDA-runtime tests, and uv run isaaclab -f passed.
@ooctipus
ooctipus deleted the fix/osc-test-feedback branch September 19, 2026 19:20
ooctipus added a commit that referenced this pull request Sep 20, 2026
…7904)

## Description

Backports ten `develop` PRs to `release/3.0.0`, including the PyTorch
2.12/CUDA 13.0 upgrade, the near-singular OSC fix, and the complete
cfg-first cloning and schema-fragment migrations.

The audit covers September 17, 2026 00:00 UTC through `develop` commit
`4269c29a39e463dd07c74a9704d890d7a5c01e08` (September 19, 23:55 UTC),
against release base `970d21ef48057a2efc3d749c0f2abab8e7e0c30e`.

| Source PR | Included change |
| --- | --- |
| #7777 | Refresh Newton solver transforms after fixed-root pose writes
|
| #7608 | Make direct cloning cfg-first, including assets, callers,
templates, tests, and migration documentation |
| #7895 | Apply the existing CI exclusion for Franka cable/cloth/soft
kitless rendering tests |
| #7872 | Load the CUDA runtime matching PyTorch for Newton graph
capture |
| #7831 | Check authored Newton mesh-collision schema tokens correctly |
| #7834 | Build documentation with the extra that provides Sphinx |
| #7838 | Migrate maintained consumers to physics schema fragments,
including readers and overrides |
| #7840 | Consistently name Isaac Lab 3.1 as the deprecation removal
release |
| #7903 | Handle near-singular operational-space control |
| #7674 | Upgrade PyTorch to 2.12, TorchVision to 0.27, and the CUDA
stack to 13.0, including installer, wheel, and cuRobo compatibility
changes |

All 305 changed non-lock files match the audited `develop` snapshot
exactly. The remaining file, `uv.lock`, matches that snapshot after
substituting the package versions from the release branch's existing
manifests; all 418 package entries were checked. Existing release
package versions, compiled changelogs, documentation routing, and OVRTX
compatibility code remain intact, including the #7880 revert. Source
commit hashes are recorded in the commit messages.

### Already present

#7866, #7870, #7867, #7869, and #7868 were included in #7851; #7819 was
backported by #7884; #7883, #7885, and #7887 were backported by #7889.
#7874, #7888, #7773, #7886, and #7042 were also already present.

The three automatic changelog/version compilation commits are excluded;
the release branch compiles its own fragments and maintains its own
versions.

## Type of change

- Release integration/backport
- Bug fixes and documentation updates
- Breaking changes already accepted for the 3.0 cloning and schema
migrations; their migration guidance and changelog fragments are
included

## Validation

- `uv run --no-sync isaaclab -f`: all checks passed using the existing
shared environment and the release base for changelog validation.
- Focused cloning, OSC feature, schema parity/shim, CLI, and task
configuration suite: **283 passed**.
- Newton fixed-root updates and CUDA runtime/capture regressions: **14
passed**.
- Complete OSC simulation suite: **21 passed**.
- Mesh-collision fragments: **17 passed**; cfg-first cloning
integration: **16 passed**; scene construction/ownership: **7 passed**.
- The above **358 tests passed with PyTorch 2.11**.
- With **PyTorch 2.12/CUDA 13.0**: installer/metadata/Docker tests **91
passed**; CPU/GPU OSC feature tests **17 passed**; complete OSC
simulation suite **21 passed**.
- `uv lock --check`: passed; release package manifests match their
lockfile entries.
- Repository skill validation: **23 skills passed**.
- File-by-file source audit, release-metadata preservation checks, and
`git diff --check`: passed.

Local simulation used RTX 5090 and Isaac Sim 6.0.1 with PyTorch 2.11 and
2.12. The full Docker image build and fresh installation matrix were not
rerun locally for this backport. Release CI uses its pinned Isaac Sim
6.1 image and remains the authoritative full integration check.
Kit-dependent suites run in separate processes to avoid loading
standalone USD before Kit.

## Release backport

Not applicable: this PR targets `release/3.0.0` directly.

## Checklist

- [x] Ran the complete pre-commit command.
- [x] Included the original migration documentation, regression tests,
and changelog fragments for every changed package.
- [x] Preserved source attribution and recorded every source commit.
- [x] Preserved the release branch's versions and existing
release-specific changes.

---------

Co-authored-by: vidurv-nvidia <vidurv@nvidia.com>
Co-authored-by: ooctipus <ooctipus@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants