Skip to content

[Camera] Remove remaining hot-path host syncs and overhead [7B/7] - #8105

Merged
ooctipus merged 24 commits into
isaac-sim:developfrom
StafaH:perf/camera-perf-07b-hot-path
Sep 30, 2026
Merged

ooctipus merged 24 commits into
isaac-sim:developfrom
StafaH:perf/camera-perf-07b-hot-path

Conversation

@StafaH

@StafaH StafaH commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Part 7B of the camera performance series, based on the latest develop (including #8039, #8042, and #8087). A sweep of the per-step hot path found remaining host-device synchronizations and code that costs more than it gives. This PR fixes the verified findings: it removes the synchronizations, launch overhead, and dead or over-defensive code, and fixes the verified bugs.

Profiling on develop motivated this. The GPU sat idle 20–27% of each step, because the CPU blocked on synchronizations and then launched hundreds of small kernels. On Isaac-Velocity-Rough-AnymalD (4096 environments), for example, the CPU was blocked 12.8 ms of a 25.4 ms step.

Changes

Host synchronizations removed

  • Episode logging: the command, termination, and curriculum managers, survival_success_rate, the pose commands, and task success rates (Cabinet, Locomotion, Lift deformable/cable, Fourbar) log 0-d device tensors instead of reading back .item()/.cpu() on every reset. RewardManager already logged tensors, and the RL wrappers accept them. Curriculum states are logged as copies, because rsl_rl keeps episode infos for a whole rollout. The inspection accessor get_active_iterable_terms() still returns Python scalars.
  • Velocity command: heading control is branchless (torch.where) instead of is_heading_env.nonzero() every step. Output is identical.
  • Event ranges: core and Lift event terms use sample_uniform_from_ranges() to draw named components directly. One private, bounded bounds cache replaces both sets of tensor-building helpers; it is keyed by values, resolved device, and dtype. Curriculum edits take effect on the next call, and every call draws fresh samples in the existing order.
  • Ray caster: reset() no longer uploads its range tensor or samples drift when the drift ranges are zero (the default). In rough locomotion this reset runs almost every step. When drift is disabled after sampling, reset clears the selected environments’ old drift.
  • DiffIK: the if ee_quat.norm() != 0 guard in apply_actions, which synchronized every physics substep, is now a torch.where with the same all-or-nothing behavior.
  • Direct tasks: Reorient (about 14 host-list indexes per substep), Cabinet, and Locomotion store joint and body indices as device tensors, precompute static joint limits and default positions, and use the non-deprecated actuators.target_command API.
  • Other task terms:
    • Locomotion walk_target_w caches its device target (it synchronized 4 times per step).
    • Reorient's EpisodeErrorRecorder uses torch.where with NaN reductions instead of boolean masks.
    • Lift reset events share the named-range sampler with the core event terms.
  • Joint limits: Newton, PhysX, and OvPhysX position-limit writes reuse the clamping counter and read it back only when the resulting message would be logged.

Launch overhead

  • RewardManager: each term writes its weighted value into a step-reward matrix column, followed by one sum and one batched episode-sum update. Reset is one mean and one fill. With 20 terms at 4096 environments, compute drops from 121 to 23 kernels (471 → 152 µs) and reset from 80 to 4 kernels (504 → 49 µs).
  • ObservationManager: the built-in noise functions (constant_noise, gaussian_noise, uniform_noise) are out-of-place, so the pre-noise clone and the second clone are skipped. Safe built-in noise functions are identified during _prepare_terms; custom noise functions, noise models, and modifiers still clone, including when a caller replaces a noise function after setup. The noise sampling order is unchanged.
  • CircularBuffer: one staged shift replaces K−1 slot copies for frames up to 2 MiB. Image-sized frames keep the slot loop, which is faster for them.
  • Lift contacts: one stacked norm replaces per-sensor norms. contacts() runs 5 times per step in Kuka Allegro, and this removes about 80 kernels per step.
  • ANYmal symmetry augmentation: one cached permutation and sign tensor per transform, bitwise-equal to the original, is 1.56 → 0.15 ms per call at 4096 environments.
  • RTX renderer: normals, motion vectors, rgb_hdr, and simple shading no longer copy each frame into a contiguous channel slice before reshaping.
  • Smaller changes:
    • Reorient reads its command slices directly instead of concatenating them.
    • sb3 computes dones on the host side.

Overengineered code removed

  • NewtonManager.step() ran the solver-internals reset every physics step. forward() already performs it when kinematics are dirty, so the extra call only launched with an all-False mask.
  • WrenchComposer.reset() returns early when inactive. Its buffers are already zero, and this removes about 7 memsets per rigid object per physics step (21 µs → 0.07 µs per call).
  • SensorBase: sensors with debug_vis=True no longer refresh on every physics substep. That refresh existed because debug callbacks once read private data. The callbacks now refresh the outdated buffers themselves. The original eager update was introduced in commit 4565902f5 to make visualization work without a data read; it did not pump rendering.
  • VisualizationMarkers.visualize() returns early when no backend is active.
  • Dead code: removed the Newton Warp renderer's output copy loop (every output is a zero-copy alias), the impossible PhysX ray-caster guard, and the redundant Lift body_state_b index conversion (redundant since Cache scene selections with automatic Torch dispatch #8087).

Bug fixes

  • Newton body_com_acc_w: the finite difference divided by the physics timestep even when an update spans several physics steps (Newton owning the decimation loop). That made the acceleration, and body_lin_acc_l2, decimation× too large. It now uses the elapsed time, matching joint_acc. The OvPhysX rigid-object implementation uses the same elapsed-time formula. PhysX and OvPhysX articulations read acceleration from the backend, so they do not need this finite-difference change.
  • reset_root_state_uniform built its range tensors in __init__ and ignored call-time pose_range/velocity_range, so curriculum updates were silently dropped.

Investigated and not included

  • Step reset, command resample, and interval-event nonzero: these are now the only synchronizations left in Isaac-Velocity-Rough-AnymalD. Each converts a mask into variable-length environment IDs consumed across the reset, command, and event APIs. Removing them requires a mask-based API redesign and is left for a separate design.
  • Needs a Kit trace before changing: the PhysX IMU/PVA per-update COM upload, PhysX contact-sensor per-substep fetches, and the Fabric render synchronizations. They rely on PhysX/Fabric semantics we have not measured. A similar "pageable upload synchronizes" assumption proved false for the Newton sensor flags.
  • Deferring the MPM sparse-grid status check: a solver reset clears the status, so a deferred check could lose capacity errors, and no test covers this path.
  • RMPflow guard: it already runs on the host, so removing the guard gains nothing. The joint_actions allocation is needed for LEAPP export tracing.
  • Lower confidence, left out: collection mask-write kernels and batching model-change notifications.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • Behavior changes with migration notes in the changelog fragments:
    • Episode log values are 0-d tensors; use float() or .item() where a Python number is needed.
    • Reorient direct index attributes (actuated_dof_indices, finger_bodies, finger_wrench_bodies) and resolve_actuated_tendons ids are tensors.
    • EpisodeErrorRecorder reports NaN for an empty selection after its first update.
    • Custom sensors should refresh their buffers inside _debug_vis_callback.

Performance

RTX PRO 6000 Blackwell Workstation Edition (driver 595.91.07); PyTorch 2.12.0+cu130; Warp 1.17.0; Newton 1.6.0; GPU otherwise idle. develop at 60bc09f17 compared with this PR on that base, with the same uv.lock. The branch has since merged develop at f25d3f09b (#8042, #8090, #8102). The numbers below were measured before that merge; #8042 changes the Kuka camera observations and #8090 touches the simulation context, so they have not been re-measured on the merged branch or after review fixes. Static Lift success markers now resend positions to support late backend activation; the earlier marker-index-only optimization is not included. Runtime benchmark (isaaclab benchmark runtime), Newton/MJWarp, seed 42, no visualizer. Three fresh processes per revision, run in alternating order. Figures are the mean step time across processes; every workload's per-process ranges are disjoint.

Workload develop This PR Change
Isaac-Ant, 4096 envs 3.369 ms 2.629 ms −22.0% (+28.1% throughput)
Isaac-Open-Drawer-Franka-Direct, 4096 envs 11.106 ms 10.100 ms −9.1% (+10.0%)
Isaac-Reorient-Cube-Allegro-Direct, 4096 envs 27.114 ms 25.221 ms −7.0% (+7.5%)
Isaac-Velocity-Rough-AnymalD, 4096 envs 23.078 ms 22.573 ms −2.2% (+2.2%)
Isaac-Lift-KukaAllegro-Camera, 256 envs, presets=duo_camera,rgb128, Newton renderer 12.396 ms 12.202 ms −1.6% (+1.6%)

Per-process step times (develop → PR):

  • Ant: 3.367, 3.359, 3.382 → 2.639, 2.638, 2.611 ms.
  • Drawer: 11.113, 11.102, 11.104 → 10.099, 10.100, 10.100 ms.
  • Reorient: 27.136, 27.136, 27.070 → 25.217, 25.250, 25.197 ms.
  • AnymalD: 23.081, 23.050, 23.103 → 22.551, 22.568, 22.599 ms.
  • Camera: 12.389, 12.374, 12.424 → 12.373, 12.119, 12.113 ms.

Host synchronizations per env.step(), attributed to call sites with PyTorch sync-debug mode over 170 steps:

Task develop This PR Remaining
Isaac-Velocity-Rough-AnymalD 6.75 3.00 step reset, command resample, and interval-event nonzero
Isaac-Lift-KukaAllegro-Camera 2.11 2.04 step reset and command resample nonzero; curriculum difficulty .item() (0.04 per step)

PyTorch labels sync-debug mode a prototype that does not detect every synchronization, and it does not see synchronizations issued by Warp, so these counts are lower bounds.

Validation

  • The shared sampling refactor passed 257 tests across the existing math, root-state event, and Lift configuration suites. The existing event test now checks exact seeded samples, key order, missing axes, fresh draws, range edits, and warm-cache CUDA synchronization. Eight standalone equivalence checks also passed across CPU/CUDA, float32/float64, and flat/nodal batch shapes. No new tests or test files were added.
  • Before the shared-sampler refactor, 205 focused tests passed, with each file run in its own process: curriculum, observation, reward, and termination managers; heading commands and mutable event ranges; DiffIK; wrench composition; ray-caster and sensor-base behavior; marker visualization; Lift and reorientation helpers; and the existing joint-limit tests on both PhysX and OvPhysX, on CPU and CUDA. Kit tests used the isaacsim extra, and OvPhysX used ovphysx with separate CPU/GPU processes.
  • Regression checks fail with the old curriculum accessor, stale drift, and static marker behavior restored. Mutations also turn the retained tests red for custom-noise aliasing, eager debug-visualization refreshes, partial reward reset corruption, and missing reorientation NaN statistics.
  • Applied the test-audit skill: seven standalone tests were consolidated into existing reward, wrench, reorientation, marker, ray-caster, sensor-update, and Lift tests. Synchronization tests run on CUDA instead of repeating the same bookkeeping on CPU. No new test functions or files were added in the review-fix commit.
  • Retained unique heading, curriculum snapshot, mutable-range, and ANYmal symmetry coverage: there is no equivalent existing owner test. The symmetry test checks the new permutation tables against the previous independent index-assignment formulation.
  • uv run isaaclab -f passed, including changelog checks against the actual PR base, and git diff --check passed.
  • The full test suite and training smoke runs have not been repeated on this revision; benchmark figures above describe the earlier measured revision.

Release backport

  • Backport this pull request to the active release branch after it merges into develop

Checklist

  • I have read and understood the contribution guidelines.
  • I have run the pre-commit checks with uv run --frozen isaaclab -f.
  • I have made corresponding changes to the documentation.
  • I have added tests that prove the behavior and checked the regressions against the unfixed code.
  • I have added a changelog fragment for every touched source package.
  • My name already exists in CONTRIBUTORS.md.

- Log command, termination, curriculum and success-rate episode metrics as
  device scalars instead of host floats.
- Select heading-controlled envs in UniformVelocityCommand with torch.where
  instead of nonzero().
- Cache event range tensors by value, and make reset_root_state_uniform
  honor the ranges passed at call time.
- Replace the DiffIK zero-quaternion guard with a sync-free torch.where.
- Batch RewardManager per-term updates and resets over a term matrix.
- Skip defensive copies around built-in noise and delay buffers.
- Shift small CircularBuffer frames in two kernels regardless of length.
- Check event modes via dict membership in the env step.
- Skip ray-caster drift sampling on reset when drift ranges are zero and
  upload ray-cast drift ranges only when they change.
- Stop eager per-step refreshes for debug-visualized sensors; debug
  callbacks now refresh outdated buffers lazily before drawing.
- Return early from VisualizationMarkers.visualize when no backend is active.
- Drop per-frame channel-compacting copies in the Isaac RTX renderer by
  indexing padded annotator tiles with their source channel count.
- Remove the Newton Warp renderer read_output copy loop (outputs alias the
  camera buffers) and an unreachable PhysX ray-caster guard.
Store direct-task joint, body, and tendon ids as device tensors, precompute static limits and default poses, use the actuator target-command API, log success metrics as 0-d tensors, compute Lift contact terms from one stacked norm, cache the ANYmal symmetry permutations and constant tensors, and send only marker indices for static success markers.
@StafaH

StafaH commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Sep 28, 2026
@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 28, 2026
…perf/camera-perf-07b-hot-path

# Conflicts:
#	source/isaaclab_tasks/test/core/test_lift_env_cfg.py
@StafaH

StafaH commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@greptile-apps

greptile-apps Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Performance optimization removing host synchronizations from hot paths.

The PR should not merge until resetting a ray caster after drift is disabled clears its previously sampled offsets.

Findings

  1. P1 Disabled drift remains active ▶
  2. P2 Late markers lack positions ▶

Summary

The PR removes device readbacks and repeated work from camera-adjacent simulation hot paths, batches reward accounting, and updates sensor, backend, and task behavior.

  • Fixes elapsed-time scaling for body acceleration and makes root-state reset ranges respond to call-time changes.
  • Adds regression and equivalence tests for the affected managers, sensors, renderers, and tasks.

Reviews (1) · Last reviewed commit: "Merge branch 'perf/hot-path-7b-tasks' in..."

Comment on lines +115 to +116
if not (sample_drift or sample_ray_cast_drift):
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Disabled drift remains active

If a ray caster’s drift range changes from nonzero to zero, reset() leaves the previously sampled drift in its buffer. Later ray casts still use that offset, so disabling drift does not stop it from perturbing sensor measurements. Clear the selected drift buffer when its range becomes zero.

Knowledge Base Used: Simulation, rendering, and sensors

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ec65628: when a drift range becomes zero, reset() now zeroes the corresponding buffer instead of skipping it (the skip only applies while nothing has ever been sampled).

Comment on lines +154 to +157
if self._static_success_vis_pos_w is not None:
# positions and environment ids were set at construction; resending them copies to the host
self.success_visualizer.visualize(marker_indices=success.int())
return

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Late markers lack positions

If the lift command is created before a marker backend is active, its one-time position update is discarded. When a backend becomes active later, these calls send only marker_indices, so the success markers never receive their per-environment positions. Resending the positions on first backend activation would preserve the markers in that case.

Knowledge Base Used: Simulation, rendering, and sensors

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ec65628: the static-marker shortcut is removed, so the success markers always receive their positions and environment ids on every visualize call and a late-activating backend gets them too.

@isaaclab-review-bot isaaclab-review-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isaac Lab Review Bot

The hot-path optimizations are broadly coherent, but CurriculumManager.get_active_iterable_terms() now returns cloned tensors while its public type contract still promises floats. Align this accessor’s implementation, annotation, documentation, and consumers before merge.

  • Design and architecture: The device-resident logging and caching strategy is consistent with the performance goal. The remaining concern is that a reset-path synchronization optimization was also applied to a public inspection accessor without defining the resulting tensor-valued contract.
  • API: CurriculumManager.get_active_iterable_terms() is declared as returning Sequence[tuple[str, Sequence[float]]], but changed tensor states from Python floats to 0-d tensors. Either retain .item() in this accessor or explicitly update its annotation, documentation, changelog guidance, and affected consumers.
  • Implementation: At curriculum_manager.py:165 and the equivalent scalar-state path, detach().clone() deterministically preserves tensor values rather than producing the floats promised by the signature. The implementation and public contract therefore need to be reconciled.

Minor fixes needed. Posted 1 actionable finding inline.

Automated review; human maintainers own approval decisions.

Comment thread source/isaaclab/isaaclab/managers/curriculum_manager.py Outdated
Comment thread source/isaaclab/isaaclab/envs/mdp/commands/velocity_command.py
Comment thread source/isaaclab/isaaclab/envs/mdp/events.py Outdated
Comment thread source/isaaclab/isaaclab/sensors/sensor_base.py
Comment thread source/isaaclab/isaaclab/managers/curriculum_manager.py Outdated
Comment thread source/isaaclab/isaaclab/managers/observation_manager.py Outdated
Comment thread source/isaaclab/isaaclab/managers/observation_manager.py Outdated
Comment thread source/isaaclab_tasks/isaaclab_tasks/core/lift/mdp/events.py Outdated

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're allocating new tensors here, we could avoid that if we pre-allocated

"""Per-environment contact force magnitude [N] of a single-body, single-filter contact sensor."""
force = sensor.data.normal_force_matrix_w.torch.view(num_envs, 3)
return torch.linalg.norm(force, dim=-1)
def _contact_force_mags(env: ManagerBasedRLEnv, sensor_names: Sequence[str]) -> torch.Tensor:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_contact_force_mag made more sense?

@StafaH

StafaH commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Sep 30, 2026
@StafaH
StafaH requested a review from myurasov-nv as a code owner September 30, 2026 17:22
@StafaH StafaH added the ci:run-docker Trigger the on-demand Docker and GPU CI workflow label Sep 30, 2026
@ooctipus
ooctipus merged commit cd91769 into isaac-sim:develop Sep 30, 2026
54 of 55 checks passed
@ooctipus
ooctipus deleted the perf/camera-perf-07b-hot-path branch September 30, 2026 18:46
kellyguo11 pushed a commit that referenced this pull request Sep 30, 2026
… uv docs CLI (#8118, #8105, #8179) (#8210)

## Summary

Backports three `develop` PRs to `release/3.0.0`, in merge order:

1. #8118 — towncrier changelog tooling
2. #8105 — camera hot-path cleanup [7B/7]
3. #8179 — uv CLI for Windows docs builds

The automatic backport failed for #8118 and #8105. For #8179, this
supersedes the bot's draft:

- #8209 — bot draft backport of #8179

Once merged, the nightly compiles `release/3.0.0` again: it skips
release branches without `tools/changelog/towncrier.toml`.

# Description

Each PR is cherry-picked with `-x`; every code file matches its source
patch. Release-specific changes:

1. #8118 (`ee99db6a820`)
- Each `CHANGELOG.rst` conflict resolved as the marker line above the
release entries.
- `uv.lock` unchanged: release's local package pins already match their
`pyproject.toml`.
- The bot failed here because `uv.lock` pushed its conflict context past
the 1.5M-character inference limit.
2. #8105 (`cd917691992`)
- Drops the `.github/workflows/build.yaml` hunk: it edits the
`performance-smoke` job, which `release/3.0.0` does not have. The bot's
validation rejected this omission.
- Splits its 10 old-format fragments into the new format with `cli.py`'s
`split_legacy`, keeping all 34 bullets; #8118's check rejects the old
format.
3. #8179 (`c1a8bfa50b5`)
   - Deletes `docs/make.bat`; release held an older copy.
   - Splits its fragment the same way.
- Release's `docs/_redirect/index.html` still hardcodes `./main/` (#7767
is not on this branch), so `isaaclab --docs_multi` writes that redirect
unchanged, as release's `make multi-docs` does.

## Type of change

- Release backport, carrying #8105's breaking change (`isaaclab` major
bump)
- New feature, bug fix and documentation update from the source PRs

## Validation

1. `cli.py check release/3.0.0` passes.
2. `tools/changelog` tests: 18 passed.
3. `pre-commit` over the range passes.
4. `cli.py compile` on a scratch copy of this branch:
- 9 packages bumped, including `isaaclab` 25.1.0 → 26.0.0 and
`isaaclab_tasks_experimental` 0.0.1 → 1.0.0
   - every fragment consumed, only its outputs staged
   - `uv.lock` changed in version lines only
5. Runtime coverage for #8105 and #8179 is left to CI.

## Release backport

This PR targets the release branch; no additional backport is needed.

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) on
the changed files
- [x] I have made corresponding changes to the documentation (carried
from the source PRs)
- [ ] My changes generate no new warnings
- [x] I have validated the changed behavior with focused coverage
(source PRs' tests carried unchanged)
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/` for every touched package (carried from the
source PRs; #8105 and #8179 converted to the new format)
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Co-authored-by: Mustafa H <34825877+StafaH@users.noreply.github.com>
kellyguo11 added a commit that referenced this pull request Oct 8, 2026
## Description

The ANYmal symmetry augmentation used by the `rsl_rl_with_symmetry`
agents hard-coded joint indices for the PhysX joint order. The velocity
tasks default to Newton, where ANYmal's native order is depth-first, so
every mirrored joint observation and action came from the wrong joint.

This PR resolves the left-right and front-back permutations and signs
from the robot's `joint_names`, using the ANYmal
`<L|R><F|H>_<HAA|HFE|KFE>` naming, and feeds them into the cached
permutation tables from #8105 (now keyed by joint order). The sign
conventions are unchanged: HAA is negated for left-right, and HFE/KFE
for front-back. With the PhysX order the tables equal the previous ones.

Resolving by name matches the ordering contract, where public order is
backend-native unless `joint_ordering` is set, and works for any
`joint_ordering`; ANYmal-B/C use the same joint names. Pinning
`joint_ordering="physx"` in the task would add a per-step reorder on
Newton and change the observation and action layout of existing Newton
checkpoints.

Fixes #8204

Verification on `Isaac-Velocity-Flat-AnymalD` and
`Isaac-Velocity-Rough-AnymalD`, on both PhysX and Newton, at the current
head:
- Wrong mirrored joints go from 12/12 to 0/12 on Newton, and stay at
0/12 on PhysX.
- The physical mirror check (base-frame feet of the mirrored
configuration against the mirrored feet) goes from 0.17-0.42 m to below
1e-5 m on Newton.
- `test_anymal_symmetry.py` keeps the PhysX reference check from #8105
and adds one check that the Newton order is mirrored like the PhysX
order with the joint columns reordered; it fails on develop and passes
with this change.
- The per-call path is unchanged: one cached gather and multiply per
transform.

## Type of change

- Bug fix (non-breaking change which fixes an issue)

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`uv run isaaclab --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have validated the changed behavior with focused coverage and
applied the [test-audit
criteria](https://github.com/isaac-sim/IsaacLab/blob/develop/skills/developer/test-audit/SKILL.md)
to any test changes
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/` for every touched package (do **not** edit
`CHANGELOG.rst` or bump package versions — CI handles that)
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Co-authored-by: Kelly Guo <kellyg@nvidia.com>
isaaclab-bot Bot pushed a commit that referenced this pull request Oct 8, 2026
## Description

The ANYmal symmetry augmentation used by the `rsl_rl_with_symmetry`
agents hard-coded joint indices for the PhysX joint order. The velocity
tasks default to Newton, where ANYmal's native order is depth-first, so
every mirrored joint observation and action came from the wrong joint.

This PR resolves the left-right and front-back permutations and signs
from the robot's `joint_names`, using the ANYmal
`<L|R><F|H>_<HAA|HFE|KFE>` naming, and feeds them into the cached
permutation tables from #8105 (now keyed by joint order). The sign
conventions are unchanged: HAA is negated for left-right, and HFE/KFE
for front-back. With the PhysX order the tables equal the previous ones.

Resolving by name matches the ordering contract, where public order is
backend-native unless `joint_ordering` is set, and works for any
`joint_ordering`; ANYmal-B/C use the same joint names. Pinning
`joint_ordering="physx"` in the task would add a per-step reorder on
Newton and change the observation and action layout of existing Newton
checkpoints.

Fixes #8204

Verification on `Isaac-Velocity-Flat-AnymalD` and
`Isaac-Velocity-Rough-AnymalD`, on both PhysX and Newton, at the current
head:
- Wrong mirrored joints go from 12/12 to 0/12 on Newton, and stay at
0/12 on PhysX.
- The physical mirror check (base-frame feet of the mirrored
configuration against the mirrored feet) goes from 0.17-0.42 m to below
1e-5 m on Newton.
- `test_anymal_symmetry.py` keeps the PhysX reference check from #8105
and adds one check that the Newton order is mirrored like the PhysX
order with the joint columns reordered; it fails on develop and passes
with this change.
- The per-call path is unchanged: one cached gather and multiply per
transform.

## Type of change

- Bug fix (non-breaking change which fixes an issue)

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/main/source/refs/contributing.html)
- [x] I have run the [`pre-commit` checks](https://pre-commit.com/) with
`uv run isaaclab --format`
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have validated the changed behavior with focused coverage and
applied the [test-audit
criteria](https://github.com/isaac-sim/IsaacLab/blob/develop/skills/developer/test-audit/SKILL.md)
to any test changes
- [x] I have added a changelog fragment under
`source/<pkg>/changelog.d/` for every touched package (do **not** edit
`CHANGELOG.rst` or bump package versions — CI handles that)
- [x] I have added my name to the `CONTRIBUTORS.md` or my name already
exists there

---------

Co-authored-by: Kelly Guo <kellyg@nvidia.com>

(cherry picked from commit abe7db4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:run-docker Trigger the on-demand Docker and GPU CI workflow infrastructure isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants