Skip to content

[Warp] Part 4/12: Share one per-environment Warp RNG state through WarpRng - #8783

Draft
hujc7 wants to merge 42 commits into
isaac-sim:mainfrom
hujc7:jichuanh/warp-rng-shared
Draft

hujc7 wants to merge 42 commits into
isaac-sim:mainfrom
hujc7:jichuanh/warp-rng-shared

Conversation

@hujc7

@hujc7 hujc7 commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Review Map

PR Status Depends on Exact changes
#8821 Part 1/12: shared id and mask helpers, mask writers without host sync — —
#8824 Part 2/12: PhysX tendon mask setters through the masks #8821 link
#6646 Part 3/12: mask-correct partial resets #8821 link
📌 #8783 Part 4/12: shared Warp RNG state #6646 link
#6607 Part 5/12: per-term capture, stable stepping parity #8783 link
#8228 Part 6/12: Warp commands, Rough tasks, live term changes #6607 link
#6582 Part 7/12: Shadow Warp twin, Allegro twin fixes #6607 link
#7012 Part 8/12: --video on the Warp frontend — —
#8856 Part 9/12: stable SceneEntityCfg Warp selections, capture utilities #8228 link
---
config:
  look: neo
  flowchart:
    curve: basis
---
flowchart TD
  P1("<b>#8821</b> · Part 1<br/>id/mask helpers")
  P2("<b>#8824</b> · Part 2<br/>PhysX tendon masks")
  P3("<b>#6646</b> · Part 3<br/>masked resets")
  P4("📌 <b>#8783</b> · Part 4<br/>WarpRng")
  P5("<b>#6607</b> · Part 5<br/>per-term capture")
  P6("<b>#8228</b> · Part 6<br/>commands, Rough")
  P7("<b>#6582</b> · Part 7<br/>Shadow twin")
  P8("<b>#7012</b> · Part 8<br/>--video, independent")
  P9("<b>#8856</b> · Part 9<br/>stable SceneEntityCfg, capture")
  P1 --> P2
  P1 --> P3 --> P4 --> P5
  P5 --> P6
  P5 --> P7
  P6 --> P9
  style P4 stroke-width:3px
Loading

Summary

Warp environments, MDP terms, managers, noise models, the ray caster and the Warp tasks draw from one per-environment RNG state per process instead of each allocating and seeding its own.
configure_seed reseeds that state in place, so CUDA graphs captured before reseeding keep drawing from it.

Description

Update (2026-10-26): merged #6646, now stacked on #8821, and through it main dae8b799146; this PR's own diff is unchanged.

Split out of #6646 (masked resets) so it is reviewed on its own. The code is 6ac6061a137 and 9328d981380 from #6646, re-applied unchanged; its changelog entries moved to this PR's fragments.

1. WarpRng

  1. isaaclab.utils.seed.WarpRng.state holds one wp.uint32 state per environment, one array per process. Environments call WarpRng.initialize(num_envs, device) at construction; configure_seed reseeds the existing array in place.
  2. Kernels copy state[env] into a local, draw from it and store it back, one thread per environment, so the next launch or graph replay draws new numbers.
  3. Unseeded Warp environments seed it from torch.initial_seed() instead of -1.

2. Consumers

The masked ray-caster drift resampling, the Warp MDP events, the command and event managers, the observation noise models, and the Cartpole, locomotion and reorient Warp tasks read WarpRng.state instead of their own arrays.

3. Breaking changes

  1. ManagerBasedEnvWarp.rng_state_wp is removed. Read WarpRng.state instead.

4. Evidence (MEASURED, desk RTX PRO 6000, Newton kitless)

  • On 3ddcf7e2a9f: the WarpRng, Warp MDP parity, Newton ray-caster, Warp observation manager and DelayBuffer tests pass (87).
  • Warp 1.17 on CPU and CUDA: a draw from a local copy advances that copy; a draw directly from state[env] draws from a temporary and leaves the array unchanged. Of the 22 kernels that read the state, 19 store the advanced local back; the 3 reorient kernels draw directly from state[env] and add 1 to it after each draw (pattern kept from main).
  • From the [Warp] Part 3/12: Reset only the masked environments and fix manager-based Warp environment construction #6646 review round (2026-10-01, older tree): Cartpole, Cartpole-Direct, Ant-Direct and Reorient-Allegro-Direct (16 envs) advance the shared state in 16/16 environments.

5. Known limitation

WarpRng.initialize allocates a new array and consumers read WarpRng.state at launch, so a second Warp environment built in the same process replaces the first one's state (read from code; Isaac Lab runs one simulation per process).

Type of change

  • New feature (non-breaking change which adds functionality)
  • Breaking change (existing functionality will not work without user modification)

Release backport

  • Backport this pull request to the active release branch after it merges into main

Screenshots

Not applicable.

Checklist

Docker and GPU tests run on demand. Push the commits you want tested, then
comment run-ci on the pull request.

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab --format
  • I have made corresponding changes to the documentation: the Warp migration guide points at WarpRng.state
  • My changes generate no new warnings
  • I have validated the changed behavior with focused coverage and applied the test-audit criteria to any test changes
  • I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump package versions — CI handles that)
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

A mask-only Articulation.reset called ActuatorCollection.reset(None), which
cleared delay buffers, network history, and Newton-native actuator state for
every environment. Newton rigid objects and collections dropped the mask
before resetting their wrench composers, clearing every environment's
persistent wrench.

ActuatorCollection.reset now takes env_mask. Isaac Lab actuator models still
reset by index, so the collection converts the mask to indices once for all
groups, and skips the conversion when no group carries reset state. Native
actuator state resets from the mask through a new
ActuatorControl.reset_native_actuators_mask hook; Newton and OVPhysX pass the
mask to the Newton actuator adapter, and the base default converts to indices
so backends that only implement the index hook stay correct.
InteractiveSceneWarp.reset passed env_ids=None to deformable objects, surface
grippers, and sensors on a masked reset, so every environment's sensor state
was cleared on each partial reset, and it never reset cable objects.

Forward the mask to every entity family in the stable scene's order. Surface
grippers use their reset_mask method. Cable objects take env_mask like the
other scene assets, so the scene does not convert the mask for a reset that
holds no state.
RigidObjectCollection.write_body_*_to_sim_mask converted masks to indices
with torch.nonzero before delegating to the index writers, so the mask
writers synchronized the host and could not be captured in a CUDA graph,
although their docstrings name them as the graph-capable path. They now
launch masked kernels over the full environment-body grid, and body_ids
selections resolve to a mask through a preallocated scratch buffer.

BaseRayCaster.reset compacted a mask with nonzero before resampling drift.
A masked reset now resamples every environment and keeps the new values
where the mask is set; index and slice selections are unchanged.
isaac-sim#8079 added ObservationGroupCfg.history_order and taught the stable
observation manager to skip it as a group setting. The Warp manager's skip
list was not updated, so it parsed the field as a term and raised a
TypeError for every observation group, and no manager-based Warp
environment could be constructed.

Skip history_order like the other group settings. The Warp manager raises
NotImplementedError for any term with history, and history_order only
takes effect with history, so a time-ordered group cannot be silently
reordered wrong.
The fixed-tendon set_*_mask methods and
write_fixed_tendon_properties_to_sim_mask converted masks to indices with
torch.nonzero and delegated to the index writers, so they synchronized the
host and could not be captured in a CUDA graph.

Write the masked cells in-kernel instead: scalar and full data go into the
staging buffers through masked kernels, and the property writer copies the
masked cells of each staging buffer into its simulation binding. The mask
compaction helpers have no remaining callers and are removed.
A masked Articulation.reset on the Newton-native actuator path allocated a
per-DOF mask for every actuator on each call, and a masked ray-caster
reset sampled drift with torch ops and uploaded its ranges, so neither
could run inside a CUDA graph captured on Warp's stream.

NewtonActuatorAdapter preallocates its per-DOF reset masks. BaseRayCaster
resamples masked drift in a Warp kernel from per-environment random states
seeded from torch's global generator at initialization, with the ranges
passed as launch arguments; index and slice resets keep the torch path.
The Newton fixed-tendon position-target setters raised a host-side dirty
flag, and write_data_to_sim scattered the buffered targets into MuJoCo's
controls only while it was set. A setter recorded in a CUDA graph raised
the flag once, at capture time, so later replays updated the target buffer
on the device but never reached the solver.

Write the buffered targets on every step whenever the articulation has
tendon actuators, and drop the flag from the Newton setters. The write
costs about 60 us per call on Shadow Hand; gating it on a device-side flag
would need Newton's control-array mapping re-implemented in a kernel.
…buffers

ActuatorBase.reset, ActuatorControl.reset_native_actuators and DelayBuffer take an explicit env_mask or batch_mask, replacing reset_native_actuators_mask. DelayedPD validates its delay range at construction because the masked path skips the host-side check.
PhysX articulation, rigid object and rigid object collection resets pass env_mask to their actuators and wrench composers instead of resetting every environment.
Environments initialize WarpRng once at construction and configure_seed reseeds it in place. The ray caster, experimental managers, events, noise models and Warp tasks read WarpRng.state instead of keeping their own copies; rng_state_wp is removed.
…nt mask

Articulation resets now forward env_mask to their actuators. The Multirotor thruster collection and the test_reset actuator stubs still took only env_ids and raised TypeError; the collection now resets the masked environments' thrusters.
…-mask-native

# Conflicts:
#	docs/source/concepts/warp_environment_migration.rst
#	source/isaaclab_newton/isaaclab_newton/assets/articulation/actuator_control.py
#	source/isaaclab_newton/isaaclab_newton/assets/articulation/articulation.py
#	source/isaaclab_newton/test/assets/test_articulation.py
#	source/isaaclab_newton/test/assets/test_newton_actuators_newton.py
#	source/isaaclab_newton/test/assets/test_rigid_object.py
#	source/isaaclab_newton/test/sensors/test_newton_raycast_sensor.py
Revert 6ac6061 and 9328d98: the process-wide WarpRng state moves to its own pull request stacked on this one. The ray caster, Warp environments, MDP terms, managers, noise models and Warp tasks keep their own RNG arrays here, as on main.
Environments initialize WarpRng once at construction and configure_seed reseeds it in place. The ray caster, experimental managers, events, noise models and Warp tasks read WarpRng.state instead of keeping their own copies; rng_state_wp is removed.

Re-applies 6ac6061 and 9328d98, split out of the masked-reset pull request.
@hujc7

hujc7 commented Oct 22, 2026

Copy link
Copy Markdown
Collaborator Author

run-ci

@isaaclab-bot isaaclab-bot Bot added ci:run-docker Trigger the on-demand Docker and GPU CI workflow and removed ci:run-docker Trigger the on-demand Docker and GPU CI workflow labels Oct 22, 2026
hujc7 added 9 commits October 26, 2026 02:48
…pers

resolve_ids resolves ids or a boolean mask to ids on a device, as resolve_1d_mask does to a mask. resolve_1d_mask takes an optional scratch mask, copies a Warp mask on another device, and accepts a boolean torch mask on its device inside a graph capture. The experimental package now re-exports it instead of keeping a copy.
…red helpers

SensorBase, Camera and WrenchComposer resolve env_ids, env_mask and body_ids through resolve_ids and resolve_1d_mask instead of their own copies. The test doubles that build these objects without their initializer get the env masks a real sensor creates.
The nodal mask writers of BaseDeformableObject, which the OVPhysX deformable object inherits, and of the PhysX deformable object called wp.nonzero on a Warp array, which raised a RuntimeError. They now resolve the mask to ids with resolve_ids and call the index writers.
…sk writers through masks

The Newton assets resolve ids and masks through resolve_ids and resolve_1d_mask. The RigidObjectCollection mask writers and the fixed-tendon mask setters write through the masks instead of converting them to ids, which waited for the GPU on every call; the joint-coordinate writes reuse a scratch mask; one generic masked 2-D write kernel replaces two copies.
The OVPhysX assets resolve env, body, joint and tendon selections through resolve_ids and resolve_1d_mask instead of their own copies.
The PhysX assets resolve env, body, joint and tendon selections through resolve_ids and resolve_1d_mask instead of their own copies. The sensor test doubles get the env masks a real sensor creates.
…-native

isaac-sim#8821 owns the Newton mask writers: the collection write_body_*_to_sim_mask methods,
the fixed-tendon mask setters and their shared masked-copy kernel. This branch takes
its version of those and keeps only the reset-mask plumbing and the tendon-target
capture fix in the shared files.
…-shared

# Conflicts:
#	source/isaaclab/isaaclab/sensors/ray_caster/base_ray_caster.py
Writing the buffered targets every step cost 2-4% physics step time for every
articulation with tendon actuators. The only Warp caller, the Shadow Hand twin,
sets its targets outside the captured stage, so the host flag still sees them.
@hujc7 hujc7 changed the title [Warp] Part 2/6: Share one per-environment Warp RNG state through WarpRng [Warp] Part 4/8: Share one per-environment Warp RNG state through WarpRng Oct 26, 2026
@hujc7 hujc7 changed the title [Warp] Part 4/8: Share one per-environment Warp RNG state through WarpRng [Warp] Part 4/12: Share one per-environment Warp RNG state through WarpRng Oct 27, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant