Skip to content

[Warp] Cleanup Part 5.1: Speed up launches outside graph capture with replay - #6551

Closed
hujc7 wants to merge 5 commits into
isaac-sim:developfrom
hujc7:jichuanh/warp-launch-record-replay
Closed

[Warp] Cleanup Part 5.1: Speed up launches outside graph capture with replay#6551
hujc7 wants to merge 5 commits into
isaac-sim:developfrom
hujc7:jichuanh/warp-launch-record-replay

Conversation

@hujc7

@hujc7 hujc7 commented Jul 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

1. Summary

  • isaaclab: adds owner-scoped WarpLaunchCache, eager/replay A/B mode, debug pointer/layout validation, and a reproducible mixed-kernel benchmark.
  • isaaclab_experimental: converts persistent direct, manager, and MDP launch sites while retaining eager execution for transient allocations.
  • isaaclab_newton: adds default-off replay for validated pointer-stable articulation, rigid-object, derived-state, and actuator paths.
  • isaaclab_tasks_experimental: converts Cartpole, locomotion, Allegro in-hand, and manager-task hot paths.

The counterbalanced actual-environment grid measured Ant and Allegro Direct at
256, 1,024, 4,096, and 16,384 environments on four L40s:

Case Continuous-throughput result
Ant, 256 environments 1.06372x (+6.372%, descriptive 95% CI 1.03690–1.09124)
Ant, 1,024–16,384 environments effectively neutral once GPU busy reached 100%
Allegro, all four scales effectively neutral; GPU busy was already 100% at 256 environments

Per-step-synchronized diagnostics exposed roughly 65–106 us/step of removed
host launch overhead. The normal pipeline hides that saving behind queued GPU
work in saturated cases. The isolated launch ablation independently confirms
that graphs remain fastest for capture-safe regions and replay removes most
Warp Python marshalling from persistent uncaptured calls.

2. Design notes

  • Capture remains the primary fast path; replay targets Torch, host-control, visualization, sensor, and lazy-state gaps around legal graph islands.
  • The first call records and executes; later calls replay. Owners control lifetime and invalidation, with no caller warm-up branches, singleton, monkeypatch, or production ctypes path.
  • Recorded arguments require persistent storage and static scalar values; dimension changes use Warp's supported set_dim() path.
  • Ant replayed 7 of 20 non-noop launches per RL step; Allegro replayed 10 of 43. Remaining Newton forward, acceleration, reset, and actuator-telemetry launches explain the next optimization opportunities.
  • Runtime manager-term curriculum changes invalidate both graph and launch caches before the updated term is reused.

3. Test plan

  • 195 focused tests with replay debug validation.
  • Curriculum invalidation regression probe failed before the fix and passed after it, including the no-change path.
  • Direct Cartpole eager/replay training: five iterations, 20,480 samples per arm, exact learning-series and final-state equality.
  • Four-replica L40 application controls for natural partial-capture Allegro and Direct Cartpole, with exact replay-hit and graph-topology audits.
  • Eight-L40 isolated launch-path ablation for dispatch overhead; it is mechanism-only evidence and is not used to predict IsaacLab task throughput.
  • Counterbalanced Ant/Allegro Direct grid at 256–16,384 environments: 32/32 primary processes and 32/32 latency diagnostics passed topology, cache, telemetry, and source/configuration provenance validation.
  • Full pre-commit, docs build, four package changelog gates, and git diff --check.

4. Follow-ups

  • Productize replay for launch-bound uncaptured boundaries, while leaving capture as the preferred legal fast path.
  • Remove the Torch actuator graph break, preallocate actuator telemetry, deduplicate Newton forward/acceleration work, fuse adjacent kernels, and partition manager stages into larger legal graph islands.
  • Evaluate backend-owned full-decimation graphs in a separate semantic-change prototype; do not fold that behavior into this replay PR.

Type of change

  • New feature (non-breaking change)

Checklist

  • I have read and understood the contribution guidelines.
  • I have run the pre-commit checks with ./isaaclab.sh -f.
  • I have added the corresponding API documentation and changelog fragments.
  • My changes generate no new warnings.
  • I have added tests that prove the feature works.
  • I have added one changelog fragment for every touched package.
  • My name already exists in CONTRIBUTORS.md.

@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Jul 15, 2026
hujc7 added 4 commits July 16, 2026 01:39
@hujc7
Add Warp launch replay cache
f636482 
Provide an owner-scoped path for pointer-stable kernels that cannot stay inside CUDA graph regions. Preserve eager A/B mode and validate recorded storage contracts in debug builds.
@hujc7
Use replay in Warp frontend hot paths
24040ad 
Route persistent direct-task, MDP, and manager launches through environment-owned caches so uncapturable stages avoid eager marshalling on every kernel. Keep transient allocations eager.
@hujc7
Add opt-in Newton launch replay
41239d5 
Gate replay for validated pointer-stable articulation, rigid-object, derived-state, and actuator paths. Leave the feature disabled by default while broader workload coverage matures.
@hujc7
Add Warp launch mode benchmarks
fbe99e6 
Add a reproducible mixed-kernel matrix and completed-step timing controls to compare eager launch, recorded replay, dynamic setters, and graph composition across launch and thread scales.
@hujc7
hujc7 force-pushed the jichuanh/warp-launch-record-replay branch from 2c874fe to fbe99e6 Compare July 16, 2026 08:41
@hujc7 hujc7 changed the title Add Warp launch record-and-replay cache [Warp] Speed up launches outside graph capture with replay Jul 16, 2026
@hujc7
Invalidate Warp caches after curriculum changes
f75ec2b 
Runtime manager-term updates can replace scalar or callable inputs that are baked into captured graphs and recorded launches. Invalidate the owner caches only when a curriculum actually writes a new value.
@hujc7 hujc7 mentioned this pull request Jul 19, 2026
Draft
8 tasks done
@hujc7 hujc7 changed the title [Warp] Speed up launches outside graph capture with replay [Warp] Cleanup Part 5.1: Speed up launches outside graph capture with replay Jul 21, 2026
@hujc7 hujc7 mentioned this pull request Jul 21, 2026
Draft
6 of 8 tasks
@hujc7

hujc7 commented Jul 21, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #6611, which now absorbs everything from this PR on top of the simplified #6607 frontend: the owner-scoped WarpLaunchCache, the launch-mode benchmark harness, the default-off Newton deep-path replay (NewtonCfg.use_warp_launch_cache), and the curriculum cache-invalidation fix. Performance evidence and the mechanism ablation from this PR remain referenced in #6611's history.

@hujc7 hujc7 closed this Jul 21, 2026
This was referenced Jul 21, 2026
Open
Merged
Draft
kellyguo11 added a commit that referenced this pull request Jul 24, 2026
@hujc7 @kellyguo11
[WARP] Cleanup Part 1: Add frontend bridge for stable manager-based t…
40d888c 
…asks (#5504)

## Review Map

- A stacked PR’s page shows the cumulative diff of its dependency chain;
the "Exact changes" link pins that PR’s own commit range.
- SHA links can go stale after a branch update; the copy on #5504 is
refreshed first.

| PR | Status | Depends on | Exact changes |
|---|---|---|---|
| 📌 #5504 Part 1: frontend bridge (stable ids on the warp runtime) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/5504?label=)
| — | — |
| #6607 Part 2: mask-first frontend (draft) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6607?label=)
| #5504 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6607/changes/00ae695ac63e67ca678b91a837d3737529d2ec91..f68d03f40854b31a36063c5068f2933d32ea5db9)
|
| #6646 Part 3: scene reset mask-native + capture-ready (draft, base
rewritten — pending rebase onto #6607) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6646?label=)
| #6607 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6646/changes/4360622bde446c52c3fc85924639f9d1119d6e95..6d49e32f11e15894a4a0e49161a356ec79679ac6)
|
| #6580 Part 4: OVPhysX backend (draft) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6580?label=)
| #5504 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6580/changes/fc37fa65ad95d1ac6e3f32e721329ef715899d37..716cef20979ca6c578a6e595812291c7b1a282fc)
|
| #6611 Part 5: persistent-launch speedups (draft, absorbs #6551) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6611?label=)
| #6646 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6611/changes/6d49e32f11e15894a4a0e49161a356ec79679ac6..6e3da794215da03745a9bc8d35790e22262b099a)
|

## 1. Summary

- `--frontend {torch,warp}` on the shared RL CLI (train **and play**)
runs **stable** task ids on the warp runtime — no duplicated configs, no
parallel warp task registrations at all (end state pinned by test). The
`rlinf` integration constructs environments inside the external
framework and is not frontend-routable (known limitation).
- **1.24–1.70× faster end-to-end** than torch on the same tasks,
identical Newton physics (§3.1).
- Manager-based: `WarpFrontend` adapts the stable cfg in place; MDP
twins resolve by name on the mirrored experimental package tree —
deterministic, registration-free, missing twins reported in one
aggregated failure.
- Direct: the stable registration declares its warp class via
`warp_entry_point`; only the env class swaps, the cfg is shared.
- Stable observation-noise cfgs (constant/uniform/gaussian) swap to
warp-native twins during adaptation; cfgs without a twin (class-based
noise models, customized funcs, tensor-valued params) are a hard error
instead of silently dropping noise. The warp observation manager rejects
non-warp noise cfgs and plumbs the shared per-env RNG state.
- Plus: warp adapters for the material/mass randomization startup
events; `isaaclab_tasks_experimental` mirrors the stable `core/<task>`
layout; `Metrics/success_rate` restored under warp.

## 2. Migration map

| Retired | Replacement |
|---|---|
| `Isaac-{Cartpole,Humanoid,Ant}-Warp-v0`, `Isaac-Reach-Franka-Warp-v0`
(+`-Play`) | stable id + `--frontend warp presets=newton_mjwarp` |
| `Isaac-Velocity-Flat-<Robot>-Warp-v0` (+`-Play`), all robots | stable
id + `--frontend warp presets=newton_mjwarp` |
| `Isaac-{Cartpole,Ant,Humanoid}-Direct-Warp-v0`,
`Isaac-Reorient-Cube-Allegro-Direct-Warp-v0` | stable id + `--frontend
warp presets=newton_mjwarp` (via `warp_entry_point`) |

## 3. Test plan

- [x] Merged current `develop`; `./isaaclab.sh -f` clean on all files.
- [x] Re-merged develop (task-id catalog update #6606, entrypoints
refactor); environments catalog regenerated; conversion sweep + warp
smokes (`Isaac-Velocity-Flat-UnitreeGo2`, `Isaac-Cartpole-Direct`)
re-run green; pre-commit clean on all files.
- [x] 142 focused tests: frontend units, cfg-conversion sweep over every
covered stable id, no-warp-registrations end-state pin, shared-CLI
tests.
- [x] 5-iteration RSL-RL trainings exit clean (`Training time:` present,
no traceback): `Isaac-Cartpole` (torch); `Isaac-Cartpole`,
`Isaac-Humanoid`, `Isaac-Cartpole-Direct`, `Isaac-Ant-Direct`,
`Isaac-Velocity-Flat-AnymalD`, `Isaac-Velocity-Flat-H1` (warp;
randomization events active); `Isaac-Reorient-Cube-Allegro-Direct`
(warp). `Metrics/success_rate` reported under warp.

### 3.1 Frontend A/B benchmark

Same stable task id and cfg on both sides, `presets=newton_mjwarp` on
**both** (identical Newton physics — only `--frontend` differs); rsl_rl,
4096 envs, 300 iterations, headless, one L40 per run, single run per
cell. Metric: rsl_rl `Training time:` — learn-loop wall-clock (env
stepping + PPO update; warp includes its one-time CUDA-graph capture).
PPO-update cost is common to both, so the env-pipeline gain exceeds the
end-to-end ratio.

| Task | torch | warp | end-to-end | final reward (torch / warp) |
|---|---|---|---|---|
| `Isaac-Cartpole` | 58.2 s | 39.7 s | 1.46× | 4.95 / 4.95 |
| `Isaac-Humanoid` | 162.0 s | 110.5 s | 1.47× | 149.1 / 179.8 |
| `Isaac-Ant` | 109.8 s | 64.5 s | 1.70× | 82.1 / 77.1 |
| `Isaac-Velocity-Flat-AnymalD` | 162.4 s | 130.7 s | 1.24× | 20.3 /
21.4 |

## 4. Out of scope / follow-ups

- No video-recorder plumbing on warp envs; rough terrain blocked on Warp
`TerrainImporter` APIs; no twins for IK/OSC actions, `height_scan`, or
Digit/Spot-specific terms.
- OVPhysX support for the warp frontend prepared as a follow-up PR
(physics gate widening + backend data-parity test).
- Candidate refactor: fold the ~300 lines of converged plumbing shared
by `DirectRLEnvWarp` and stable `DirectRLEnv` into a common base.

---------

Co-authored-by: Kelly Guo <kellyguo123@hotmail.com>
matthewtrepte pushed a commit to matthewtrepte/IsaacLab that referenced this pull request Aug 4, 2026
@hujc7 @kellyguo11 @matthewtrepte
[WARP] Cleanup Part 1: Add frontend bridge for stable manager-based t…
c2ae68d 
…asks (isaac-sim#5504)

## Review Map

- A stacked PR’s page shows the cumulative diff of its dependency chain;
the "Exact changes" link pins that PR’s own commit range.
- SHA links can go stale after a branch update; the copy on isaac-sim#5504 is
refreshed first.

| PR | Status | Depends on | Exact changes |
|---|---|---|---|
| 📌 isaac-sim#5504 Part 1: frontend bridge (stable ids on the warp runtime) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/5504?label=)
| — | — |
| isaac-sim#6607 Part 2: mask-first frontend (draft) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6607?label=)
| isaac-sim#5504 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6607/changes/00ae695ac63e67ca678b91a837d3737529d2ec91..f68d03f40854b31a36063c5068f2933d32ea5db9)
|
| isaac-sim#6646 Part 3: scene reset mask-native + capture-ready (draft, base
rewritten — pending rebase onto isaac-sim#6607) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6646?label=)
| isaac-sim#6607 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6646/changes/4360622bde446c52c3fc85924639f9d1119d6e95..6d49e32f11e15894a4a0e49161a356ec79679ac6)
|
| isaac-sim#6580 Part 4: OVPhysX backend (draft) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6580?label=)
| isaac-sim#5504 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6580/changes/fc37fa65ad95d1ac6e3f32e721329ef715899d37..716cef20979ca6c578a6e595812291c7b1a282fc)
|
| isaac-sim#6611 Part 5: persistent-launch speedups (draft, absorbs isaac-sim#6551) |
![](https://img.shields.io/github/pulls/detail/state/isaac-sim/IsaacLab/6611?label=)
| isaac-sim#6646 |
[link](https://github.com/isaac-sim/IsaacLab/pull/6611/changes/6d49e32f11e15894a4a0e49161a356ec79679ac6..6e3da794215da03745a9bc8d35790e22262b099a)
|

## 1. Summary

- `--frontend {torch,warp}` on the shared RL CLI (train **and play**)
runs **stable** task ids on the warp runtime — no duplicated configs, no
parallel warp task registrations at all (end state pinned by test). The
`rlinf` integration constructs environments inside the external
framework and is not frontend-routable (known limitation).
- **1.24–1.70× faster end-to-end** than torch on the same tasks,
identical Newton physics (§3.1).
- Manager-based: `WarpFrontend` adapts the stable cfg in place; MDP
twins resolve by name on the mirrored experimental package tree —
deterministic, registration-free, missing twins reported in one
aggregated failure.
- Direct: the stable registration declares its warp class via
`warp_entry_point`; only the env class swaps, the cfg is shared.
- Stable observation-noise cfgs (constant/uniform/gaussian) swap to
warp-native twins during adaptation; cfgs without a twin (class-based
noise models, customized funcs, tensor-valued params) are a hard error
instead of silently dropping noise. The warp observation manager rejects
non-warp noise cfgs and plumbs the shared per-env RNG state.
- Plus: warp adapters for the material/mass randomization startup
events; `isaaclab_tasks_experimental` mirrors the stable `core/<task>`
layout; `Metrics/success_rate` restored under warp.

## 2. Migration map

| Retired | Replacement |
|---|---|
| `Isaac-{Cartpole,Humanoid,Ant}-Warp-v0`, `Isaac-Reach-Franka-Warp-v0`
(+`-Play`) | stable id + `--frontend warp presets=newton_mjwarp` |
| `Isaac-Velocity-Flat-<Robot>-Warp-v0` (+`-Play`), all robots | stable
id + `--frontend warp presets=newton_mjwarp` |
| `Isaac-{Cartpole,Ant,Humanoid}-Direct-Warp-v0`,
`Isaac-Reorient-Cube-Allegro-Direct-Warp-v0` | stable id + `--frontend
warp presets=newton_mjwarp` (via `warp_entry_point`) |

## 3. Test plan

- [x] Merged current `develop`; `./isaaclab.sh -f` clean on all files.
- [x] Re-merged develop (task-id catalog update isaac-sim#6606, entrypoints
refactor); environments catalog regenerated; conversion sweep + warp
smokes (`Isaac-Velocity-Flat-UnitreeGo2`, `Isaac-Cartpole-Direct`)
re-run green; pre-commit clean on all files.
- [x] 142 focused tests: frontend units, cfg-conversion sweep over every
covered stable id, no-warp-registrations end-state pin, shared-CLI
tests.
- [x] 5-iteration RSL-RL trainings exit clean (`Training time:` present,
no traceback): `Isaac-Cartpole` (torch); `Isaac-Cartpole`,
`Isaac-Humanoid`, `Isaac-Cartpole-Direct`, `Isaac-Ant-Direct`,
`Isaac-Velocity-Flat-AnymalD`, `Isaac-Velocity-Flat-H1` (warp;
randomization events active); `Isaac-Reorient-Cube-Allegro-Direct`
(warp). `Metrics/success_rate` reported under warp.

### 3.1 Frontend A/B benchmark

Same stable task id and cfg on both sides, `presets=newton_mjwarp` on
**both** (identical Newton physics — only `--frontend` differs); rsl_rl,
4096 envs, 300 iterations, headless, one L40 per run, single run per
cell. Metric: rsl_rl `Training time:` — learn-loop wall-clock (env
stepping + PPO update; warp includes its one-time CUDA-graph capture).
PPO-update cost is common to both, so the env-pipeline gain exceeds the
end-to-end ratio.

| Task | torch | warp | end-to-end | final reward (torch / warp) |
|---|---|---|---|---|
| `Isaac-Cartpole` | 58.2 s | 39.7 s | 1.46× | 4.95 / 4.95 |
| `Isaac-Humanoid` | 162.0 s | 110.5 s | 1.47× | 149.1 / 179.8 |
| `Isaac-Ant` | 109.8 s | 64.5 s | 1.70× | 82.1 / 77.1 |
| `Isaac-Velocity-Flat-AnymalD` | 162.4 s | 130.7 s | 1.24× | 20.3 /
21.4 |

## 4. Out of scope / follow-ups

- No video-recorder plumbing on warp envs; rough terrain blocked on Warp
`TerrainImporter` APIs; no twins for IK/OSC actions, `height_scan`, or
Digit/Spot-specific terms.
- OVPhysX support for the warp frontend prepared as a follow-up PR
(physics gate widening + backend data-parity test).
- Candidate refactor: fold the ~300 lines of converged plumbing shared
by `DirectRLEnvWarp` and stable `DirectRLEnv` into a common base.

---------

Co-authored-by: Kelly Guo <kellyguo123@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@hujc7