Skip to content

Redirect the Newton RTX visualizer shader cache - #8504

Draft
hujc7 wants to merge 6 commits into
isaac-sim:mainfrom
hujc7:jichuanh/newton-rtx-shader-cache
Draft

hujc7 wants to merge 6 commits into
isaac-sim:mainfrom
hujc7:jichuanh/newton-rtx-shader-cache

Conversation

@hujc7

@hujc7 hujc7 commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The Newton RTX viewer now honours an explicitly set OVRTX_SHADER_CACHE_PATH. CI restores its OVRTX shader cache into that directory, but Newton's viewer creates its own OVRTX renderer and never used it, so it compiled every shader on each run. Without the variable, OVRTX keeps its default cache location; nothing changes.

The redirect also no longer initializes OVRTX. It only sets the cache path, leaving initialization and its configuration to whichever renderer is created first.

Description

  1. Isaac Lab's NewtonViewerRTX subclass overrides _init_ovrtx, where Newton's ViewerRTX creates its renderer on the first frame, and calls redirect_shader_cache() before Newton's implementation. The subclass already overrides Newton's _init_window the same way. No Newton configuration is copied into Isaac Lab.
  2. redirect_shader_cache, added in Add OVRTX shader cache for CI rendering tests #6905 for CI cache persistence, took a RendererConfig because reaching the OVRTX settings extension called ovrtx_initialize with it. It now loads the library without initializing, as OVRTX itself does when registering schema paths, and applies the two /rtx/shaderDb paths. OVRTXRenderer calls it without a config, and its one-caller helper apply_shader_cache_settings is folded into it.
  3. Tests check the redirect never initializes OVRTX and that the viewer redirects before Newton creates its renderer. The viewer test also fails if Newton renames _init_ovrtx.

Why it was missing

The Newton RTX visualizer (#6658) predates the shader cache (#6905), which added the redirect inside OVRTXRenderer for the rendering-correctness jobs. #8006 later restored the cache in the isaaclab_visualizers job, but Newton's own renderer never read it.

Effect

Where Before After
x86 CI, isaaclab_visualizers job (run), cache restored but unused 104 s not yet measured
x86 desk, cold then warm cache 142 s 49–57 s
ARM CI (#8233 benchmark), warm cache 301 s 27 s

On ARM the cache seeded only by the Cartpole rendering tests fully covered the viewer (0 MB growth), so the cache-warm job needs no change.

This is interim. #8303 plans to adopt Newton #4627 (ViewerRTX(renderer=...)), after which the viewer borrows the renderer Isaac Lab creates and the override can be removed.

Validation

On x86 desk with ovrtx 0.5.1.385782, the develop pin:

  1. test_ovrtx_shader_cache_redirect.py and test_ovrtx_renderer_contract.py: 63 passed. test_newton_adapter.py -k newton_rtx: 2 passed; the viewer test fails with the redirect removed.
  2. With an empty OVRTX_SHADER_CACHE_PATH, test_visualizer_integration_newton_rtx.py (3 passed, 2 skipped) and the legacy-newton-ovrtx_renderer-static Cartpole kitless case (1 passed) each filled the directory (119 MB).
  3. Setting the paths before initialization survives a later ovrtx_initialize. A renderer created after an earlier initialization still applies its own log level and log file.

Type of change

  1. Bug fix (non-breaking for users; redirect_shader_cache drops its config parameter and apply_shader_cache_settings is removed).

Checklist

  • I have read and understood the contribution guidelines
  • I have run the pre-commit checks with uv run isaaclab --format
  • I have added changelog fragments for isaaclab_ov and isaaclab_visualizers
  • I have added tests that fail without the change
  • My name already exists in CONTRIBUTORS.md

@github-actions github-actions Bot added the isaac-lab Related to Isaac Lab team label Oct 12, 2026
hujc7 added a commit that referenced this pull request Oct 13, 2026
…8233)

## Summary

ARM CI restores compatible Warp and OVRTX caches and runs Newton/OVPhysX
marker coverage through the shared x86 test action. Camera smoke
coverage stays enabled, and previously skipped ARM checks are restored.

# Description

1. Route ARM marker tests through `run-package-tests`, with explicit
`linux/arm64` passed through image lookup, dependency hashing and
fallback builds. PRs restore ARM caches; protected-branch pushes publish
them. Test files run in four slots like the x86 test jobs; rendering
files still run one at a time.
2. Run the existing state-training, camera-rendering and camera-training
Cartpole probes as their own ARM step after the marker tests, reusing
the restored Warp and OVRTX cache directories that `run-package-tests`
now exposes as outputs. Installation CI now covers state training
separately, so the camera probes remain necessary. The step checks the
probe report explicitly, since `run-tests` records failures in the
report rather than its exit code.
3. Restore shadow-hand kitless rendering and operational-space
controller coverage on ARM. Carry the Isaac Sim PhysX training re-enable
into the replacement uv workspace test rather than restoring a deleted
installation test, and preserve the required ARM system OpenMP preload
when the probe adds the Isaac Sim shim. Remove the Kuka visual-material
ARM marker to match x86 rendering selection. Supersedes #8230.
4. Build the image in the runner's Docker daemon when no registry cache
is configured, writing new layers with zstd. The disposable
`docker-container` builder exported and reloaded the ~29 GB image every
build and discarded its layer cache. Only ARM takes this path today: x86
callers resolve ECR and keep the `docker-container` builder. The action
is part of the dependency-cache hash, so the first build after merge
misses that cache once.
5. Count kitless rendering files that use `rendering_test_utils` as
renderers, so their cold shader compiles no longer run concurrently.

## Validation

1. `uv run --frozen --extra test isaaclab -f`: all checks passed.
2. Docker-storage and file-scheduler coverage: 19 tests passed.
Actionlint passed for ARM and x86 workflows; 50 local action calls
validated against their declared inputs.
3. A real subprocess regression check failed before the preload fix and
passes afterward for ARM with/without the shim and x86 with the shim.
4. On `7b73dafe1fe`, [ARM
CI](https://github.com/isaac-sim/IsaacLab/actions/runs/37401182263)
passed 71 marker cases (3 expected skips) and all 3 state/camera smoke
probes. [ARM
installation](https://github.com/isaac-sim/IsaacLab/actions/runs/37401182231)
passed 3/3 attempts on three NVIDIA GB10 runners, with all 8 contracts
including PhysX executed each time; x86 installation also passed all 8
contracts.
5. On the final head `01a3ff4737d`, the [x86 Docker
suite](https://github.com/isaac-sim/IsaacLab/actions/runs/38312079847)
passed: 30 jobs succeeded and 3 post-merge-only jobs were skipped as
expected. Hosted pre-commit, Tools Tests, Docs, kitless image
validation, installation and multi-GPU CI also passed.
6. ARM job time, all on NVIDIA GB10 with driver 580.159.03. A temporary
commit (since reverted) let PR runs save caches for this measurement:

   | Run | Build | Tests | Total |
   | --- | ---: | ---: | ---: |
| Before,
[cold](https://github.com/isaac-sim/IsaacLab/actions/runs/37401182263/job/112609307602)
| 8m30 | 16m52 | 25m42 |
| [Warm, new
runner](https://github.com/isaac-sim/IsaacLab/actions/runs/38298855064)
(3 runs) | 3m06–3m23 | 5m22–5m38 | 9m01–9m06 |
   | Warm, runner with layer cache | 0m56 | 5m42 | 6m57 |
| [Final head, new
runner](https://github.com/isaac-sim/IsaacLab/actions/runs/38312048044)
| 3m31 | 9m22 | 13m11 |

The first four rows also included the Newton RTX shader-cache fix, now
split into #8504; the final head does not. Warm per-file times: Cartpole
54 s (was 297 s), Kuka 51 s, Shadow Hand 41 s, operational space 118 s
alongside them. Newton RTX takes 254 s without #8504 and 27 s with it.
Every run passed 71 marker cases (3 expected skips) and 3 smoke probes.
Image export on the runners' containerd store fell from 243 s with gzip
to 44–46 s with zstd. Protected-branch publication remains unmeasured
before merge.

## Type of change

1. Bug fix (non-breaking CI and test coverage changes).

## Release backport

- [x] <!-- backport-active-release --> Backport this pull request to the
active release branch after it merges into `develop`

## Checklist

- [x] I have read and understood the [contribution
guidelines](https://isaac-sim.github.io/IsaacLab/develop/source/refs/contributing.html)
- [x] I have run the `pre-commit` checks with `uv run isaaclab --format`
- [x] Documentation describing the smoke-test caller matches the current
workflow; rendered documentation is unaffected
- [x] No new project warning sites were added; CI still reports existing
dependency and action deprecations
- [x] I have validated the changed GPU behavior with focused coverage
and applied the test-audit criteria to test changes
- [x] I have added changelog fragments for the touched source packages
- [x] My name already exists in `CONTRIBUTORS.md`

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants