[ooctipus]

Summary

Backports #8027 (b05a35a0eedda86662f82e663993c2b7046bf94f) to release/3.0.0.

Newton cameras, raycasters, and visualizers acquire shared native resources through the simulation registry and consume transforms and geometry through SDP. The original API migration notes and changelog fragments are included unchanged.

This is an exact, conflict-free cherry-pick. The prerequisite #8021 is already present on the release branch. No additional refactoring or fixes are included.

Validation

• Exact-patch backport validation passed.
• Formatting, lint, and changelog checks passed.
• 14 focused registry/lifecycle tests passed.
• A broader local test invocation encountered Rerun/package-import collection errors. Further local testing was stopped at the user's request; full package, GPU, rendering, and documentation validation is delegated to CI.

[ooctipus]

run-ci

[greptile-apps]

Confidence Score: 3/5

[High risk] Restructures how physics and rendering backends are owned and initialized.

The PR is not yet safe to merge because foreign-physics Newton rendering can fail to initialize and web visualizers can select incompatible camera batches.

Findings

1.  Joint labels break binding ▶
2.  Fallback camera count unchecked ▶

Summary

The backport moves Newton builders and native models into the simulation resource registry and makes cameras, ray casters, and visualizers consume SDP data directly.

• It also adds consumer-owned query graphs, renderer-configuration selection for streaming cameras, and lifecycle updates across rendering backends.
• The review identifies foreign-physics joint-path binding and fallback-camera selection regressions.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Clone[Clone plan] --> Builder[Shared Newton builder]
  Builder --> Native[Registry-owned Newton backend]
  Native --> Camera[Camera renderer]
  Native --> Ray[Ray caster]
  Native --> Viewer[Visualizers]
  SDP[Scene Data Provider] --> Camera
  SDP --> Ray
  SDP --> Viewer

Loading

_{Reviews (1) · Last reviewed commit: "Backport Newton consumer resource owners..."}

[greptile-apps]

Joint labels break binding

In a foreign-physics scene, a Newton body can be labeled with its joint prim path, while PhysX publishes the rigid-body prim path. Passing the joint label directly to create_mapping raises KeyError and prevents this renderer from initializing. The removed visualization path resolved joint labels to their target bodies; the Newton visualizers and ray caster now use this direct mapping too.

[greptile-apps]

Fallback camera count unchecked

If a scene has multiple cameras and the first covers fewer environments than the simulation, this selects it without checking its view count. The streaming path then uses simulation environment IDs to index that camera's frames, which can raise an out-of-range error even when a suitable camera appears later. Viser makes the same selection.

[isaaclab-review-bot]

Isaac Lab Review Bot

The Newton consumer-ownership backport coherently moves builders, native models, renderers, and query consumers onto the simulation registry and SDP paths. Before merge, restore Kit's camera-enablement gate, make the new PhysX synchronization CPU-safe, and document the newly public NewtonQueries symbol.

• Design and architecture: The NewtonBuilderCfg/NewtonBackendCfg lifetime split and consumer-owned query graphs align with the registry and SDP ownership rules. However, centralizing early renderer acquisition in BaseVisualizer removed Kit's /isaaclab/cameras_enabled guard, so an auto-camera configuration can initialize RTX infrastructure even when cameras were not enabled.
• API: The breaking Newton manager and streaming-renderer migrations include migration guidance. NewtonQueries is now exported through the package API and described in the changelog, but it is missing from the updated generated API reference; add an autosummary/autoclass entry or keep the helper private.
• Implementation: The Newton renderer, ray-caster, and visualizer producer/consumer paths consistently acquire shared resources and use SDP. Two concrete fixes remain: preserve Kit's camera-availability precondition before renderer construction, and replace or guard wp.synchronize_stream(self.device) because CPU-backed rigid-object mass, COM, and inertia setters cannot synchronize a CUDA stream.

Minor fixes needed. Posted 3 actionable findings inline.

Automated review; human maintainers own approval decisions.

[isaaclab-review-bot]

🟡 Warning · Implementation — Kit streaming renderer loses cameras-enabled guard

The registration moved from KitVisualizer.__init__ drops the /isaaclab/cameras_enabled condition. With streaming_view=True and an auto-created camera target but no --enable_cameras, IsaacRtxRendererCfg is now constructed, enabling omni.replicator.core and attaching the RTX Hydra engine. KitVisualizerCfg's unchanged docstring still promises the streaming view is silently skipped in that case. Re-apply the camera-availability gate before acquiring the renderer.

[isaaclab-review-bot]

🟡 Warning · Implementation — synchronize_stream is invalid on CPU devices

wp.synchronize_stream(device) resolves Device.stream, which is only defined for CUDA devices. When self.device is "cpu", the public set_masses_index, set_coms_index, and set_inertias_index writes (and the mask variants plus mass/inertia randomization events routed through them) now raise instead of writing. Use wp.synchronize_device(self.device), which is a no-op on CPU, or guard the call on a CUDA device.

[isaaclab-review-bot]

🔵 Suggestion · Api — Exported NewtonQueries missing from API docs

NewtonQueries is added to the package's public __all__ and stub exports and described as public in the changelog, but docs/source/api/lab_newton/isaaclab_newton.physics.rst was updated only for NewtonBuilderCfg and create_newton_builder. The new query/capture helper is therefore absent from the generated reference. Add it to the autosummary list with a matching autoclass block, or keep it private.