[rwiltz]

Description

Stacked on #8158 (keyboard input from visualizer windows); review that first. This is the controller integration that uses its API.

Keyboard, gamepad and SpaceMouse teleop can now run as Isaac Capture pipelines in isaaclab_teleop, with the same keys and buttons as the isaaclab.devices devices:

• se2/se3_keyboard_teleop_cfg, se2/se3_gamepad_teleop_cfg and se2/se3_spacemouse_teleop_cfg build an IsaacTeleopCfg for each device. The keyboard is read in process from whichever visualizer
  window has focus (Kit or Newton), with no input group and no keys from other apps; while it is read, the visualizers' keyboard camera movement is turned off. The gamepad and SpaceMouse are read by the
  Isaac Capture gamepad and spacemouse plugins, started with the session. Without --xr they run headset-free on the standalone CloudXR profile.
• IsaacTeleopDevice starts, pauses and resets teleoperation on B / P / R and resets on the SpaceMouse right button, the bindings the scripts made through a Carb keyboard.
• device_selection translates an isaaclab.devices keyboard, gamepad or SpaceMouse config into the equivalent pipeline. teleop_se3_agent.py and record_demos.py use it, so --teleop_device and
  environment teleop_devices entries work unchanged. Without the isaacteleop package they keep using the isaaclab.devices devices.

Follow-up: #8798 moves the remaining keyboard scripts (replay, annotate, consolidated Mimic, pick-and-place) and deprecates the isaaclab.devices keyboard, gamepad and SpaceMouse devices.

Fixes # (issue)

Type of change

• New feature (non-breaking change which adds functionality)

Checklist

Docker and GPU tests run on demand. Push the commits you want tested, then
comment run-ci on the pull request.

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with uv run isaaclab --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have validated the changed behavior with focused coverage and applied the test-audit criteria to any test changes
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (do not edit CHANGELOG.rst or bump package versions — CI handles that)
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Confidence Score: 5/5

[Medium impact] No new blocking defect was established in the changes since the previous review.

Summary

Routes keyboard, gamepad, and SpaceMouse teleoperation through Isaac Capture while keeping the native-device fallback.

• Kit and Newton visualizers now report keyboard events through a shared input source.
• Keyboard pipelines now drive SE(2) and SE(3) teleoperation from the focused viewer.
• Gamepads now drive SE(2) and SE(3) teleoperation through Isaac Capture.
• SpaceMouse devices now drive SE(2) and SE(3) teleoperation through Isaac Capture.
• Teleoperation scripts now select matching Isaac Capture pipelines from device requests.

Previous-thread context:

• rwiltz explained that Isaac Capture tests device behavior upstream. Greptile conceded that broad request; the thread was resolved.
• The later-window finding is addressed by checking for unattached visualizers on each step. Its thread was manually resolved without an explanatory reply.
• The recording thread was manually resolved without an explanatory reply. The new guard prevents the direct demo discard, but it does not stop an operator reset from pausing the Capture session before the script checks the guard.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    Config[Device configuration] --> Selection[Select Capture pipeline or native fallback]
    Window[Focused visualizer window] --> Keys[Key events]
    Keys --> Capture[Isaac Capture session]
    Plugins[Gamepad or SpaceMouse plugin] --> Capture
    Selection --> Capture
    Capture --> Device[IsaacTeleopDevice]
    Device --> Scripts[Teleoperation and recording scripts]

Loading

_{Reviews (4) · Last reviewed commit: "Relock with the repository's uv and tidy..." · Reviewed by Greptile}

[rwiltz]

@greptile

[isaaclab-review-bot]

Isaac Lab Review Bot

Needs attention: Design and architecture (1), Scope and over-engineering (2), Compatibility and deprecation (2), Style consistency (1), Test quality (1). Other reviewed categories look good.

• PR description: The body omits the isaacteleop-to-isaaccapture dependency and lockfile migration, retains the empty issue placeholder, and provides no validation commands; add these details, while no performance benchmark is required.
• Design and architecture: TeleopSessionLifecycle retrieves visualizers through SimulationContext.instance() and suppresses lookup failures, hiding keyboard loss; pass owned visualizers from the composition root and remove the broad guard.
• Scope and over-engineering: Minimal implementation: extend BaseVisualizer, KitVisualizer, NewtonVisualizer, TeleopSessionLifecycle, and both scripts; keep key_event_source, _plugins, device_selection, six builders, six export files, and Kit/Newton sources; conditionally forward SpaceMouse buttons and reuse reset_recording_instance. Suggested PR breakdown: 1) visualizer key API/backends; 2) dependency/lock migration; 3) builders/lifecycle controls; 4) script routing/docs, each with focused package tests.
• API: API changes look good.
• Compatibility and deprecation: Breaking changes: explicit device selection silently changes established routing; preserve old routing through a warned, documented deprecation with transition coverage, and regenerate uv.lock using repository uv.
• Implementation: Implementation looks good.
• Style consistency: IsaacTeleopDevice places four unannotated class constants among private methods; move them before instance methods and declare them as ClassVar[int].
• Test quality: The key-source, routing, session, and operator tests protect distinct contracts, but the RTX camera-key row repeats the same inherited GL-viewer path; keep one backend-independent case.

Significant concerns. Posted 7 inline findings.

Automated review; human maintainers own approval decisions.

[rwiltz]

@greptile

[rwiltz]

run-ci