Repository navigation
[Core] Remove parallel launch and preset mechanisms - #8112
Conversation
|
run-ci |
|
There was a problem hiding this comment.
Isaac Lab Review Bot
The consolidation of launcher argument normalization, visualizer aliases, and Hydra preset resolution is coherent, but two issues need maintainer action: exported Hydra utilities are removed without the required deprecation period, and build_simulation_context now depends on a launcher-private argparse parser that changes its programmatic error behavior.
- Design and architecture: The normalized dict-based launcher path and centralized visualizer type table reduce duplicated decisions. However,
isaaclab.sim.build_simulation_contextnow imports the privateisaaclab.app.sim_launcher._parse_visualizer_csv, creating a reverse dependency on the launcher and causing unknown programmatic visualizer names to raiseargparse.ArgumentTypeErrorrather than follow the prior warning-and-skip path. Alias normalization should remain in the visualizer configuration layer. - API:
initialize_visualizers(config_filter=...)is additive and documented, and the deprecatednewtonvisualizer name remains accepted through the reviewed entry points. However,hydra_task_config,parse_overrides, andapply_overridesare removed despite being exported or directly importable public utilities. Migration notes do not satisfy the repository requirement for prior deprecation; compatibility wrappers should remain for a deprecation cycle. - Implementation: The launcher normalization flow, canonical visualizer comparisons, and consolidated task-config resolution are internally consistent. The required fixes are localized: retain deprecated Hydra compatibility symbols that delegate to the production path, and avoid calling the launcher's private argparse parser from
build_simulation_contextwhile preserving its prior programmatic behavior.
Minor fixes needed. Posted 2 actionable findings inline.
Automated review; human maintainers own approval decisions.
| "resolve_presets", | ||
| "setup_preset_cli", | ||
| "SuccessMonitor", | ||
| "SuccessMonitorCfg", | ||
| ] | ||
|
|
||
| from .hydra import PresetCfg, preset, hydra_task_config, resolve_task_config, resolve_presets | ||
| from .hydra import PresetCfg, preset, resolve_task_config, resolve_presets |
There was a problem hiding this comment.
🟡 Warning · Api — Public symbols removed without prior deprecation
hydra_task_config was an exported symbol of isaaclab_tasks.utils, and parse_overrides/apply_overrides were importable from ...utils.hydra; deleting them breaks user entrypoints at import time. The repository rule requires a prior deprecation in addition to migration notes. Keep thin wrappers that emit DeprecationWarning and delegate to resolve_task_config / the production override path, and drop them in a later release.
| @@ -1140,15 +1079,17 @@ def build_simulation_context( | |||
| sim_cfg.device = device | |||
|
|
|||
| if visualizers: | |||
| SettingsManager.instance().set_string("/isaaclab/visualizer/types", " ".join(visualizers)) | |||
| from ..app.sim_launcher import _parse_visualizer_csv # noqa: PLC0415 | |||
There was a problem hiding this comment.
🔵 Suggestion · Design Architecture — Core helper imports launcher's private CSV parser
build_simulation_context now imports isaaclab.app.sim_launcher._parse_visualizer_csv, inverting the layering (sim -> launcher, whose module scope imports every backend package) and changing the failure mode: an unknown name previously warned and was skipped, but now raises argparse.ArgumentTypeError from a programmatic API. This PR already made isaaclab.visualizers.visualizer_cfg the single alias table, which this module imports; map names with _VISUALIZER_ALIASES from there.
|
run-ci |
6028c98 to
5757f48
Compare
- Delete hydra_task_config (no callers) and the second preset-override parser (parse_overrides/apply_overrides); tests go through register_task and resolve_task_config, the production parser. - Parse --viz once: the normalizer no longer joins the parsed list back into a string and parses it again. - Keep one visualizer type table and alias map; SimulationContext consumes canonical names instead of remapping aliases itself. - SimulationContext: drop SettingsHelper (SettingsManager.set already dispatches on type), return cfg.dt from get_physics_dt, merge initialize_visualizers with its private forwarder, normalize visualizer_cfgs once, read has_gui from the property, and drop use_stage's Isaac Sim < 5 branch.
5757f48 to
1a96515
Compare
|
run-ci |
|
Backported to |
## Description Follows #8111 (merged). This is the first step toward launch and preset consistency: one mechanism per decision. This PR removes parallel mechanisms that nothing needs. Follow-ups will make `launch_simulation` the single place visualizers and the device are decided, and replace `--physics` with `physics=` presets. ### Changes - **One preset-override parser.** `hydra_task_config` (no callers) is removed, and its callback indirection is folded into `resolve_task_config`. So is the second override parser (`parse_overrides` / `apply_overrides`). - `test_hydra.py` and the rendering test helper now go through `register_task` / `resolve_task_config`, the production parser, instead of a different one. - Five parser-only tests are dropped. One of them tested behavior production never had. - **`--viz` parsed once.** `_parse_visualizer_csv` accepts the argparse-parsed list, so `_normalize_launcher_args` no longer joins it back into a string and parses it again. Invalid values still fail at argparse time. - **One visualizer type table** (with the deprecated `newton` → `newton_gl` alias) lives in `isaaclab.visualizers.visualizer_cfg`. `SimulationContext` consumes canonical names instead of remapping aliases itself; the two tables disagreed about `none`. - **`SimulationContext` setup:** - `SettingsHelper` is removed, since `SettingsManager.set` already dispatches on the value type; - `get_physics_dt()` returns `cfg.dt`, and a `getattr(cfg.physics, "dt")` that never matched is dropped; - `initialize_visualizers(config_filter=None)` replaces a public/private forwarder pair; - the visualizer config list is normalized once instead of three times; - `build_simulation_context` reads `sim.has_gui`; - `use_stage` drops its Isaac Sim < 5 branch (the repo pins 6.1). Net: +209 / −500. ## Type of change - Breaking change (removed `hydra_task_config`, `parse_overrides`, `apply_overrides`, and `SettingsHelper`; `initialize_visualizers` gained `config_filter`) ## Validation All runs on GPU 0. - **Tests:** - `isaaclab/test/app` (27) and `isaaclab_physx/test/app` (90); - hydra (67), preset CLI (11), preset Kit decision (6), runtime compatibility (27), visualizer intent (8), distributed device (11), agent preset composition (13); - simulation context (26), stage utils (15), headless `build_simulation_context` (7), RL entrypoints (44 + 27). - **Training:** Cartpole with `physics=isaacsim_physx` and `physics=newton_mjwarp` runs 2 iterations, and the run summary names each backend correctly. - **Pre-existing failures on this machine, identical on the base branch:** one visualizer test that needs `carb`, and 8 golden-image diffs in `test_rendering_cartpole_kitless`. ## Release backport - [x] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop` ## Checklist - [x] I have read and understood the contribution guidelines - [x] I have run the pre-commit checks - [x] My changes generate no new warnings - [x] I have added changelog fragments for every touched package (cherry picked from commit b744dbe)
…8125) ## Description This is step 2 of the launch-consistency series, after #8111 and #8112: each run-time choice is decided once, in `launch_simulation`, and consumers read the resolved config instead of re-deciding it. ### Visualizers - **Resolved once.** `launch_simulation` resolves the final visualizer list: `--viz none` gives none; an explicit `--viz a,b` gives exactly those, reusing a configured visualizer of the same type (with its settings) or else a default; with no `--viz`, the config's own list; livestreaming adds Kit; and `--max_visible_envs` applies to the result. The list is written into the config's `sim.visualizer_cfgs` before any launcher starts, so a missing visualizer package fails before Kit starts. - **An explicit `--viz` governs everything derived from visualizers.** A configured `newton_rtx` or Kit visualizer that `--viz` drops no longer starts OVRTX, trips the Kit/OVRTX conflict check, or turns on the Kit streaming view. XR auto-starts only when the run has no Kit visualizer. - **`SimulationContext` reads `cfg.visualizer_cfgs`.** Its settings-based re-resolution and the `/isaaclab/visualizer/explicit` and `/disable_all` settings are gone. Code that builds a `SimulationContext` directly gets its config's visualizers. - **`build_simulation_context(visualizers=...)` is removed.** It had no callers and on develop only set a settings key without creating the visualizers. - **Public selection helpers.** `parse_visualizer_csv`, `resolve_visualizer_cfgs`, `get_visualizer_install_hint`, `VISUALIZER_TYPES` and `VISUALIZER_ALIASES` live in `isaaclab.visualizers.visualizer_cfg`. `VISUALIZER_TYPES` maps each type to its default config as a `"module:Class"` string, and `scan()` returns the typed `SimulationCfg` it found, so `launch_simulation` no longer probes with `getattr`/`hasattr`. - **Remaining fallback:** a launch with no `SimulationCfg` to write into (a physics config or `None`, as in several tutorials) still passes the selection through `/isaaclab/visualizer/types`, which `SimulationContext` applies once with the same function. It goes away when those entry points launch with their real config. The command-line convention changes (`--viz none` removal, no visualizer without `--viz`, `--video [SOURCE]`) are split out into the follow-up PR. ### Device - **Resolved once:** `--device`, or this rank's GPU in a distributed run, falling back to the config's device when `--device` is not given, then the launcher's refinement (for example XR picks the CPU), then `cuda` normalized to `cuda:N`. The result is written once to the config and to the launcher args. `validate_distributed_device` and its six callers are removed, since each rank now resolves its own `cuda:N`. - **Consequences:** `apply_env_overrides` no longer writes the device (its `apply_device` argument is removed), and `simple_agents` no longer overwrites `--device` with the task's device. ### One config scan per run - **RL run summary:** it no longer runs its own `scan()`. The RL backends print it inside `launch_simulation`, from the resolved config. - **Mimic:** `generate_dataset.py` launches with its real config instead of a throwaway one. - **Two locomanipulation scripts** still scan separately, because their task configs import USD and can only be parsed after Kit starts. ### Follow-up in this PR - **Top-level imports where cheap.** Function-local imports move to module top wherever measuring shows they load no `pxr`, `omni`, `carb`, `isaacsim`, or `usdrt` before Kit starts: 100 → 71 local imports across the launch stack, `SimulationContext`, the RL entrypoints, and mimic. The rest stay local for a stated reason: Kit-only, an optional dependency, or kept cheap on purpose (for example `isaaclab.test.utils`, which many Kit-free tests import, and the benchmark dispatcher). - **`SimulationContext` visualizer list.** `visualizer_cfgs` is turned into a list once (direct users now see `[]` instead of `None`), and default visualizer configs are built without a catch-all `try/except`. - **Video recording after launch.** `apply_video_recording` is wired after launch everywhere and reads the resolved visualizer list. `pre_launch_video_config` is the only place that adds a headless Kit visualizer, before launch, so a Kit visualizer is never injected too late to start Kit. A config with only streaming visualizers now errors on `--video` instead of picking a source that cannot capture. ### Bugs fixed - **`--viz none`:** the run summary listed the config's visualizers although none ran; it now shows `none (headless)`. - **Distributed runs:** the summary showed the default device instead of each rank's device. The summary now names the concrete backend (for example `ovphysx`) rather than `physx (ovphysx)`, because the resolved config no longer records which automatic selector chose it. Net: +667 / −853. ## Type of change - Bug fix - Breaking change: removed the `/isaaclab/visualizer/explicit` and `/disable_all` settings, the `has_kit_streaming_view` key of `visualizer_intent`, `build_simulation_context(visualizers=...)`, `apply_env_overrides(apply_device=...)` and `validate_distributed_device`; dropped the leading underscore from the visualizer selection helpers, and `show_run_summary` must be called inside the launch ## Validation All runs on GPU 0 except the multi-GPU check. - **New tests, each failing on the base:** - `--viz none` summary; - summary device taken from the resolved config; - device resolution for `--device cuda:1`, `cuda`, and `cpu`, and Kit starting on the config's device without `--device`; - visualizers resolved into the config for none and explicit selections, plus the fallback path; - an explicit `--viz` that drops a configured `newton_rtx` or Kit streaming view starts neither. - **Kit-free tests (266):** launcher and app, visualizer intent, runtime compatibility, distributed device, simulation-context visualizers, visualizers, RL entrypoints and video recording. - **Kit tests, each in its own process:** `test_kit_launcher` (86), `test_simulation_context` (26), `test_simulation_context_kit` (4). - **Training:** Cartpole with `physics=isaacsim_physx` and `physics=newton_mjwarp` each run 2 iterations; the summaries show the right backend and `cuda:0`. - **Video:** `train.py ... physics=isaacsim_physx --video --video_length 20 --video_interval 20` writes two mp4 clips headless. - **Multi-GPU:** `torch.distributed.run --nproc_per_node=2 train.py --task Isaac-Cartpole --distributed` on GPUs 0 and 1, with `physics=newton_mjwarp` and `physics=isaacsim_physx` (Kit in each rank): both exit 0, and rank 0 and rank 1 each report and run on their own `cuda:0` / `cuda:1`. - **Import purity:** importing the changed modules loads no `pxr`, `omni`, `carb`, `isaacsim`, or `usdrt`, and the env-config import-purity tests pass. - **Not run:** XR, livestream, and the imitation-learning scripts. ## Release backport - [ ] <!-- backport-active-release --> Backport this pull request to the active release branch after it merges into `develop` ## Checklist - [x] I have read and understood the contribution guidelines - [x] I have run the pre-commit checks - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] I have added changelog fragments for every touched package
Description
Follows #8111 (merged).
This is the first step toward launch and preset consistency: one mechanism per decision. This PR removes parallel mechanisms that nothing needs. Follow-ups will make
launch_simulationthe single place visualizers and the device are decided, and replace--physicswithphysics=presets.Changes
hydra_task_config(no callers) is removed, and its callback indirection is folded intoresolve_task_config. So is the second override parser (parse_overrides/apply_overrides).test_hydra.pyand the rendering test helper now go throughregister_task/resolve_task_config, the production parser, instead of a different one.--vizparsed once._parse_visualizer_csvaccepts the argparse-parsed list, so_normalize_launcher_argsno longer joins it back into a string and parses it again. Invalid values still fail at argparse time.newton→newton_glalias) lives inisaaclab.visualizers.visualizer_cfg.SimulationContextconsumes canonical names instead of remapping aliases itself; the two tables disagreed aboutnone.SimulationContextsetup:SettingsHelperis removed, sinceSettingsManager.setalready dispatches on the value type;get_physics_dt()returnscfg.dt, and agetattr(cfg.physics, "dt")that never matched is dropped;initialize_visualizers(config_filter=None)replaces a public/private forwarder pair;build_simulation_contextreadssim.has_gui;use_stagedrops its Isaac Sim < 5 branch (the repo pins 6.1).Net: +209 / −500.
Type of change
hydra_task_config,parse_overrides,apply_overrides, andSettingsHelper;initialize_visualizersgainedconfig_filter)Validation
All runs on GPU 0.
isaaclab/test/app(27) andisaaclab_physx/test/app(90);build_simulation_context(7), RL entrypoints (44 + 27).physics=isaacsim_physxandphysics=newton_mjwarpruns 2 iterations, and the run summary names each backend correctly.carb, and 8 golden-image diffs intest_rendering_cartpole_kitless.Release backport
developChecklist