[StafaH]

Description

This PR came out of a sweep of PR CI timings: job timings from 54 recent PR runs, plus step logs and JUnit reports from run 39969811204. The median PR run takes 23.3 min, and nine GPU jobs finish within about 3 minutes of each other (12.9–15.8 min median). Wall time only drops if the slow jobs get faster together. This PR changes three things:

Change	Problem	Fix
Contrib envs on the base image	test-contrib-environments is the last job to finish in 22/54 runs. It waits for the cuRobo image build, which takes more than 5 min in 13/54 runs (p90 24 min, max 46.6 min). No contributed environment imports cuRobo, and the base image installs a superset of the cuRobo image's Python extras	Run the job on the base image. Give the contrib files their own CONTRIB_ENVIRONMENT_TESTS skip list so that CUROBO_TESTS only lists tests that need cuRobo
Warp warmer coverage	PR jobs only restore the Warp kernel cache. The post-merge warmer fills it from the rigid Newton core environments only (211 MB). Jobs compile far beyond that: contrib +238 MB, isaaclab_tasks [2/3] +101 MB. Newton's deterministic modes generate different kernels, so test_environment_determinism.py (584 s, the long pole of [2/3]) recompiles on every PR	Add test_environment_determinism.py and test_contrib_environments_kitless.py to the warmer. The old -k prefix matched every test in both existing files, so the expression is now not (Soft or Cloth or Cable)
Standalone demo workers	The two standalone jobs (15.6 / 13.5 min median) each run one file split over only 2 xdist workers. CI per-case times sum to 1056 s / 965 s, while the longest case is only 240 s / 141 s	Use 4 workers and 4 slots. Other package jobs already run four simulator processes at once on these runners

No test, source, or production code changes. CI only: .github/workflows/build.yaml and tools/test_settings.py.

Validation

All local runs: RTX PRO 6000 Blackwell, taskset -c 0-15 to match the g7.4xlarge's 16 vCPUs, from a clean uv sync --frozen worktree.

Warp warmer. I filled a fresh cache with exactly the old warmer selection, which gave 204 MB (CI: 211 MB). Then I ran the determinism file twice:

test_environment_determinism.py	Wall	Open-Drawer run_to_run case
After old warmer (what PRs get today)	344 s	165 s
Fully cold	354 s	177 s
After its kernels are cached (with this PR)	92 s	38 s

test_contrib_environments_kitless.py: 244 s cold vs 78 s warm, 14/14 passed both times. --collect-only with the new expression selects the same 36 newton/multi-agent tests as before, plus 7 determinism and 14 contrib tests.

Contrib envs without cuRobo (cuRobo is not installed in the worktree): kitless 14 passed, kit 29 passed / 4 skipped, kit_cameras 6 passed. These match the counts in the cuRobo-image CI job. The collector selects the same 3 files for the contrib job and the same 58 files for the isaaclab_tasks shards as before.

Standalone demos, after one warm-up pass:

Group	2 workers	4 workers	Results	Peak host RAM / VRAM at 4 workers
Kit	195 s	105 s	59 passed both	35 GB / 22 GB
non-Kit	195 s	123 s	53 passed both	17 GB / 17 GB

On CI's own per-case durations, xdist-style greedy assignment reproduces today's file times (489 s / 630 s, observed 493 s / 634 s). With 4 workers it predicts 261 s / 411 s before contention.

pre-commit passes on the changed files. End-to-end CI savings still need a run-ci comparison.

Findings left for follow-ups

Measured during the sweep, not changed here:

• Image pull, ~4.2 min in every GPU job. The runners are ephemeral (empty Docker store, 0 Already exists layers). Two large layers dominate: about 2 min to download, then about 2.2 min of sequential extraction. Fixing this is runner infrastructure: base layers baked into the AMI, an in-VPC registry mirror, SOCI lazy loading on ECR, or zstd layers.
• Runner pickup, median 2.7 min (p90 4.6 min) between the image build finishing and test jobs starting.
• Renderer serialization. The scheduler never overlaps two renderer files, partly because OVRTX logs to one fixed /tmp/ovrtx_renderer.log. Simulating two concurrent renderers with this run's file times: isaaclab_visualizers 550→304 s, rendering-correctness 522→284 s, rendering-correctness-kitless (legacy) 679→349 s, before GPU contention. These three become the tail after this PR. Overlapping them needs per-process OVRTX log paths.
• Duration-blind sharding and ordering. Shards are round-robin over sorted paths, and files start alphabetically. Longest-first ordering alone saves about 60 s each in core 1/3, assets and ov. In isaaclab_tasks [3/3], the 367 s test_environments_isaacsim_physx.py and 75 s test_franka_cable_partition_bounds.py both start cameras, so they run back to back.
• cuRobo image rebuilds. test-curobo still waits on the cuRobo build. Its cuRobo install runs after the lockfile uv sync, so any lockfile change recompiles cuRobo's CUDA extension.

Type of change

• CI performance improvement

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with uv run isaaclab --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have validated the changed behavior with focused coverage and applied the test-audit criteria to any test changes
• I have added a changelog fragment under source/<pkg>/changelog.d/ for every touched package (no source package touched)
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

This PR shortens CI jobs without changing production code:

• Standalone demo jobs run with four test slots.
• Contributed-environment checks run from the base image.
• The Warp cache warmer covers determinism and kitless contrib tests.

No actionable issues were found. Review checks were static; no tests were run.

StafaH explicitly deferred image pulls, runner pickup, renderer overlap, duration-aware scheduling, and cuRobo rebuilds to follow-ups.

_{Reviews (1) · Last reviewed commit: "Run four standalone demo launches at a t..." · Reviewed by Greptile}

[isaaclab-review-bot]

Isaac Lab Review Bot

All reviewed code categories look good.

• PR description: The description matches the diff and supplies focused hardware, timing, warm-cache, memory, and collection evidence; shorten the unrelated follow-up inventory and retain the explicit note that end-to-end CI comparison remains pending.
• Design and architecture: Architecture looks good.
• Scope and over-engineering: No over-engineering issues.
• API: API changes look good.
• Compatibility and deprecation: No breaking changes identified.
• Implementation: Implementation looks good.
• Style consistency: Style consistency looks good.
• Test quality: Test quality looks good.

No blocking issues. No inline findings.

Automated review; human maintainers own approval decisions.