Skip to content

[CI] Share one path-gating engine across build, install, and Newton workflows - #6465

Closed
hujc7 wants to merge 16 commits into
isaac-sim:developfrom
hujc7:jichuanh/detect-changes-action
Closed

hujc7 wants to merge 16 commits into
isaac-sim:developfrom
hujc7:jichuanh/detect-changes-action

Conversation

@hujc7

@hujc7 hujc7 commented Jul 10, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Extracts the changed-paths gating engine — duplicated as near-identical ~80-line blocks in three workflows — into a single reusable workflow .github/workflows/detect-changes.yml (net -98 lines, and the next gated workflow gets it for free).
  • The gate job keeps its original shape exactly: one ubuntu-latest job with a single API-only step and no checkout — workflow_call workflows are fetched server-side from the caller's own commit, unlike local composite actions which need the repo on disk.
  • Behavior-preserving per caller: identical pattern lists, identical fail-open semantics (non-PR events and changed-file listing failures default to run), and the skip-reports-green contract for branch protection is untouched.

Stacked on #6355 — review the last two commits; the rest is the base PR. Commit 1 is the composite-action prototype, commit 2 reworks it as a reusable workflow to keep the checkout-free gate job; kept separate for comparison.

1. Design

1.1 Reusable workflow over composite action

A composite action would force each caller to add a checkout step (local ./ actions resolve against the runner workspace). workflow_call keeps the gate byte-for-byte equivalent to the previous inline jobs. This is the repo's first reusable workflow; it is GitHub's intended mechanism for a callable job.

1.2 Inputs / output

  • patterns: one regex<whitespace>description per line (regexes are whitespace-free; the first whitespace run is the column separator — avoids the tab-quoting fragility of the previous bash arrays).
  • summary-title / summary-note: absorb the per-workflow summary wording, including build.yaml's push-vs-PR "Triggered jobs" line via a caller-side expression.
  • Single run output; consumers reference needs.changes.outputs.run (five mechanical renames from the old per-workflow output names).

1.3 Self-triggering

Each workflow's pattern list includes ^\.github/workflows/detect-changes\.yml$, so changes to the gate re-run the gated jobs.

2. Validation

  • All four YAML files parse; caller wiring and the absence of stale output references verified programmatically.
  • The engine script was executed directly against all three pattern inputs with 12 simulated cases (pipe-containing regexes, push events, failing gh api) — decisions match the previous per-workflow implementations exactly.
  • Pre-commit passes on all files.

3. Notes

  • The gate check is now reported as "Detect Changes / Detect Changes" (caller job / called job). It is not in the required-checks set, so branch protection is unaffected.
  • install-ci's step summary previously printed only the match table; it now also gets the standard heading and decision line (cosmetic unification).

hujc7 added 13 commits July 4, 2026 11:38
Prepare an Ubuntu-based image for headless Newton training without
the Isaac Sim runtime. Document local build and validation while keeping
the image independent of existing workflows.
Build docker/Dockerfile.newton from the public ubuntu:24.04 base and
run the documented cartpole training probe plus a non-root runtime
check, mirroring the folded verification steps used for the base and
cuRobo images. Also pin the builder venv to the system Python 3.12 so
it cannot drift from the runtime stage's libpython3.12, and fix the
copyright header year range.
Force run_docker_tests=false in build.yaml's changes job and
run_install_tests=false in install-ci.yml so all gated jobs skip via
their existing if-gates, and run build-newton unconditionally on PRs.
Must be reverted before final review.
The bare train/play console scripts were removed from the isaaclab
package by the central pyproject consolidation (isaac-sim#6009); the only
remaining entry point is isaaclab. CI builds the PR merge commit, so
the kitless image no longer contains .venv/bin/train and
'docker run ... train' fails with 'executable file not found'.
Switch the docs verification command and the CI smoke test to
'isaaclab train', matching the uv workflow docs.
The kitless image copied environment.yml and tools/ into the
dependency-install layer, although the install path reads neither:
version pins come from [tool.isaaclab.versions] in the root
pyproject.toml, and environment.yml is only read by the conda envs
command. Both invalidated the cached install layer on unrelated
edits. Also drop the uv cache clean step, a no-op under
UV_NO_CACHE=1, and document why apps/ must stay: isaaclab
parses apps/isaaclab.python.kit at import time to resolve the
Nucleus asset root.
Rebuilding the Newton image from scratch on every triggered run
re-downloads the full wheel set and occupies a GPU runner for the
whole build. Reuse the ecr-build-push-pull action like the base and
curobo images: an existing per-commit or deps-hash tag in ECR skips
the build entirely, and full builds reuse the registry layer cache.

The training smoke now mounts source/, scripts/ and apps/ from the
checkout so a deps-cached image still runs the PR's code; this is
sound because the isaaclab packages are installed editable from
source/.
The Newton kitless job validates the containerized install/build
path; library code is covered by the test matrix and the native
install CI. Give it a dedicated run_newton_tests gate that fires
only on install inputs (Dockerfile.newton, isaaclab.sh, root and
per-package pyproject.toml, the install CLI implementation, and the
job's own wiring), aligned with the deps-hash inputs of the ECR
cache action, so unrelated source PRs no longer occupy a GPU runner.

Also add pyproject.toml and isaaclab.sh to the shared docker-test
patterns so dependency pin bumps trigger the docker jobs at all.

Dependency drift with no repo change is accepted to surface on the
next install-input PR rather than on a schedule.
The ECR login lives in a temporary docker config that the action
removes in its cleanup step, so a job step running after the action
cannot pull. On a deps-cache hit the action only aliased the commit
tag registry-side, leaving jobs that docker-run the image in later
steps of the same job without a local copy and without credentials
to fetch one.

Add an opt-in pull-on-deps-hit input (default false, no behavior
change for existing callers) that pulls and tags the aliased image
while the login is still valid, and reduce the Newton job's local
image check to a fail-fast assertion.
The Newton kitless job is an install/build test of the shipped
container artifact, matching the Installation Tests workflow in
shape rather than the build.yaml docker jobs that feed the test
matrix. Give it a dedicated workflow with its own single-output
changes gate and concurrency group, and revert build.yaml's changes
job to its original single-gate form (keeping the pyproject.toml and
isaaclab.sh trigger patterns, which the other images consume too).

The job's steps, the ecr-build-push-pull usage with pull-on-deps-hit,
the per-commit tag scheme, and the ECR cache tags are unchanged, so
cache history carries over.
The Newton docker workflow carries its own pyproject.toml and
isaaclab.sh trigger patterns, so the copies added to build.yaml's
shared docker-test gate no longer serve this PR. Removing them
leaves build.yaml untouched by the PR; widening the shared gate for
the other images can be considered separately.
Three workflows (build.yaml, install-ci.yml, newton-docker.yml)
carried near-identical ~80-line copies of the changed-paths gating
engine, differing only in pattern lists, output names, and summary
wording. Move the engine into .github/actions/detect-changes with
patterns/summary-title/summary-note inputs and a single run output;
each caller shrinks to a sparse checkout plus one action call, with
its existing output name mapped so downstream if: conditions are
untouched.

Gating semantics are preserved per workflow: same patterns, fail-open
on non-PR events and on changed-file listing failures, and the
skip-reports-green contract for branch protection.
@github-actions github-actions Bot added documentation Improvements or additions to documentation isaac-lab Related to Isaac Lab team infrastructure labels Jul 10, 2026
hujc7 added 3 commits July 10, 2026 21:19
The composite-action form required each caller to add a checkout
step, since local ./ actions resolve against the runner workspace.
A workflow_call reusable workflow is fetched server-side from the
caller's own commit, so the gate job keeps its original shape: a
single API-only step with no checkout.

Callers shrink to a uses: job with the patterns input; consumers
reference the generic run output instead of the per-workflow output
names (five mechanical renames). The gate check is now reported as
'Detect Changes / Detect Changes'; it is not a required check, so
branch protection is unaffected.
@StafaH

StafaH commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

Resolved by #6699

@StafaH StafaH closed this Sep 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation infrastructure isaac-lab Related to Isaac Lab team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants