Skip to content

Install only the ROCm that CI needs and pull images in the background - #255

Merged
derobins merged 1 commit into
developfrom
derobins/faster_ci
Oct 18, 2026
Merged

derobins merged 1 commit into
developfrom
derobins/faster_ci

Conversation

@derobins

Copy link
Copy Markdown
Collaborator

The Build, clang-tidy, IWYU, and MinIO Integration workflows ran in the rocm/dev-ubuntu-26.04:10.1.0-full image (and its 24.04 counterpart), which is 8.9 GB compressed, almost all of it in one layer. Pulling it took 2 to 5 minutes of every job, more than building and testing did.

These jobs now run in plain ubuntu:26.04 or ubuntu:24.04 images, and a new composite action, .github/actions/install-rocm, installs only what hipObject needs from AMD's package repository (stable.repo.amd.com): amdrocm-runtime-dev10.1 (HIP's headers, CMake package, and runtime) and amdrocm-llvm10.1 (amdclang). That's about 330 MB to download. The action trusts only the repository key with the pinned fingerprint, and sets ROCM_PATH and LD_LIBRARY_PATH for the steps after it.

  • These packages install ROCm into /opt/rocm/core-10.1 and don't create the /opt/rocm links that the full images have, so the workflows use the version's own directory, as hipobject-nightly.yml does.
  • Each job installs build-essential before ROCm, so ROCm's compiler packages use the release's libstdc++ (GCC 15 on 26.04, GCC 13 on 24.04).
  • In the Build workflow, amdclang is now the default compiler for jobs that don't set cc and cxx, so the ROCm version appears in one place.
  • The four workflows also run when the action changes.
  • DEBIAN_FRONTEND=noninteractive keeps apt from waiting for answers in the plain images.
  • CI now picks up 10.1.x patch releases from the repository, where the image was pinned to 10.1.0.

AGENTS.md no longer says CI uses the full image.

The five emulated-hardware lanes pulled the ernic image (218 MB) in the foreground when they extracted rocm-ernic from it, and the two GPU lanes pulled the rocjitsu image (1.1 GB) in the foreground when they generated ip_discovery.bin. Both pulls now run as a background task alongside the guest image fetch, and a wait step joins it before the first step that uses them. That took 12 to 15 s off each lane, and about 55 s off the GPU lanes, in recent runs.

Testing: built against only these packages, bind-mounted over /opt/rocm so the build saw only /opt/rocm/core-10.1. With amdclang:

  • RelWithDebInfo with the MinIO bridge and HIPOBJ_WERROR built with no warnings and passed 33/33 tests, including the install, package, and subproject tests.
  • ASan/UBSan, TSan, and coverage builds each passed 32/32, and the coverage report built (96.05% line coverage). I also checked that the pinned key verifies the repository's signed index and that a wrong fingerprint exports no key. actionlint, shellcheck (including the action's script), and codespell are clean.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor
Security scan found no critical or high findings on lines changed by this pull request

4 medium, low, or unrated finding(s) are not listed; the scan run has them.

View the security scan run for complete reports.

“New” means the finding is located in a changed file on an added or modified line; it is not a base-versus-head findings delta.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The installer omits gpg-agent, which can cause key import to abort in clean containers.

1 open issue
What changed in this PR

Reduces hipObject CI startup time by installing minimal ROCm packages and overlapping hardware-test image pulls with guest setup.

Changes:

  • Adds a shared ROCm installer with a pinned signing-key fingerprint.
  • Moves four workflows to plain Ubuntu images and versioned ROCm paths.
  • Starts image pulls in the background across five hardware-test workflows.
File Description
AGENTS.md Updates CI environment documentation.
.github/​workflows/​hipobject-minio-integration-check.yml Uses minimal ROCm installation.
.github/​workflows/​hipobject-iwyu.yml Uses minimal ROCm installation.
.github/​workflows/​hipobject-hardware-test-two-vm-protocol.yml Backgrounds ernic image pull.
.github/​workflows/​hipobject-hardware-test-two-vm-gpu-direct.yml Backgrounds ernic and rocjitsu pulls.
.github/​workflows/​hipobject-hardware-test-s3-backend.yml Backgrounds ernic image pull.
.github/​workflows/​hipobject-hardware-test-minio-bridge.yml Backgrounds ernic image pull.
.github/​workflows/​hipobject-hardware-test-gpu-direct.yml Backgrounds ernic and rocjitsu pulls.
.github/​workflows/​hipobject-clang-tidy.yml Uses minimal ROCm installation.
.github/​workflows/​hipobject-build.yml Uses minimal ROCm and defaults to amdclang.
.github/​actions/​install-rocm/​action.yml Installs ROCm packages and exports paths.

Review effort: Balanced

Comment thread .github/actions/install-rocm/action.yml Outdated
The Build, clang-tidy, IWYU, and MinIO Integration workflows ran in the
rocm/dev-ubuntu-26.04:10.1.0-full image (and its 24.04 counterpart),
which is 8.9 GB compressed, almost all of it in one layer. Pulling it
took 2 to 5 minutes of every job, more than building and testing did.

These jobs now run in plain ubuntu:26.04 or ubuntu:24.04 images, and a
new composite action, .github/actions/install-rocm, installs only what
hipObject needs from AMD's package repository (stable.repo.amd.com):
amdrocm-runtime-dev10.1 (HIP's headers, CMake package, and runtime) and
amdrocm-llvm10.1 (amdclang). That's about 330 MB to download. The
action trusts only the repository key with the pinned fingerprint, and
sets ROCM_PATH and LD_LIBRARY_PATH for the steps after it.

- These packages install ROCm into /opt/rocm/core-10.1 and don't create
  the /opt/rocm links that the full images have, so the workflows use
  the version's own directory, as hipobject-nightly.yml does.
- Each job installs build-essential before ROCm, so ROCm's compiler
  packages use the release's libstdc++ (GCC 15 on 26.04, GCC 13 on
  24.04).
- In the Build workflow, amdclang is now the default compiler for jobs
  that don't set cc and cxx, so the ROCm version appears in one place.
- The four workflows also run when the action changes.
- DEBIAN_FRONTEND=noninteractive keeps apt from waiting for answers in
  the plain images.
- CI now picks up 10.1.x patch releases from the repository, where the
  image was pinned to 10.1.0.

AGENTS.md no longer says CI uses the full image.

The five emulated-hardware lanes pulled the ernic image (218 MB) in the
foreground when they extracted rocm-ernic from it, and the two GPU
lanes pulled the rocjitsu image (1.1 GB) in the foreground when they
generated ip_discovery.bin. Both pulls now run as a background task
alongside the guest image fetch, and a wait step joins it before the
first step that uses them. That took 12 to 15 s off each lane, and
about 55 s off the GPU lanes, in recent runs.

Testing: built against only these packages, bind-mounted over
/opt/rocm so the build saw only /opt/rocm/core-10.1. With amdclang:
- RelWithDebInfo with the MinIO bridge and HIPOBJ_WERROR built with no
  warnings and passed 33/33 tests, including the install, package, and
  subproject tests.
- ASan/UBSan, TSan, and coverage builds each passed 32/32, and the
  coverage report built (96.05% line coverage).
I also checked that the pinned key verifies the repository's signed
index and that a wrong fingerprint exports no key. actionlint,
shellcheck (including the action's script), and codespell are clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Package installation and dependency completeness across both Ubuntu releases still need live CI confirmation.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@derobins
derobins merged commit 7d1d633 into develop Oct 18, 2026
39 checks passed
@derobins
derobins deleted the derobins/faster_ci branch October 18, 2026 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants