Skip to content

Install only the ROCm that CI needs and pull images in the background #255

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
derobins merged 1 commit into from
Oct 18, 2026
Merged

Install only the ROCm that CI needs and pull images in the background #255

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions .github/actions/install-rocm/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
---
name: Install ROCm
description: >-
Install the HIP runtime and amdclang, and nothing else of ROCm, from
AMD's package repository, in an Ubuntu container job. hipObject is
host-only, so it needs no more, and these packages are a ~330 MB
download, where the rocm/dev-ubuntu full images are ~9 GB.

inputs:
version:
description: >-
ROCm's major and minor version, such as 10.1. Only ROCm 10.0 and
later, which come from stable.repo.amd.com.
required: true

outputs:
rocm_path:
description: Directory ROCm is installed in, /opt/rocm/core-<version>
value: ${{ steps.install.outputs.rocm_path }}

runs:
using: composite
steps:
# The packages install ROCm into /opt/rocm/core-<version>. They don't
# make the links in /opt/rocm that the full images have (ROCm's larger
# packages, such as amdrocm-core, make those), so everything uses the
# version's own directory, as hipobject-nightly.yml does.
#
# apt only trusts the key with the known fingerprint, and only for this
# repository. Exporting the key by its fingerprint from a scratch
# keyring leaves out any other key in the download, and exports nothing
# if the key isn't there.
#
# amdrocm-runtime-dev is HIP's headers and CMake package, and brings in
# the runtime. amdrocm-llvm is amdclang, which the runtime already
# needs; it's named because the builds use it. Their recommendations
# (gcc, g++, and their multilib packages) are left out: the job installs
# the compilers it wants. It should do that first, since the compiler
# packages need some release of libstdc++'s development package, and apt
# otherwise picks the oldest that the Ubuntu release has.
#
# The packages don't add ROCm's lib directory to the loader cache, so
# programs linked with libamdhip64, such as the tests, find it through
# LD_LIBRARY_PATH.
- id: install
shell: bash
env:
ROCM_MAJOR_MINOR: ${{ inputs.version }}
# apt takes the packages' defaults rather than asking questions
DEBIAN_FRONTEND: noninteractive
# The fingerprint of AMD's signing key for the repository, which is
# public
AMD_SIGNER_FINGERPRINT: D0F004A0025A1145C7807FCD0701EAC4D5E02107
run: |
set -euo pipefail

if [[ ! "${ROCM_MAJOR_MINOR}" =~ ^([0-9]+)\.[0-9]+$ ]] \
|| (( BASH_REMATCH[1] < 10 )); then
echo "::error::version must be 10.0 or later, as major.minor, not '${ROCM_MAJOR_MINOR}'"
exit 1
fi

# The repository has a directory for each Ubuntu release, such as
# ubuntu2604
ubuntu_version=$(awk -F= '$1 == "VERSION_ID" { gsub(/"/, "", $2); print $2 }' \
/etc/os-release)
repo="https://stable.repo.amd.com/rocm/core/packages/ubuntu${ubuntu_version//./}"

# gpg imports the key without gpg-agent, but then exits with status 2
# because it can't start the agent, so the agent is installed too.
apt-get update
apt-get install -y --no-install-recommends \
ca-certificates curl gpg gpg-agent

curl -fsSL https://stable.repo.amd.com/rocm/gpg/packages.gpg \
-o /tmp/amdrocm.asc
GNUPGHOME=$(mktemp -d)
export GNUPGHOME
gpg --batch --import /tmp/amdrocm.asc
install -d -m 0755 /etc/apt/keyrings
gpg --batch --export "${AMD_SIGNER_FINGERPRINT}" \
> /etc/apt/keyrings/amdrocm.gpg
if [[ ! -s /etc/apt/keyrings/amdrocm.gpg ]]; then
echo "::error::AMD's ROCm repository key isn't ${AMD_SIGNER_FINGERPRINT}"
exit 1
fi
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/amdrocm.gpg]" \
"${repo} stable main" > /etc/apt/sources.list.d/amdrocm.list

apt-get update
apt-get install -y --no-install-recommends \
"amdrocm-runtime-dev${ROCM_MAJOR_MINOR}" \
"amdrocm-llvm${ROCM_MAJOR_MINOR}"

rocm_path="/opt/rocm/core-${ROCM_MAJOR_MINOR}"
for compiler in amdclang amdclang++; do
if [[ ! -x "${rocm_path}/llvm/bin/${compiler}" ]]; then
echo "::error::${compiler} isn't in ${rocm_path}/llvm/bin"
exit 1
fi
done
"${rocm_path}/llvm/bin/amdclang++" --version

{
echo "ROCM_PATH=${rocm_path}"
echo "LD_LIBRARY_PATH=${rocm_path}/lib"
} >> "${GITHUB_ENV}"
echo "rocm_path=${rocm_path}" >> "${GITHUB_OUTPUT}"
66 changes: 34 additions & 32 deletions .github/workflows/hipobject-build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,10 @@ on:
# used with HIPOBJ_BUILD_DOCS, which is OFF here (the Documentation
# Check lane configures and builds those), the .clang-tidy files only
# configure the clang-tidy lane, and .github/ holds the other
# workflows and their configuration. This workflow's own file is added
# back last; a later pattern overrides an earlier one, which is why
# this is a paths list with negations and not paths-ignore, which
# can't add a path back.
# workflows and their configuration. This workflow's own file, and the
# action that installs ROCm, are added back last; a later pattern
# overrides an earlier one, which is why this is a paths list with
# negations and not paths-ignore, which can't add a path back.
pull_request:
branches: [develop]
paths: &paths
Expand All @@ -27,6 +27,7 @@ on:
- '!requirements.txt'
- '!.github/**'
- '.github/workflows/hipobject-build.yml'
- '.github/actions/install-rocm/**'
push:
branches: [develop]
paths: *paths
Expand Down Expand Up @@ -61,9 +62,10 @@ jobs:
# The jobs are listed one by one rather than made by crossing axes,
# since most of them change one thing from the first job, and
# crossing would build combinations nobody needs. Every job sets
# label, ubuntu, cc, cxx, and build_type. A job that leaves out one
# of the other keys gets its default:
# label, ubuntu, and build_type. A job that leaves out one of the
# other keys gets its default:
#
# cc, cxx ROCm's amdclang and amdclang++
# shared_libs ON. TheRock only builds shared libraries.
# sanitizer None. asan is HIPOBJ_USE_SANITIZERS (ASan, UBSan,
# and the compiler's other checks), and tsan is
Expand All @@ -74,65 +76,49 @@ jobs:
# HIPOBJ_IONIC
# packages None. Extra apt packages, such as a compiler.
#
# Every job uses ROCm 10.1.0, the latest ROCm (see the image below).
# Every job uses ROCm 10.1, the latest ROCm (see Install ROCm below).
matrix:
include:
# The core jobs build with ROCm's amdclang on the newest Ubuntu
# release, as TheRock will build hipObject.
- label: amdclang, RelWithDebInfo
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: RelWithDebInfo
# Debug, so the library's assert()s are checked with the
# sanitizers on, and the reports aren't muddied by optimization
- label: amdclang, Debug, ASan and UBSan
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: Debug
sanitizer: asan
# RelWithDebInfo, as TheRock's ThreadSanitizer builds are, so the
# threads' timing is closer to a normal build's
- label: amdclang, RelWithDebInfo, TSan
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: RelWithDebInfo
sanitizer: tsan
# Debug, so the report's lines match the source. Coverage needs
# Clang.
- label: amdclang, Debug, coverage
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: Debug
coverage: ON
# Smoke tests of what the core jobs don't build. Release is
# TheRock's default build type.
- label: amdclang, Release, static
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: Release
shared_libs: OFF
- label: amdclang, no NIC backends
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: RelWithDebInfo
backends: OFF
- label: amdclang, C++23
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: RelWithDebInfo
cxx_standard: 23
# C++26 needs CMake 3.30 or later, which Ubuntu 24.04 doesn't have
- label: amdclang, C++26
ubuntu: '26.04'
cc: /opt/rocm/llvm/bin/amdclang
cxx: /opt/rocm/llvm/bin/amdclang++
build_type: RelWithDebInfo
cxx_standard: 26
# Upstream Clang, the version Ubuntu installs as clang
Expand Down Expand Up @@ -169,14 +155,15 @@ jobs:
cxx: clang++-18
build_type: RelWithDebInfo
packages: clang-18
# A plain Ubuntu image, with only the ROCm that hipObject needs
# installed (see Install ROCm below)
container:
image: rocm/dev-ubuntu-${{ matrix.ubuntu }}:10.1.0-full
image: ubuntu:${{ matrix.ubuntu }}
options: --user root
env:
ROCM_PATH: /opt/rocm
# The full ROCm image ships libamdhip64 in /opt/rocm/lib but leaves it out
# of the loader cache, so ctest binaries fail to start without this.
LD_LIBRARY_PATH: /opt/rocm/lib
# Packages such as tzdata ask questions when they're installed into a
# plain image; this takes their defaults instead.
DEBIAN_FRONTEND: noninteractive
# The tests that load the system's libibverbs fail, rather than skip,
# if it doesn't load (see test/unit/require-libibverbs.h)
HIPOBJ_TEST_REQUIRE_LIBIBVERBS: '1'
Expand All @@ -185,11 +172,14 @@ jobs:
with:
persist-credentials: false

# build-essential is the release's GCC (GCC 15 on 26.04, GCC 13 on
# 24.04) and make. amdclang uses GCC's libstdc++ too.
#
# rpm provides rpmbuild, so hipObject can build RPM packages here
# as well as DEB packages, and the RPM package test runs.
#
# The MinIO bridge needs nlohmann-json3-dev and libpugixml-dev.
# libcurlpp-dev saves downloading and building curlpp.
# The MinIO bridge needs nlohmann-json3-dev, libpugixml-dev, and
# zlib1g-dev. libcurlpp-dev saves downloading and building curlpp.
#
# A job's packages (EXTRA_PACKAGES) are separated by spaces, so read
# splits them into an array with one package in each element.
Expand All @@ -200,6 +190,7 @@ jobs:
read -ra extra_packages <<< "${EXTRA_PACKAGES}"
apt-get update
apt-get install -y \
build-essential \
cmake \
git \
libcurl4-openssl-dev \
Expand All @@ -209,8 +200,16 @@ jobs:
libssl-dev \
nlohmann-json3-dev \
rpm \
zlib1g-dev \
"${extra_packages[@]}"

# After the compilers, so the libstdc++ that ROCm's compiler
# packages need is the one already installed
- name: Install ROCm
uses: ./.github/actions/install-rocm
with:
version: '10.1'

# HIPOBJ_WERROR makes a new warning fail the build. It only
# covers hipObject's own targets, not the code FetchContent pulls in.
#
Expand All @@ -219,14 +218,17 @@ jobs:
# package tests include it. The coverage report only measures src/,
# so the bridge would only slow that job down.
#
# CMake picks the compilers up from CC and CXX.
# CMake picks the compilers up from CC and CXX. A job that doesn't
# set them gets amdclang, which is in ROCm's directory.
- name: Configure
env:
CC: ${{ matrix.cc }}
CXX: ${{ matrix.cxx }}
run: |
export CC="${CC:-${ROCM_PATH}/llvm/bin/amdclang}"
export CXX="${CXX:-${ROCM_PATH}/llvm/bin/amdclang++}"
cmake -B build \
-DCMAKE_PREFIX_PATH=/opt/rocm \
-DCMAKE_PREFIX_PATH="${ROCM_PATH}" \
-DBUILD_TESTING=ON \
-DHIPOBJ_WERROR=ON \
-DHIPOBJ_BNXT=${{ matrix.backends || 'ON' }} \
Expand Down
44 changes: 30 additions & 14 deletions .github/workflows/hipobject-clang-tidy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@ name: hipObject - clang-tidy
# This uses upstream clang-tidy rather than ROCm's, which is built
# without the clang-analyzer-* checks, as hipFile's clang-tidy workflow
# does. The compiler is still ROCm's amdclang++. Keep LLVM_VERSION at
# the LLVM major version of the image's ROCm where apt.llvm.org has a
# the LLVM major version of the job's ROCm where apt.llvm.org has a
# release of it, so clang-tidy understands the compiler's flags. ROCm
# 10.1 is LLVM 24, which only has snapshot builds, so this uses LLVM 23,
# the newest release. That works while the build uses no warning flag
# newer than Clang 23 (see HIPOBJClangCompilerOptions.cmake).

on:
# The same triggers and paths as the Build workflow, since anything that
# it compiles can have findings, plus this workflow's own file. The
# it compiles can have findings, plus this workflow's own file and the
# action that installs ROCm. The
# .clang-tidy files are matched by '**'. The paths are a list with
# negations and not paths-ignore, so the last entry can add a path back
# (see hipobject-build.yml).
Expand All @@ -33,6 +34,7 @@ on:
- '!requirements.txt'
- '!.github/**'
- '.github/workflows/hipobject-clang-tidy.yml'
- '.github/actions/install-rocm/**'
push:
branches: [develop]
paths: *paths
Expand Down Expand Up @@ -60,11 +62,15 @@ jobs:
clang-tidy:
name: clang-tidy
runs-on: ubuntu-latest
# A plain Ubuntu image, with only the ROCm that hipObject needs
# installed (see Install ROCm below)
container:
image: rocm/dev-ubuntu-26.04:10.1.0-full
image: ubuntu:26.04
options: --user root
env:
ROCM_PATH: /opt/rocm
# Packages such as tzdata ask questions when they're installed into a
# plain image; this takes their defaults instead.
DEBIAN_FRONTEND: noninteractive
LLVM_VERSION: 23
# The Ubuntu release of the image, for apt.llvm.org
UBUNTU_CODENAME: resolute
Expand All @@ -75,15 +81,16 @@ jobs:
with:
persist-credentials: false

# Ninja keeps each command's output together, so clang-tidy's
# output for one file isn't interleaved with another's in the
# build log. The MinIO bridge needs nlohmann-json3-dev and
# libpugixml-dev, and libcurlpp-dev saves downloading and building
# curlpp.
# build-essential is GCC, whose libstdc++ amdclang uses. Ninja keeps
# each command's output together, so clang-tidy's output for one
# file isn't interleaved with another's in the build log. The MinIO
# bridge needs nlohmann-json3-dev, libpugixml-dev, and zlib1g-dev,
# and libcurlpp-dev saves downloading and building curlpp.
- name: Install dependencies
run: |
apt-get update
apt-get install -y \
build-essential \
ca-certificates \
cmake \
curl \
Expand All @@ -95,7 +102,15 @@ jobs:
libpugixml-dev \
libssl-dev \
ninja-build \
nlohmann-json3-dev
nlohmann-json3-dev \
zlib1g-dev

# After the compilers, so the libstdc++ that ROCm's compiler
# packages need is the one already installed
- name: Install ROCm
uses: ./.github/actions/install-rocm
with:
version: '10.1'

# apt only trusts the key with the known fingerprint, and only for
# this repository. Exporting the key by its fingerprint from a
Expand Down Expand Up @@ -130,13 +145,14 @@ jobs:
#
# HIPOBJ_WERROR is on, so a compiler warning fails this job too, as
# it does the Build workflow's jobs.
#
# CMake picks the compilers up from CC and CXX.
- name: Configure
env:
CC: /opt/rocm/llvm/bin/amdclang
CXX: /opt/rocm/llvm/bin/amdclang++
run: |
export CC="${ROCM_PATH}/llvm/bin/amdclang"
export CXX="${ROCM_PATH}/llvm/bin/amdclang++"
cmake -B build -G Ninja \
-DCMAKE_PREFIX_PATH=/opt/rocm \
-DCMAKE_PREFIX_PATH="${ROCM_PATH}" \
-DBUILD_TESTING=ON \
-DHIPOBJ_WERROR=ON \
-DHIPOBJ_BNXT=ON \
Expand Down
Loading
Loading