Skip to content

Precompile common operations for AMDGPU - #197

Open
maleadt wants to merge 1 commit into
mainfrom
tb/amdgpu-workload
Open

maleadt wants to merge 1 commit into
mainfrom
tb/amdgpu-workload

Conversation

@maleadt

@maleadt maleadt commented Oct 28, 2026

Copy link
Copy Markdown
Member

AMDGPU's precompilation workload doesn't use the GPU, which isn't even discovered while AMDGPU precompiles, so the first reduction, scan or predicate of a session compiles AcceleratedKernels' host code and kernels.

This runs them in the workload of a new AMDGPU extension, which precompiles with AMDGPU loaded and ROCm discovered, when there is a device. AMDGPU.functional() only checks that ROCm's libraries were found, so the guard also counts the devices (with the unchecked HIP call; older AMDGPU versions without it skip the workload). The arrays are just large enough to select the kernels that a vector of 10^6 elements and a 1000×1000 matrix do; at AMDGPU's default tunings that takes longer matrix columns than on CUDA. Like AMDGPU's own workload it needs Julia 1.12.

On a gfx1036 with Julia 1.13, the first sum of 10^6 elements takes 17 ms instead of 1.1 s, cumsum 0.21 instead of 1.4 s, any and all 70 ms instead of 1.1 s, and sum along either dimension of a 1000×1000 matrix 0.45 instead of 1.6 s. The extension's image is 9 MB and takes 4 s to precompile; load time doesn't change measurably.

AMDGPU's workload doesn't use the GPU, which isn't discovered while
AMDGPU precompiles, so the first reduction, scan or predicate of a
session compiles AcceleratedKernels' host code and kernels.

Run them in the workload of a new AMDGPU extension, which precompiles
with AMDGPU loaded and ROCm discovered, when there is a device. The
arrays are just large enough to select the kernels that a vector of
10^6 elements and a 1000×1000 matrix do. Like AMDGPU's own workload, it
needs Julia 1.12.

On a gfx1036 with Julia 1.13, the first `sum` of 10^6 elements takes
17 ms instead of 1.1 s, `cumsum` 0.21 s instead of 1.4 s, `any` and
`all` 70 ms instead of 1.1 s, and `sum` along either dimension of a
1000×1000 matrix 0.45 s instead of 1.6 s. The extension's image is 9 MB,
and it takes 4 s to precompile.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant