Skip to content

Precompile common operations in the oneAPI extension - #198

Open
maleadt wants to merge 1 commit into
mainfrom
tb/oneapi-workload
Open

maleadt wants to merge 1 commit into
mainfrom
tb/oneapi-workload

Conversation

@maleadt

@maleadt maleadt commented Oct 28, 2026

Copy link
Copy Markdown
Member

oneAPI can't precompile AcceleratedKernels' operations for this extension: the extension isn't loaded while oneAPI precompiles, so kernels cached there would use the default tunings rather than the ones set here (predicate_tuning even selects different kernels for any/all).

When oneAPI can use the GPU while precompiling, which it reports with oneAPI.can_precompile (JuliaGPU/oneAPI.jl#753), this runs sum of a vector and along each dimension of a matrix, cumsum, any, all and an Int sum in a @compile_workload of the extension, at sizes that select the same kernels as a vector of 10^6 elements and a 1000×1000 matrix (checked by recording the kernels compiled at both sizes). With oneAPI versions without can_precompile the workload is skipped. Like oneAPI's own workload it needs Julia 1.12.

On an Arc A770 with Julia 1.13, together with that oneAPI PR, the first sum of 10^6 elements takes 0.17 instead of 1.2 s, sum along both dimensions 0.25 instead of 2.0 s, cumsum 0.41 instead of 2.2 s and all 0.25 instead of 1.7 s. The extension's image grows by 11 MB, its precompilation from 1.4 to 6.9 s, and using oneAPI takes about 80 ms longer. Most of what remains is oneAPI compiling the kernels' SPIR-V in every session, as it doesn't persist compiled kernels.

oneAPI can't precompile these itself: this extension isn't loaded while
oneAPI precompiles, so the kernels it would cache use the default
tunings rather than the ones set here. When oneAPI can use the GPU
while precompiling, run `sum` of a vector and along each dimension of
a matrix, `cumsum`, `any` and `all` in a `@compile_workload` of the
extension, at sizes that use the same kernels as arrays of millions of
elements.

On an Arc A770 with Julia 1.13, the first `sum` of a million elements
takes 0.17 s rather than 1.2 s, and the first `cumsum` 0.41 s rather
than 2.2 s, for an extension image that is 11 MB larger and about 80 ms
more load time.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant