Repository navigation
Conversation
Replace early and repeated device-library linking with an ordered late selection. Preserve memory effects and convergence on declarations before optimization, and apply hostcall, wavefront and inlining policy after importing definitions. Use session snapshots of immutable ROCm bitcode to avoid per-launch filesystem checks. Make sincos, signal-store and Boolean wave operations match the actual vendor ABIs, including typed-pointer Julia versions. Pin the supporting GPUCompiler PR with [sources] and cover transitive imports, controls, convergence and runtime isolation.
Member
|
Seems it's the KA many argument test that's failing on CSCS CI |
maleadt
marked this pull request as draft
October 20, 2026 13:55
The reworked GPUCompiler device-library support imports definitions before optimization, as AMDGPU did before, and only imports late what lowering after optimization introduces. Library code is therefore optimized with the kernel again (e.g. loop-invariant math calls are hoisted), so drop what only worked around late-only linking: copying bitcode attributes onto declarations, the byte snapshots and the attribute caches. Describe the language libraries as `always_inline`, keep the HSA signal operations out of line as before, and fold wavefront-size queries and detect hostcalls after both the early (`finish_linked_module!`) and the late (`finish_imports!`) imports. Cache the library selection per ISA and wavefront size to avoid filesystem checks on every compilation.
Pin GPUCompiler to the tb/device-libraries branch rather than a commit, so the source follows JuliaGPU/GPUCompiler.jl#1044 as it is rebased.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AMDGPU links the ROCm device libraries in
finish_module!and in bothlink_libraries!calls, with a fixed-point loop over the vendor bitcode. This PR replaces that with GPUCompiler'sdevice_libraries(job)(JuliaGPU/GPUCompiler.jl#1044). AMDGPU now only lists its libraries; GPUCompiler resolves references across them on demand.The language libraries (hc, hip, opencl, ocml, ockl, irif) are declared as bitcode libraries with
always_inline=true, which keeps the existing inlining policy. Theoclccontrol libraries come last and are not force-inlined. The selection is cached per library directory, ISA and wavefront size, so a compilation doesn't touch the file system.GPUCompiler links these libraries before optimization, as AMDGPU did before. This avoids copying attributes onto declarations up front, which the first version of this PR needed, and which blocked LICM. Definitions imported after optimization, such as runtime functions introduced by GC lowering, go through
finish_imports!. Early imports go throughfinish_linked_module!. Both:llvm.amdgcn.wavefrontsize;__ockl_hsa_signal*out of line, as onmain.Validation
On a gfx1036 GPU, compared to
main:core/device_libraries,core/codegen,device/math,device/wavefrontanddevice/exceptionspass on Julia 1.13 and 1.10, and the hostcall/output checks pass (41 of 41).sincos, which is now inlined instead of calling__ocml_sincos_*.sincosis twice as fast (14.0 → 6.9 ms), and every other case matchesmain. The LICM case (Float64: 465 ms) no longer regresses; it took 2058 ms with the first version of this PR.@roc launch=falsetakes about 110 ns with no allocations, and a launch about 905 ns.main.Depends on JuliaGPU/GPUCompiler.jl#1044. The
[sources]pins inProject.tomlandtest/Project.tomlstill point at an old revision of that PR and need to be updated, then replaced by a compat bound once it is released.