[maleadt]

The call generator zipped tuples of argument expressions, recompiling for each new number of kernel arguments. Build the call with vectors, as JuliaGPU/KernelAbstractions.jl#931 does for PoCL.

Precompile pool_free, as CUDA.jl does, to avoid about 0.9 s of compilation during the first garbage collection that frees a ROCArray.

Also precompile hipfunction_link for the name keyword argument passed by KernelInterface.kernel_function. The existing directive misses this argument type, causing compilation on the first KernelAbstractions launch.

Median over 5 interleaved fresh processes (9950X iGPU, gfx1036; Julia 1.12.7, KernelAbstractions 0.10.3, KernelInterface 0.4.3; base 3e70165):

	before	after
first KernelAbstractions launch in a session	2689 ms	2426 ms
launch of a new kernel with a new number of arguments	230 ms	160 ms
launch of a new kernel with a known number of arguments	123 ms	107 ms
first GC.gc() that frees a ROCArray	1113 ms	230 ms

The "after" runs also include #1196, which only changes per-launch cost.

Tests: the full suite on gfx1036 (30553 passed, 14 broken), including kernelabstractions_tests and kernelinterface_tests.

[luraess]

cscs-ci run MI250,MI300

[github-actions]

Benchmark results

Compared with 3e7016582, the merge base.
The environment differs from the baseline's: AMDGPUExt 0.5.9 → 0.10.3, AcceleratedKernels 0.5.9 → 0.5.10, Atomix 1.6.0 → 1.7.0, BFloat16sExt 10.4.1 → 10.4.2, JSON 1.12.2 → 1.13.0, LLVM 10.4.1 → 10.4.2, Parsers 3.0.1 → 3.0.2, StructUtils 2.11.1 → 2.11.2, StructUtilsStaticArraysCoreExt 2.11.1 → 2.11.2, StructUtilsTablesExt 2.11.1 → 2.11.2, pocl_standalone_jll 7.2.3+1 → 7.2.3+2.

Times are the minimum time of every benchmark. A change is unusual when it exceeds both three times the benchmark's typical change between consecutive runs on the default branch and a minimum per category (performance 3%, latency 5%, applications 3%, other 3%). A benchmark that starts or stops allocating is listed however few the allocations. A single run can still be a fluke, so rerun before drawing conclusions.

	Geometric mean	Unusual: slower	Unusual: faster	Compared
Performance	−2.4%	3	10	69
Latency	−2.0%	0	0	3
Applications	−2.2%	0	1	14

Benchmark	Baseline	This PR	Change
Performance
array/copyto!/gpu_to_gpu	35.1 µs	58.8 µs	+67.4%	🔴 slower
array/reverse/1dL_inplace	78.8 µs	55.0 µs	−30.2%	🟢 faster
array/random/rand!/Float32	50.9 µs	66.0 µs	+29.7%	🔴 slower
kernel/rand	48.3 µs	38.3 µs	−20.6%	🟢 faster
array/reductions/reduce/Float32/dims=2L	72.5 µs	58.9 µs	−18.8%	🟢 faster
array/iteration/findmin/2d	79.3 µs	65.1 µs	−17.9%	🟢 faster
array/accumulate/Int64/1d	104 µs	86.0 µs	−17.4%	🟢 faster
array/random/rand/Int64	46.4 µs	38.6 µs	−16.7%	🟢 faster
array/accumulate/Int64/dims=2	91.0 µs	109 µs	+19.5%	🔴 slower
array/random/rand/Float32	46.9 µs	39.3 µs	−16.1%	🟢 faster
array/accumulate/Int64/dims=1L	111 µs	93.5 µs	−15.6%	🟢 faster
array/sorting/1d	343 µs	296 µs	−13.6%	🟢 faster
array/reductions/mapreduce/Int64/dims=2	60.7 µs	57.3 µs	−5.7%	🟢 faster
Applications
convolution	76.0 µs	61.7 µs	−18.8%	🟢 faster

All benchmarks

Benchmark	Baseline	This PR	Change
Performance
amdgpu/synchronization/context/device	1.79 µs	1.78 µs	−0.7%
amdgpu/synchronization/stream/blocking	228 ns	223 ns	−2.2%
amdgpu/synchronization/stream/nonblocking	350 ns	343 ns	−2.1%
array/accumulate/Float32/1d	82.6 µs	81.7 µs	−1.0%
array/accumulate/Float32/dims=1	64.2 µs	65.9 µs	+2.7%
array/accumulate/Float32/dims=1L	105 µs	107 µs	+1.3%
array/accumulate/Float32/dims=2	105 µs	105 µs	+0.5%
array/accumulate/Float32/dims=2L	124 µs	123 µs	−1.1%
array/accumulate/Int64/1d	104 µs	86.0 µs	−17.4%	🟢 faster
array/accumulate/Int64/dims=1	75.1 µs	76.3 µs	+1.7%
array/accumulate/Int64/dims=1L	111 µs	93.5 µs	−15.6%	🟢 faster
array/accumulate/Int64/dims=2	91.0 µs	109 µs	+19.5%	🔴 slower
array/accumulate/Int64/dims=2L	157 µs	156 µs	−0.7%
array/broadcast	50.6 µs	50.5 µs	−0.1%
array/construct	2.29 µs	2.18 µs	−5.0%
array/copy	40.3 µs	39.5 µs	−2.2%
array/copyto!/cpu_to_gpu	85.9 µs	86.0 µs	+0.1%
array/copyto!/gpu_to_cpu	85.8 µs	85.9 µs	+0.1%
array/copyto!/gpu_to_gpu	35.1 µs	58.8 µs	+67.4%	🔴 slower
array/iteration/findall/bool	94.7 µs	89.4 µs	−5.6%
array/iteration/findall/int	107 µs	104 µs	−2.8%
array/iteration/findfirst/bool	54.9 µs	52.1 µs	−5.0%
array/iteration/findfirst/int	56.6 µs	59.6 µs	+5.3%
array/iteration/findmin/1d	95.0 µs	96.8 µs	+1.9%
array/iteration/findmin/2d	79.3 µs	65.1 µs	−17.9%	🟢 faster
array/iteration/logical	91.0 µs	85.8 µs	−5.7%
array/iteration/scalar	307 µs	306 µs	−0.6%
array/permutedims/2d	62.1 µs	62.3 µs	+0.3%
array/permutedims/3d	49.4 µs	59.7 µs	+20.7%
array/permutedims/4d	54.1 µs	54.2 µs	+0.0%
array/random/rand!/Float32	50.9 µs	66.0 µs	+29.7%	🔴 slower
array/random/rand!/Int64	59.8 µs	58.6 µs	−2.1%
array/random/rand/Float32	46.9 µs	39.3 µs	−16.1%	🟢 faster
array/random/rand/Int64	46.4 µs	38.6 µs	−16.7%	🟢 faster
array/random/randn!/Float32	82.6 µs	72.2 µs	−12.5%
array/random/randn/Float32	70.5 µs	67.2 µs	−4.6%
array/reductions/mapreduce/Float32/1d	71.0 µs	71.2 µs	+0.2%
array/reductions/mapreduce/Float32/dims=1	55.5 µs	54.0 µs	−2.7%
array/reductions/mapreduce/Float32/dims=1L	59.7 µs	58.0 µs	−2.9%
array/reductions/mapreduce/Float32/dims=2	59.6 µs	57.3 µs	−3.7%
array/reductions/mapreduce/Float32/dims=2L	60.7 µs	58.7 µs	−3.2%
array/reductions/mapreduce/Int64/1d	71.7 µs	71.6 µs	−0.2%
array/reductions/mapreduce/Int64/dims=1	51.6 µs	50.3 µs	−2.6%
array/reductions/mapreduce/Int64/dims=1L	70.9 µs	69.8 µs	−1.6%
array/reductions/mapreduce/Int64/dims=2	60.7 µs	57.3 µs	−5.7%	🟢 faster
array/reductions/mapreduce/Int64/dims=2L	78.4 µs	75.9 µs	−3.3%
array/reductions/reduce/Float32/1d	70.9 µs	70.6 µs	−0.4%
array/reductions/reduce/Float32/dims=1	55.7 µs	54.2 µs	−2.7%
array/reductions/reduce/Float32/dims=1L	50.4 µs	48.6 µs	−3.5%
array/reductions/reduce/Float32/dims=2	59.4 µs	57.2 µs	−3.7%
array/reductions/reduce/Float32/dims=2L	72.5 µs	58.9 µs	−18.8%	🟢 faster
array/reductions/reduce/Int64/1d	71.4 µs	69.0 µs	−3.3%
array/reductions/reduce/Int64/dims=1	51.6 µs	50.1 µs	−3.1%
array/reductions/reduce/Int64/dims=1L	70.9 µs	69.5 µs	−2.0%
array/reductions/reduce/Int64/dims=2	60.6 µs	57.7 µs	−4.8%
array/reductions/reduce/Int64/dims=2L	78.2 µs	75.5 µs	−3.4%
array/reverse/1d	37.3 µs	37.2 µs	−0.4%
array/reverse/1d_inplace	40.3 µs	44.5 µs	+10.5%
array/reverse/1dL	60.5 µs	68.2 µs	+12.7%
array/reverse/1dL_inplace	78.8 µs	55.0 µs	−30.2%	🟢 faster
array/reverse/2d	47.8 µs	47.3 µs	−1.0%
array/reverse/2d_inplace	56.1 µs	42.8 µs	−23.8%
array/reverse/2dL	80.5 µs	75.1 µs	−6.7%
array/reverse/2dL_inplace	90.5 µs	68.2 µs	−24.7%
array/sorting/1d	343 µs	296 µs	−13.6%	🟢 faster
kernel/indexing	32.1 µs	31.1 µs	−3.1%
kernel/indexing_checked	37.0 µs	57.8 µs	+56.2%
kernel/launch	900 ns	893 ns	−0.8%
kernel/rand	48.3 µs	38.3 µs	−20.6%	🟢 faster
Latency
import	1.37 s	1.35 s	−1.7%
precompile	29.0 s	28.2 s	−2.9%
ttfp	1.77 s	1.74 s	−1.3%
Applications
bitonic_sort	1.11 ms	1.02 ms	−8.0%
byval/reference	39.2 µs	40.9 µs	+4.4%
byval/slices=1	41.1 µs	40.6 µs	−1.3%
byval/slices=2	158 µs	144 µs	−9.0%
byval/slices=3	233 µs	236 µs	+1.4%
convolution	76.0 µs	61.7 µs	−18.8%	🟢 faster
floyd_warshall	9.06 ms	8.98 ms	−0.9%
gemm/tiled	2.75 ms	2.77 ms	+0.9%
gemm/tiled_unbounded	2.79 ms	2.78 ms	−0.3%
histogram	782 µs	794 µs	+1.7%
prefix_sum	194 µs	194 µs	+0.0%
stencil/diffusion3d	1.57 ms	1.56 ms	−0.4%
stencil/diffusion3d_checked	1.61 ms	1.63 ms	+0.6%
volumerhs	4.86 ms	4.94 ms	+1.8%

History: https://juliagpu.org/perf/?pkg=amdgpu