[maleadt]

Every launch takes a global lock, builds the compiler configuration and job, queries GPUCompiler's cache, and then looks the HIPKernel up in _kernel_instances, a dictionary keyed on the kernel's callable. That dictionary keeps every callable alive: a closure launched over a fresh array adds an entry on every launch.

This wraps the lookup in GPUCompiler.cached_lookup (JuliaGPU/GPUCompiler.jl#1061), keyed on the device and the compiler options, and constructs the HIPKernel on each launch instead.

gfx1036 iGPU, Julia 1.13:

	before	after
hipfunction (hit)	0.11 µs, 2892 insns	0.011 µs, 250 insns
@roc	0.84 µs, 9160 insns	0.64 µs, 6600 insns
KernelAbstractions launch	1.17 µs, 15900 insns	1.01 µs, 13250 insns

[github-actions]

AMDGPU.jl Benchmarks

Details

Benchmark suite	Current: 5945b68	Previous: 02ce473	Ratio
amdgpu/synchronization/context/device	1737.5 ns	1770 ns	0.98
amdgpu/synchronization/stream/blocking	227.5 ns	225 ns	1.01
amdgpu/synchronization/stream/nonblocking	347.5 ns	335 ns	1.04
applications/bitonic_sort	1110854.5 ns	1116901.25 ns	0.99
applications/convolution	76256 ns	76561.25 ns	1.00
applications/floyd_warshall	9064347.25 ns	9076293 ns	1.00
applications/histogram	794150.25 ns	797711.5 ns	1.00
applications/prefix_sum	195942.5 ns	194945.25 ns	1.01
array/accumulate/Float32/1d	81511 ns	82208.75 ns	0.99
array/accumulate/Float32/dims=1	63413.25 ns	63535.75 ns	1.00
array/accumulate/Float32/dims=1L	104731.5 ns	104561.75 ns	1.00
array/accumulate/Float32/dims=2	85931 ns	104481.25 ns	0.82
array/accumulate/Float32/dims=2L	106511.25 ns	124041.75 ns	0.86
array/accumulate/Int64/1d	103641.25 ns	104509 ns	0.99
array/accumulate/Int64/dims=1	74538.25 ns	74518.5 ns	1.00
array/accumulate/Int64/dims=1L	110284 ns	111181.75 ns	0.99
array/accumulate/Int64/dims=2	107614 ns	90456.5 ns	1.19
array/accumulate/Int64/dims=2L	155644.5 ns	161315 ns	0.96
array/broadcast	48350.5 ns	58075.75 ns	0.83
array/construct	2135 ns	2240.25 ns	0.95
array/copy	38268 ns	39370.75 ns	0.97
array/copyto!/cpu_to_gpu	86108.5 ns	86096.25 ns	1.00
array/copyto!/gpu_to_cpu	85936.25 ns	86051.25 ns	1.00
array/copyto!/gpu_to_gpu	45805.75 ns	58188.5 ns	0.79
array/iteration/findall/bool	92986.25 ns	93828.75 ns	0.99
array/iteration/findall/int	104476.25 ns	105911.5 ns	0.99
array/iteration/findfirst/bool	54333.25 ns	55656 ns	0.98
array/iteration/findfirst/int	55633.25 ns	54208.25 ns	1.03
array/iteration/findmin/1d	91781.25 ns	94508.75 ns	0.97
array/iteration/findmin/2d	77876 ns	82231.25 ns	0.95
array/iteration/logical	88321.25 ns	92776.5 ns	0.95
array/iteration/scalar	305344 ns	299499.25 ns	1.02
array/permutedims/2d	60358.25 ns	61353.25 ns	0.98
array/permutedims/3d	59483.25 ns	48565.75 ns	1.22
array/permutedims/4d	62993.25 ns	62578.5 ns	1.01
array/random/rand/Float32	47093.25 ns	46603.25 ns	1.01
array/random/rand/Int64	45498 ns	45630.75 ns	1.00
array/random/rand!/Float32	66135.75 ns	42780.75 ns	1.55
array/random/rand!/Int64	46098.25 ns	58325.75 ns	0.79
array/random/randn/Float32	69783.25 ns	70771.25 ns	0.99
array/random/randn!/Float32	69303.5 ns	58726 ns	1.18
array/reductions/mapreduce/Float32/1d	70623.5 ns	70116 ns	1.01
array/reductions/mapreduce/Float32/dims=1	54925.75 ns	55520.75 ns	0.99
array/reductions/mapreduce/Float32/dims=1L	58973.25 ns	59623.25 ns	0.99
array/reductions/mapreduce/Float32/dims=2	58608.25 ns	60480.75 ns	0.97
array/reductions/mapreduce/Float32/dims=2L	71938.5 ns	61478.5 ns	1.17
array/reductions/mapreduce/Int64/1d	71743.5 ns	70951 ns	1.01
array/reductions/mapreduce/Int64/dims=1	51238 ns	51600.75 ns	0.99
array/reductions/mapreduce/Int64/dims=1L	70848.5 ns	71021 ns	1.00
array/reductions/mapreduce/Int64/dims=2	60040.75 ns	61213.25 ns	0.98
array/reductions/mapreduce/Int64/dims=2L	77686 ns	78406 ns	0.99
array/reductions/reduce/Float32/1d	59785.75 ns	70376 ns	0.85
array/reductions/reduce/Float32/dims=1	54985.75 ns	55701 ns	0.99
array/reductions/reduce/Float32/dims=1L	58455.75 ns	51005.5 ns	1.15
array/reductions/reduce/Float32/dims=2	58675.75 ns	59641 ns	0.98
array/reductions/reduce/Float32/dims=2L	71978.5 ns	73648.5 ns	0.98
array/reductions/reduce/Int64/1d	71273.5 ns	70628.5 ns	1.01
array/reductions/reduce/Int64/dims=1	51050.75 ns	51455.75 ns	0.99
array/reductions/reduce/Int64/dims=1L	69881 ns	70796 ns	0.99
array/reductions/reduce/Int64/dims=2	59838.25 ns	60858.5 ns	0.98
array/reductions/reduce/Int64/dims=2L	77168.5 ns	78563.5 ns	0.98
array/reverse/1d	42160.5 ns	36473 ns	1.16
array/reverse/1dL	66968.25 ns	67668.5 ns	0.99
array/reverse/1dL_inplace	77936 ns	77413.5 ns	1.01
array/reverse/1d_inplace	39233 ns	47360.5 ns	0.83
array/reverse/2d	47325.5 ns	46673.25 ns	1.01
array/reverse/2dL	75013.5 ns	77341 ns	0.97
array/reverse/2dL_inplace	55315.75 ns	89556.5 ns	0.62
array/reverse/2d_inplace	49630.5 ns	56001 ns	0.89
array/sorting/1d	341189.25 ns	343477.5 ns	0.99
gemm/tiled	1869891.75 ns	2769699.75 ns	0.68
gemm/tiled_unbounded	1883714.5 ns	2762064.75 ns	0.68
integration/byval/reference	40800 ns	40050 ns	1.02
integration/byval/slices=1	41060 ns	40701 ns	1.01
integration/byval/slices=2	158302 ns	158743 ns	1.00
integration/byval/slices=3	231083 ns	233063 ns	0.99
integration/volumerhs	4858612 ns	4863370 ns	1.00
kernel/indexing	32248 ns	41963.25 ns	0.77
kernel/indexing_checked	49535.75 ns	58815.75 ns	0.84
kernel/launch	895 ns	1255 ns	0.71
kernel/rand	72526 ns	72256 ns	1.00
latency/import	1409718079 ns	1413343174 ns	1.00
latency/precompile	29718199650 ns	30124216661 ns	0.99
latency/ttfp	1785570104 ns	1782788848 ns	1.00
stencil/diffusion3d	1586943 ns	1581332.75 ns	1.00
stencil/diffusion3d_checked	1622006 ns	1637933.5 ns	0.99

This comment was automatically generated by workflow using github-action-benchmark.