Skip to content

Integrate KleidiAI PF32 SME2 GEMM and GEMV kernels - #10992

Open
melkap01-Arm wants to merge 7 commits into
google:masterfrom
melkap01-Arm:KleidiAI_pf32_sme2_gemm_gemv
Open

Integrate KleidiAI PF32 SME2 GEMM and GEMV kernels#10992
melkap01-Arm wants to merge 7 commits into
google:masterfrom
melkap01-Arm:KleidiAI_pf32_sme2_gemm_gemv

Conversation

@melkap01-Arm

@melkap01-Arm melkap01-Arm commented Aug 17, 2026

Copy link
Copy Markdown

New KleidiAI pf32 Gemm-Gemv kernel families integrated.
Performances on the X300 hardware with Geekbech models:

Model New kernel mean (ms) Master mean (ms) Mean Δ New kernel median (ms) Master median (ms) Median Δ
Transformer 4.647 4.687 +0.86% 4.464 4.648 +3.96%
BERT tiny 0.709 0.733 +3.18% 0.719 0.745 +3.49%
EfficientNet Lite v3 51.120 51.909 +1.52% 50.699 51.640 +1.82%
DeepLab v3 55.193 54.904 -0.53% 54.434 54.213 -0.41%
Image Transform 244.044 245.133 +0.44% 244.579 244.757 +0.07%
MobileNet v1 5.559 5.524 -0.64% 5.485 5.435 -0.92%
MobileNet SSD 13.992 13.823 -1.22% 14.076 13.779 -2.16%
OpenPose 311.646 311.912 +0.09% 310.867 310.901 +0.01%
RetinaFace 39.425 39.837 +1.03% 39.240 39.308 +0.17%
RFDN 18.976 19.027 +0.27% 18.834 18.779 -0.29%
  • Mean latency: 0.52% faster (1.0052×)
  • Median latency: 0.60% faster (1.0060×)

Mac mini M4 machine :

Model Master mean New kernel mean Change Master median New kernel median Change
Transformer complex 1549.09 µs 1476.22 µs +4.94% 1531 µs 1436 µs +6.62%
BERT tiny 344.50 316.19 +8.95% 331 306 +8.17%
EfficientNet Lite V3 17927.20 16956.60 +5.72% 17947 16963 +5.80%
DeepLab V3 19656.00 18493.80 +6.28% 19639 18474 +6.31%
Image Transform Net 82758.90 82972.70 −0.26% 82771 82970 −0.24%
MobileNet V1 2124.55 1970.18 +7.84% 2011 1859 +8.18%
MobileNet V1 SSD 4348.15 3990.17 +8.97% 4300 3938 +9.19%
OpenPose V2 82230.80 82219.60 +0.01% 82216 82225 −0.01%
RetinaFace 17455.40 16141.90 +8.14% 17447 16125 +8.20%
RFDN 5533.40 5348.13 +3.46% 5473 5291 +3.44%

Overall geometric-mean speedup:

  • Mean latency: 5.36% faster
  • Median latency: 5.51% faster

Signed-off-by: melkap01 <melike.kaptan@arm.com>
Signed-off-by: melkap01 <melike.kaptan@arm.com>
@melkap01-Arm
melkap01-Arm marked this pull request as draft August 17, 2026 08:58
@melkap01-Arm
melkap01-Arm marked this pull request as ready for review August 17, 2026 12:41
copybara-service Bot pushed a commit that referenced this pull request Aug 24, 2026
--
70a9c5e by melkap01 <melike.kaptan@arm.com>:

Integrate KleidiAI PF32 SME2 GEMM and GEMV kernels

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
c473183 by melkap01 <melike.kaptan@arm.com>:

KleidiAI version v1.30 update

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
245a7e7 by melkap01 <melike.kaptan@arm.com>:

GEMMBenchmark updated for sme2 benchmarks, KleidiAI version related files updated

Signed-off-by: melkap01 <melike.kaptan@arm.com>
FUTURE_COPYBARA_INTEGRATE_REVIEW=#10992 from melkap01-Arm:KleidiAI_pf32_sme2_gemm_gemv 74c378a
PiperOrigin-RevId: 967093981
copybara-service Bot pushed a commit that referenced this pull request Aug 25, 2026
--
70a9c5e by melkap01 <melike.kaptan@arm.com>:

Integrate KleidiAI PF32 SME2 GEMM and GEMV kernels

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
c473183 by melkap01 <melike.kaptan@arm.com>:

KleidiAI version v1.30 update

Signed-off-by: melkap01 <melike.kaptan@arm.com>

--
245a7e7 by melkap01 <melike.kaptan@arm.com>:

GEMMBenchmark updated for sme2 benchmarks, KleidiAI version related files updated

Signed-off-by: melkap01 <melike.kaptan@arm.com>
FUTURE_COPYBARA_INTEGRATE_REVIEW=#10992 from melkap01-Arm:KleidiAI_pf32_sme2_gemm_gemv 74c378a
PiperOrigin-RevId: 967093981
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant