Repository navigation
Require PoCL's NUMA-aware scheduling, and measure the NUMA example - #928
Merged
Merged
Conversation
pocl_standalone_jll 7.2.3+1 gives each PoCL thread a contiguous block of a kernel's workgroups, so that a kernel touching memory first and a later kernel with as many workgroups work on each part of an array from the same thread, and makes POCL_AFFINITY=1 spread the threads over the NUMA domains. Compat can't name a build, so require 7.2.3, which resolves to the latest one. The NUMA example's page said that it hadn't been measured on a system with several memory domains. Explain the scheduling and the pinning, and add measurements on 2x EPYC 7502: 242 GB/s with 64 pinned threads and parallel initialization, against 142 with serial initialization and 130-155 without pinning; a 32-thread team gets 157 against 124.
Member
Author
|
Note that we'll have to re-do this when switching to ThreadTeams.jl, so I plan on getting this out quickly such that NUMA users have something to work with. |
Contributor
|
Documentation preview: https://juliagpu.github.io/KernelAbstractions.jl/previews/PR928/ |
Contributor
Benchmark ResultsShow table
Benchmark PlotsA plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## tb/pocl-parallel-zeros #928 +/- ##
=======================================================
Coverage 83.13% 83.13%
=======================================================
Files 31 31
Lines 2568 2568
=======================================================
Hits 2135 2135
Misses 433 433 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
KernelAbstractions 0.10 lost much of the NUMA performance of 0.9's
CPU(; static=true): SAXPY on 2x EPYC 7763 dropped from ~330 to ~250 GB/s. There were two causes:muladd#922 (0.10.2) fixed this.pocl_standalone_jll7.2.3+1 (JuliaPackaging/Yggdrasil#15319, from JuliaGPU/pocl#7 and #8) fixes the PoCL side:POCL_AFFINITY=1pins the threads to the CPUs the process may use, one per core first, spread over the NUMA domains.This PR:
Raises the compat bound to 7.2.3. Compat can't name a build, but 7.2.3 resolves to +1, and it excludes the 7.2.2 builds without these changes.
Rewrites the NUMA example's remarks to explain how scheduling, pinning and initialization (including
zeros/onesfrom POCL: initialize large arrays with a kernel inzerosandones#927) interact.Replaces the "not measured on NUMA yet" note with measurements on 2x EPYC 7502 (2 domains), using the registered JLL and POCL: initialize large arrays with a kernel in
zerosandones#927. SAXPY bandwidth in GB/s:POCL_AFFINITY=1zeros/oneszeros/onesPinned
Threads.@threads :staticgets 240 GB/s with 64 threads. On a single domain (Ryzen 9950X), all configurations get 53–55 GB/s.