Repository navigation
Recycle command lists submitted to explicit queues - #713
Merged
Merged
Conversation
Moves the execute! methods that submit to a ZeCommandQueue next to the queue type, so that the queue can refer to command lists.
execute!(f, queue) created a command list per submission and left its destruction to the GC, which doesn't know about the driver memory each list holds (about 1.3 MB with NEO on Linux). Launching a kernel in a loop with `@oneapi` queue=... thus used gigabytes of host memory before a GC happened to run, and NEO keeps that memory for reuse rather than returning it. #610 removed this from the default submission path; the explicit-queue path still did it. Keep the submitted lists on the queue with a fence each, and reset and reuse them once the fence has signaled. A list per submission is still created when the caller passes its own fence or list flags. Fixes #327
maleadt
added
runtime
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #713 +/- ##
==========================================
- Coverage 84.08% 83.57% -0.51%
==========================================
Files 52 51 -1
Lines 4386 4299 -87
==========================================
- Hits 3688 3593 -95
- Misses 698 706 +8 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
oneL0.execute!(f, queue::ZeCommandQueue)created a command list for every submission and left its destruction to the finalizer. Each list holds about 1.3 MB of driver memory (NEO 26.35, Linux) that Julia's GC doesn't know about. NEO also keeps that memory around for reuse after the list is destroyed. So a loop of`@oneapi` queue=...launches used gigabytes of host memory before a GC happened to run, and didn't get it back afterwards. This is #327. #610 already fixed it for the default submission path; the explicit-queue path still had it.The queue now keeps the lists it submitted, each with a fence. Once a list's fence has signaled, the list is reset and reused. The fences are raw handles owned by the queue, because Level Zero requires them to be destroyed before their queue. Lists that fail to submit are kept but never reused. Passing a
fenceor list keyword arguments still creates a fresh list, since a submission can carry only one fence.Measured on an Iris Xe, launching a kernel 5000 times with
queue=ZeCommandQueue(...):GC.gc(true)The first commit only moves the queue-side
execute!methods intocmdqueue.jland includescmdlist.jlfirst, so the queue type can refer toZeCommandList.Fixes #327