Repository navigation
Mark kernel entry points norecurse - #1066
Merged
Merged
Conversation
The AMDGPU back-end assumes a call to a function without `norecurse` may recurse, and gives the calling kernel a dynamic stack and enables scratch memory, even when the call is never made. Our pipeline never infers `norecurse`, so every kernel with an out-of-line call paid for this, up to 2x on an MI210. Device code cannot call a kernel, so mark kernel entry points `norecurse` like clang does for CUDA and HIP kernels, and run the top-down `ReversePostOrderFunctionAttrsPass` after the runtime has been linked to propagate it to the internal functions they call. The native target is excluded, as a kernel there is an ordinary function that may be called back from outside the module. Fixes #1065, closes #262.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1066 +/- ##
=======================================
Coverage 92.42% 92.42%
=======================================
Files 31 31
Lines 7323 7326 +3
=======================================
+ Hits 6768 6771 +3
Misses 555 555 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The AMDGPU back-end assumes that a call to a function without
norecursemay recurse. It then gives the calling kernel a dynamic stack (.amdhsa_uses_dynamic_stack 1) and enables scratch memory, even when the call is never made. Julia marks@noinlinefunctions onlynoinline, and our pipeline infers nonorecurse, so every kernel with a non-inlined call pays for this. #1065 measured a 2x slowdown on an MI210 for a call on a branch that is never taken. The check is the same in every LLVM from 15 to main, and the AMDGPU attributor doesn't infernorecurseeither.Device code cannot call a kernel, so this PR marks kernel entry points
norecurse, like clang does for CUDA and HIP__global__functions. It then runsReversePostOrderFunctionAttrsPassafter the runtime is linked and the final inlining, to propagate the attribute to the internal functions the kernel calls. That is the top-down pass LLVM's default pipeline runs at the start of its optimization pipeline. The bottom-upfunction-attrspass would only cover leaf functions, and doesn't infernorecurseon LLVM 15 when a callee calls intrinsics.The native target is excluded: a kernel there is an ordinary function, which code outside the module could call back through a function pointer.
Effect on the other back-ends:
norecursecan significantly improve the performance on an AMD MI210 #1065 ran at the same speed with or without it, and a non-inlined call doesn't cost anything to begin with.Fixes #1065, closes #262.