Repository navigation
A device's FFT plans each hold a work area, four states' worth at 768 x 768 #529
Description
What happens
A free-space run's FFT plans are the largest row of its memory on a device. At 768 x 768 x 600, field-free, by memory_ledger() at 9a5d2346:
| State | Plans | Total | |
|---|---|---|---|
| NVIDIA H200 | 5.27 GiB | 21.09 GiB | 38.96 GiB |
| AMD Instinct MI325X | 5.27 GiB | 22.02 GiB | 39.64 GiB |
| NVIDIA A40 | 5.27 GiB | 21.09 GiB | 38.46 GiB |
That's more than the state, its register and its scratch together.
Why, on cuFFT
Each plan's own work area, asked of cuFFT directly with cufftSetAutoAllocation(plan, 0) and cufftMakePlanMany on the A40 (CUDA 12.8.1), Z2Z, in units of the state. It doesn't depend on T, across nine values from 512 to 1024:
| Plan | N in 320, 384, 400, 432, 450, 480, 512, 576, 625, 729, 750, 800, 900, 1024 | N in 500, 540, 600, 640, 648, 675, 720, 768, 810, 864, 960, 1000, 1080, 1152, 1200 |
|---|---|---|
Rank 3, {T, N, N}: VolumePlan, a low-storage Kerr step |
1.00 | 1.00 |
Rank 2, {N, N} batched over T: TransversePlan |
0 | 1.00 |
A forward and an inverse of each makes twice the state on the left and four times on the right, which is what a real run holds. On the A40, 8 steps, the median of a runner's second and third point:
| N x N x 600 | State, GiB | Plans, GiB | Total, GiB | Seconds per GiB of state |
|---|---|---|---|---|
| 512 | 2.34 | 4.69 | 13.20 | 3.86 |
| 576 | 2.97 | 5.93 | 16.32 | 4.00 |
| 640 | 3.66 | 14.65 | 27.14 | 4.35 |
| 720 | 4.64 | 18.55 | 33.98 | 4.32 |
| 750 | 5.03 | 10.06 | 26.68 | 4.04 |
| 768 | 5.27 | 21.09 | 38.46 | 4.07 |
| 800 | 5.72 | 11.45 | 30.17 | 4.04 |
So 800 x 800 holds 8.3 GiB less than 768 x 768 with 8.5 % more points. The 864 x 864 row ran out of the A40's 46 GiB, where 900 x 900 would likely have fitted. These are CUDA 12.8.1's choices: CUDA 13.1 on the H200 and rocFFT on the MI325X are being measured, and I'll add them here.
Two things that would help
Share one work area. These plans run one after another, so they shouldn't need one each. On the A40 a forward and an inverse rank-3 plan on one cufftSetWorkArea buffer round-trip a state to 3.3e-15 at 512, 768 and 800, against one state's work area rather than two. All four sharing would take 768 x 768 x 600 from 21.09 GiB of plans to 5.27 GiB, a total of 38.46 GiB to about 22.6 GiB, by arithmetic: I ran two plans, not four, and not inside solprop.
Kokkos-FFT has the route: DynPlan makes its plan with auto-allocation off and takes set_work_area, and compute_required_workspace_size(plans...) sizes one buffer for several, while Plan, which general.cpp uses, keeps its allocation private. rocFFT takes a work buffer in its execution info, which Kokkos-FFT's HIP plan already wraps.
Say which sizes are cheap. Nothing in docs/guide/performance/tuning.md tells a reader that 768 costs 9 GiB more than 800 on an NVIDIA card. That's cheap to document once the three toolkits are measured, though it can't be promised across CUDA versions.
Not checked
- Whether rocFFT promises that sequential plans may share a work buffer. Its guide doesn't address it, and says that
rocfft_executeallocates a buffer and frees it when none is set, so Kokkos-FFT's internal buffer is what stays resident. cuFFT's guide does promise it: "If multiple cuFFT plans are to be launched sequentially it is possible to assign the same memory chunk as work area to all those plans and reduce memory overhead." - That all four plans run on one stream.
TransversePlantakesfft_space_for, andVolumePlantakesexecution_space_. Two streams running at once couldn't share. - The time a shared area costs, if any. I wouldn't expect one, but it isn't measured.
- Why
{T, N, N}always takes a work area when{N, N, T}doesn't at the left column's sizes. If a volume transform could run in the other axis order, the left column would need none.
To do
- Measure the table on CUDA 13.1 (below): the same at every size.
- Measure it on rocFFT.
- Check the streams, and what rocFFT allows.
- Time
CUFFT_WORKAREA_MINIMAL, which that guide says replans without a work area, on a different algorithm. - Try one shared area in
general.cpp, listed inmemory_ledger()as its own row. - Document the cheap sizes, if they hold.
Done when
A device run's plans hold at most one state's work area between them, with fields unchanged to rounding and the step within its spread, or this closes with the reason it can't.
Activity
jtravs commentedon Nov 5, 2026
CUDA 13.1 on an NVIDIA H200 gives the same table as 12.8.1 at every size asked, so this isn't one toolkit's accident. Real runs there, at
8cd170e2, 8 steps, check off, the median of a runner's second and third point:So the sizes without a rank-2 work area are also the faster per GiB on this card, by about 4 to 13 % at neighbouring sizes, where on the NVIDIA A40 they were level. 900 x 900 holds less than 768 x 768 with 37 % more points, for 19 % more time a step; 750 x 750 is 11 % faster than 768 x 768 with 5 % fewer points, and holds 30 % less.
The MI325X's rocFFT table is still to come. Nothing here has changed solprop's behaviour yet.
jtravs commentedon Nov 5, 2026
CUFFT_WORKAREA_MINIMALdoesn't pay. It does take the work area to nothing, bycufftXtSetWorkAreaPolicyafter the plan is made, in the plans' real axis order, but each transform then takes about two to three times as long. On an NVIDIA A40 (CUDA 12.8.1), in-place Z2Z, the mean of the last 4 of 6:These also confirm the sizes: the rank-2 plans with no work area are the ones the policy leaves alone. So one shared work area is the route to try, since it changes no transform.
jtravs commentedon Nov 5, 2026
A probe of the sharing, on an NVIDIA A40 (CUDA 12.8.1) and not inside solprop: all four plans of the volume route at T = 600, a rank-3 forward and inverse and a rank-2 forward and inverse, on one work area of the largest plan's size.
The rank-3 pairs match the same plan with its own area, 433.2 ms and 460.2 ms. A dedicated single-transform engine runs on one shared area in production, with one rank-3 plan for both directions and a rank-1 batch, at 21.19 GiB on an NVIDIA H200 at 768 x 768 x 600.
Not checked:
KokkosFFT::Plankeeps its plan private, so this needs another way to set the area, and rocFFT's guide doesn't say whether sharing is allowed. See #365 for the transform order, which also removes the rank-3 work area at these sizes: 750 has none in{N,N,T}, and neither does 384, 800 or 900.jtravs commentedon Nov 5, 2026
The sharing inside solprop, on a local branch, not yet pushed: a device's plans of the scratch become
KokkosFFT::DynPlans on one work area, and a host keeps its plans. On an NVIDIA A40 (CUDA 12.8.1), 8 steps, check off, against the commit before:The time per point doesn't measurably change.
ab --device, 3 rounds, the same steps and final fields with no difference:kerr_beam_384x0.994 (0.984–1.048),kerr_beam_192x1.030 (1.001–1.035, a window of two steps).pytest -m gpu tests251 passed, 2 skipped;native/tests31 passed.This changes the advice above: with one area, 768 x 768 x 600 holds less than 900 x 900 x 600, so a size's memory now follows its state and the largest plan's work area. The size still sets the seconds per GiB.
Not run: HIP compiles but hasn't run, so whether rocFFT accepts the sharing is unknown. It goes in the follow-up PR after #520, and if an AMD card shows rocFFT won't share, it stays on CUDA alone.
jtravs commentedon Nov 6, 2026
rocFFT shares too. This is from an AMD Instinct MI300X, ROCm 10.0.0,
gfx942, at #542's head.Each plan's own work area, asked of rocFFT directly, in place, Z2Z, in units of the state:
{T, N, N}, frequency innermost{N, N}batched over TSo rocFFT on its own would hold four areas at every size, 22.0 GiB at 768 x 768 x 600, matching the MI325X's 22.02 GiB at
9a5d2346. With #542's one shared area, a run at 8 steps with the check off holds:The device tests passed on that card, including
test_the_plans_of_the_scratch_hold_one_work_area. #542 closes this when it merges.Corrected 2026-11-06: rocFFT lists lengths innermost first, so the rank-3 plan measured here, lengths
{T, N, N}, holds the frequency innermost: the(y, x, omega)order #365 adopted on CUDA, not HIP's current(omega, y, x). The run totals in the second table are solprop's own plans and stand. The rank-2 and rank-1 plans used rocFFT's default contiguous batches.Share one work area across a device's plans of the scratch