Skip to content

drm/asahi: g17: walk only uncovered ranges when validating a submission - #658

Merged
RyanTheTide merged 1 commit into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:j616c/g17-validation-cache
Nov 6, 2026
Merged

RyanTheTide merged 1 commit into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:j616c/g17-validation-cache

Conversation

@AxxzyWasTaken

@AxxzyWasTaken AxxzyWasTaken commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

The per-queue Proof in g17/validation.rs skips the page-table walk only when it covers every range a submission names. Otherwise the whole request set is walked again. On top of that, every GPUVM map step moves the mapping epoch, which throws away all cached proofs.

Mesa records each frame's commands into command-buffer chunks at fresh addresses, so nearly every submission names at least one range the proof hasn't seen yet. Each of those submissions re-walks everything it names, render targets included: thousands of pages per full-resolution attachment, under the GPUVM exec lock.

Change

  • Walk only what isn't covered. Under an unchanged epoch the cached proofs still hold, so validate() walks only the ranges they don't cover (Proof::uncovered), then records the whole request set exactly as before. If the epoch moved between the cache check and the walk, it falls back to walking everything, as before.
  • Map steps keep the epoch. A GPUVM map step only maps addresses nothing maps yet: overlaps go through the unmap and remap steps first, and those remain full mutations. Every range a proof covers stays mapped across a map step, so sm_step_map now uses mapping_addition(). That keeps the epoch value and still pauses cached validation while the step's leaf writes are in flight.

Failure behaviour is unchanged: a range that isn't mapped still drops to per-range checked validation.

Testing

  • Two new KUnit cases in asahi_g17_validation_pages:
    • only_uncovered_ranges_are_walked_again: covered, partially covered, and stronger-access ranges.
    • a_partial_walk_extends_the_proof: after recording, the proof covers the union and stays epoch-bound.
  • All 9 cases (the 7 existing + 2 new) pass when built and run as a host Rust test against this file's Requests/Proof code.
  • drivers/gpu/drm/asahi builds on integration/aurora (23827fe87f4f) with tools/aurora-bindgen, with no new warnings.
  • An earlier version of this (same idea, on our T6041 tree) has been in daily use on a J616C (M4 Max) with Vulkan and GL games.

Benchmark

Result: the part this changes gets about 31 % faster and walks about 97 % less, but in this microbench that is too small to move total CPU time or wall time beyond run-to-run noise.

per submission / pass, 32 runs (16 each) before after change
Proof::validate() time per submission 11.64 us (sd 0.87) 8.00 us (sd 0.39) -31.2 %
Bytes walked per submission 30.08 MiB 0.76 MiB -97.5 %
Process system CPU per pass 52.3 us (sd 3.4) 50.6 us (sd 2.6) -3.2 %
Process total CPU per pass 68.5 us (sd 4.6) 67.6 us (sd 2.8) -1.3 %
Wall time per pass 100.0 us (sd 7.8) 99.9 us (sd 7.7) -0.1 %

Paired by round (mean of round differences, sd over 8 rounds): validation -3.64 us (sd 0.66), system CPU -1.69 us (sd 2.20), wall -0.08 us (sd 0.11). So the validation gain is clear, the system CPU change is within noise, and wall time doesn't change.

Walk kinds, 480,336 submissions each way: before 0 hits, 0 partial, 480,336 full; after 389 hits, 479,915 partial, 32 full.

Method

  • Machine: J616C (M4 Max, T6041), 14 cores, on mains, no desktop running, one-shot boot. integration/aurora has no T6041 GPU yet, so this isn't that tree: the kernel is our 7.2.10 J616C tree with our T6041 driver port, with this PR's g17/validation.rs dropped in.
  • One driver, live switch: the test build adds g17_val_partial_walk and counters around validate() (ktime_get_ns()). The port already has g17_val_additive, which turns mapping_addition() back into a full mutation. Both off is the base logic ("before": it walks every range when any is uncovered, and every map step moves the epoch), both on is this PR ("after"). The bench hooks aren't in this PR.
  • Workload: a surfaceless GLES microbench, render-to-texture ping-pong between two 3456x2234 RGBA8 textures, 30,000 passes a run, one submission per pass, command buffers recorded fresh each pass.
  • Order: 8 rounds of base, PR, PR, base on the same boot, 32 runs. CPU is ru_utime/ru_stime from wait4(). The clock wasn't pinned.
round validate before (us) after system CPU before (us/pass) after wall before (us/pass) after
0 10.54 7.61 48.4 49.9 91.1 91.0
1 10.57 7.76 47.5 48.6 90.9 91.0
2 10.75 7.72 50.1 49.1 93.0 92.7
3 11.78 8.16 52.2 49.0 97.6 97.5
4 12.36 7.92 55.0 50.5 102.3 102.2
5 12.30 8.07 55.2 51.5 105.5 105.6
6 12.35 8.04 53.7 50.8 108.4 108.3
7 12.43 8.72 56.5 55.5 111.1 111.0

Limits

  • This names only two attachments at about 30 MiB of walked range. The saving scales with the size of what's re-walked, so more or larger render targets should gain more, but that's not measured.
  • Nothing here was run in a game, and it wasn't run on the J616s or the Neo.
  • vkmark doesn't exercise this: it reuses recorded command buffers, so 400,779 of 400,785 submissions were already full cache hits on this logic. Workloads like that won't gain.
  • For how often real use takes this path: counters on our daily driver, which has an earlier version of this change, read 2,027,636 partial, 108,034 hits and 63,942 full after about a day of desktop and app use.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Reviewed for the M4 merge train. Two things:

  • It conflicts in mmu.rs. integration/aurora now bumps a bind_sequence counter in the same map step, so the change to mapping_addition() has to keep that increment. Please rebase onto 19f71a1.
  • This is a performance change to submission validation, and every G17 machine runs it (A18 Pro and M4 Pro as well as T6041). Performance PRs need a paired benchmark in the description: the method, raw before/after numbers and run-to-run variance, for example submit-path CPU time per frame in a game or a microbench. The KUnit cases cover correctness but not the gain.

After that we'll run it on the J616s and the Neo.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Paired benchmark is in the description, with the method, per-round numbers and variance. Short version: validate() is about 31 % faster and walks about 97 % less, but in this microbench total CPU and wall time don't move beyond noise. It ran on a J616C with our T6041 driver port and this PR's validation.rs, since integration/aurora has no T6041 GPU yet, so a J616s or Neo run would be the first on the real tree.

The mmu.rs conflict is resolved and sm_step_map() keeps the bind_sequence bump. That went in as a merge of integration/aurora rather than a rebase onto 19f71a1.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Merge train (MTL, 06/11): can't take this yet. Commit 5f0c7fc ("drm/asahi: g17: walk only uncovered ranges when validating a submission") has no Signed-off-by: from its author, and a sign-off is a hard requirement here (we can't add one on someone else's behalf). Could you amend with git commit --amend -s (or git rebase --signoff) and force-push the branch? I'll gate it again once it's signed.

The per-queue Proof skips the page-table walk only when it covers every
range a submission names. Otherwise the whole request set is walked
again, and every GPUVM map step moves the mapping epoch, which discards
all cached proofs.

Mesa records each frame's commands into command-buffer chunks at fresh
addresses, so nearly every submission names at least one range the
proof has not seen. Each of those re-walks every range in the
submission, including the render targets: thousands of pages per
full-resolution attachment, under the GPUVM exec lock.

Under an unchanged epoch the cached proofs still hold, so walk only the
ranges they do not cover and record the whole request set as before. If
the epoch moved between the cache check and the walk, walk everything,
as before.

A GPUVM map step only maps addresses nothing maps yet: GPUVM runs the
unmap and remap steps for any overlap first, and those remain full
mutations. Every range a proof covers therefore stays mapped across a
map step, so it no longer advances the epoch. Cached validation still
pauses while the step's leaf writes are in flight.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken
AxxzyWasTaken force-pushed the j616c/g17-validation-cache branch from a9e3886 to 8d88d95 Compare November 5, 2026 17:44
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Signed off and force-pushed: one commit now, on 565c802, so the merge commit is gone and the tree is unchanged.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Merge train (MTR, 06/11): merging, now signed off. I gave this an adversarial review because it decides what a submission may touch:

  • A map step only maps addresses nothing maps yet. GPUVM runs unmap and remap steps for any overlap first, and those still bump the epoch, so every range a cached proof covers stays mapped across a map step.
  • The partial walk is only taken when the cached epoch equals the current one. If the epoch moved, everything is walked again.
  • covers_ranges_batch() reads the epoch under the same VM reservation that every mapping change holds.
  • Stronger access than what was proven is walked again (covered by the new test).

The Neo (G17P) ran vkcube and glmark2 on it with no GPU faults or recoveries.

Gates: test merge 2d3579ef = integration/aurora 11eccbe + #1005 #994 #930 #658 #992 #1006 #1007 #1008 #531 (#992 has since landed on its own); base 11eccbe.

  • KUnit: drm/apple 112/113 (1 skip); Rust (asahi, incl. asahi_g17_validation_pages) 200/201 (1 skip); arm64 apple-isp-*, apple-pmc, apple-pmgr-pwrstate and thunderbolt 137/138 (1 skip).
  • W=1 on drm/apple, media/apple/isp, pmdomain/apple, thunderbolt and nvmem/apple-spmi-nvmem: 0 new warnings.
  • dt_binding_check (apple,dcp, apple,dart, apple,t8132-isp, apple,spmi-nvmem, apple,pmc): 0 new.
  • Host tests run-host-tests.py 185/185; apple-dcp selftests 86/86.

Hardware (7.3.0-aurora-mtr1106a, one-shot boots, fleet recipe):

  • J616s (M4 Pro) vs fleet17: 0 kernel errors, 0 AVC, 0 failed units; eDP + DP-2, Wi-Fi, Vulkan G16S. Camera NV12 1080p and 720p, and P010 720p, streamed 30 frames each, with no ISP errors. vkcube OK; glmark2 6617/6738/6683 vs fleet17 6646/6555/6664.
  • Neo (A18 Pro), two boots vs fleet16: 0 kernel errors, 0 AVC, os_uuid pin present; eDP, Wi-Fi, Vulkan G17P. Camera NV12 1080p and 720p, no ISP errors. vkcube OK; glmark2 2718/2729/2726/2725 vs fleet16 2043–2108; 0 GPU faults.
  • Both Macs were restored to their default kernels afterwards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants