Repository navigation
drm/asahi: g17: M5: drop repeat uncached reads from submission, render publication and the event pass - #1504
Merged
Merged
Conversation
Every M5 compute publication ran the queue-install check, which read the recovery state and the item-ring cursors from shared GPU memory (four uncached loads under the device mutex) before deciding that a queue with a published kick needs nothing. Return first when the queue is installed, its configuration is released and a kick was published; the outcome is the same, without the loads. Signed-off-by: Diljit Singh <hey@djdev.me>
The item-ring snapshot that #1401 takes for an M5 compute publication served its room, record-slot and reclaim checks, but both appends (the configuration and its notification) still read the ring cursors again: three uncached loads each under the device mutex. The same argument covers them: the producer is the host's and only this publication moves it, and the firmware's consumer only advances, so the snapshot's consumer is conservative. The room check already required space for both items. Append against the snapshot, with the producer the publication has reached. Signed-off-by: Diljit Singh <hey@djdev.me>
An M5 compute publication read the shared compute ring's three cursors twice under the device mutex: for the room check before any state changes, and again when it wrote its record. Between the two, only this publication can move the producer, the firmware's consumers only advance and off-lock cancellation only lowers the reserved credits, so the first sample stays conservative. Keep it in the publication's host and write the record against it; any other record reads the ring as before. Signed-off-by: Diljit Singh <hey@djdev.me>
replay_compute runs on every event pass and before every inline compute publication, and it read the recovery state from shared GPU memory (an uncached load under the device mutex) before finding, almost always, that no queue retains a command for replay. Count the retained commands first, from host state, and return when there are none. The replay wait markers then name no command, so they are cleared as before; the recovery state still gates every replay. Signed-off-by: Diljit Singh <hey@djdev.me>
Every M5 doorbell (compute and render) first counts the submission in the firmware's history, and read its geometry each time: the generation, the four period and slot-count words, and the generation again, all uncached loads under the device mutex. The firmware changes the generation around every change of the geometry, so a geometry read under an even generation stands for as long as the generation reads the same. Keep it with its generation and read the words again only when the generation differs, or after a recovery transition. Signed-off-by: Diljit Singh <hey@djdev.me>
…ains The M5 event pass drains each role's statistics ring between two power observations, and each observation reads the GPU's core power register. With two roles the observation after the first drain and the one before the second ran back to back, under the device mutex. Observe once before the first drain and once after each; every drain is still bracketed. Signed-off-by: Diljit Singh <hey@djdev.me>
KernelObject::read_words copied records with one 32-bit volatile load per word. Firmware records live in write-combined (uncached) views, so an 18-word event or report record took 18 uncached loads, under the device mutex in the event pass. Where the copy is 8-byte aligned, load two words at a time; the words and their order in memory are unchanged. Signed-off-by: Diljit Singh <hey@djdev.me>
…ication A render publication checks that both outer destinations, tiling and fragment, have room before any state changes. On M5 both are the shared compute ring of the priority, and both checks asked for the same two free slots, so the second one read the ring's three cursors from shared memory again under the device mutex for the same answer. Check once on M5; other GPUs keep both checks. Signed-off-by: Diljit Singh <hey@djdev.me>
…state replay_render runs on every event pass and read the recovery state from shared GPU memory (an uncached load under the device mutex) before looking for a parked render group, of which there almost never is one. Look first, from host state; the recovery state still gates every replay. Signed-off-by: Diljit Singh <hey@djdev.me>
Every render publication (through its pool preparation) and the event worker observed the render pools' free-list releases by reading the device-control ring's cursors from shared GPU memory, two uncached loads under the device mutex. A lease uses them only while a release it sent has not been seen consumed. Return before the read when no lease waits for one; the leases end up in the same state. Signed-off-by: Diljit Singh <hey@djdev.me>
An M5 render publication read the shared compute ring's three cursors twice under the device mutex: for the room check before pool accounting, and again when it wrote the pair. As for a compute publication, only this publication can move the producer between the two and the firmware's consumers only advance, so the first sample stays conservative. Keep it in the publication's host for the first pair written to the ring; any other pair reads the ring as before. Signed-off-by: Diljit Singh <hey@djdev.me>
Each render observation snapshot loaded the pair's private job-list head and tail and discarded both values. Nothing depends on them (relaxed loads whose values are unused order nothing), and the view is uncached, so each was a DRAM round trip in the event worker's device-mutex hold. Signed-off-by: Diljit Singh <hey@djdev.me>
A render publication read both item rings' done, read and write cursors, six uncached loads under the device mutex, to check that the producers are where the pair left them. Behind a command in flight the done and read cursors are unused: the next command's previous cursors come from that command's windows. Read the two producers alone then; an idle pair reads all six as before. Signed-off-by: Diljit Singh <hey@djdev.me>
…s consumers An M5 render publication writes its pair deferred and then releases it, and the release read the shared ring's three cursors again under the same hold of the device mutex. Its consumers from the room check stay conservative for the whole hold: the firmware only advances them, and with the publication's own credits lent to it the other reservations can only shrink. Use them, with the producer read again: the release still revalidates the producer the pair was prepared against. Signed-off-by: Diljit Singh <hey@djdev.me>
DjDeveloperr
changed the title
This was referenced Nov 11, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The firmware's shared objects are mapped write-combined, so each read of a ring cursor, the recovery word or a firmware record is an uncached DRAM round trip, made with the device mutex held. An M5 compute publication made about 21 of them and much of it was repetition; the event worker, which takes the same mutex, made more on every pass. This series removes the repeats:
None of them changes what is published or in which order: the reused values are either the host's own (producers, cached geometry under an unchanged generation) or conservative (firmware consumers that only advance), and every recovery check that gates a publication remains.
Measured on an M5 Pro (J714s, firmware 26A428), on the M5's 7.3.0 kernel (integration/aurora plus the M5 stack as it was before #1012 merged), gpubench pinned to a P core. The changes were switched interleaved on one boot, then run as a clean module across fresh boots:
dEQP-VK (multi-queue, synchronization, synchronization2, s0, compute, glsl, draw, texture, occlusion query samples) with the M5 bring-up Mesa (with its occlusion-query and compute-visibility fixes) and with the device's older Mesa: 0 Fail, the same counts as before; gpubench core-a,feat 41/41; no GPU recoveries.
Built and run as this branch's head on integration/aurora (dc8f6f7 + the ten commits, the M5's .config, in-tree J714s DT with the unit's GPU calibration words): the whole kernel builds with no new warnings; on the M5 Pro, dEQP-VK mq-tl 214, mq-v1 427, mq-all 290 (106 NotSupported) and synchronization2 600 (421 NotSupported) pass with both Mesas, 0 Fail; gpubench core-a,feat 41/41; no GPU recoveries or timeouts.
Render publication and observation (four more commits)
A hold probe on the M5 showed render.light holding the device mutex for about 25 µs of each 27.8 µs submit. The render publication holds it for 10.8 µs and waits 4.1 µs to get it, so render throughput is bound by total hold time. Four more repeat reads go:
They were switched interleaved on one boot against the ten commits above, as paired per-round medians: render.light d16 -2.7%, d64 -3.2% for the first three, then -0.4% each for the fourth. compute.light is unchanged. Fresh boots, against macOS on the same firmware:
The same dEQP-VK lists with both Mesas: 0 Fail, the same counts; gpubench core-a,feat 41/41; no GPU recoveries. On integration/aurora, as this branch's head (the kernel above with this driver): mq-tl, mq-v1, mq-all and synchronization2 pass with both Mesas, gpubench 41/41, no recoveries.
🤖 Generated with Claude Code