Repository navigation
Conversation
DjDeveloperr
left a comment
There was a problem hiding this comment.
Reviewed at f7604d7, and run on an M5 Pro (J714s, T6050, G17S, firmware 26A428). Test kernel: integration/aurora 69a9fbb + d76f650 + 86708ce + #1608..#1611. This series' files are byte-identical to the PR head.
Verdict: approve.
Other GPUs (checked by reading):
Telemetry::m5()is chosen only forvariant.m5_commands, and both new record arms areif self.m5. On the T8140 (G17P), the default telemetry ignores tags 0x19/0x1b,busy_statusstays unsupported, andreports_pstate()still returns false.- The fdinfo keys need the board table (
cfg.m5). Without it,frequency()returnsNonebefore it reads anything, so a non-M5 fdinfo read takes no device mutex. - The larger
Readout(80 bytes) is filled only byread_g17; the legacy GPUs format throughread_frequency. The C side's asserts match. - The J714 board table has the same 14 entries as J716c's (state 1 = 338 MHz, state 13 = 1620 MHz), so the decode applies unchanged.
On the M5 Pro (J714s):
agx_stats sampled every 100 ms, with fdinfo read during each phase. Baseline (integration/aurora 447b752 without the series, idle plus the same compute.heavy load): pstate, achieved_frequency_hz and busy_ns read unsupported throughout.
| phase (gpubench, ~6.4 s) | samples | pstate / frequency | util_device | busy_ns advance |
|---|---|---|---|---|
| idle | 32 | stale | 0 | 0 |
| compute.heavy alu, depth 8 | 64 | 13 / 1620 MHz in 63, 1 stale at the start | mean 98% | +6345 ms in 6.38 s (99%) |
| idle | 33 | stale | 0 | 0 |
| compute.light, depth 1 (~160 us per submit) | 64 | 13 / 1620 MHz in all 64 | utilisation stale |
0 |
| idle | 29 | stale | 0 | 0 |
| render.heavy fill 1080p, depth 8 | 64 | 13 / 1620 MHz in 63, 1 stale at the start | mean 98% | +6360 ms in 6.38 s (100%) |
| idle | 34 | stale | 0 | 0 |
- fdinfo: under load every client has
drm-curfreq-{vertex,fragment,compute}: 1620000000 Hzbesidedrm-maxfreq-*: 1620000000 Hz. With the GPU gated it hasdrm-maxfreq-*only. Thedrm-engine-*keys are still there beside them. - Other states: no state other than 13 appeared in these loads, so the board-table lookup for lower states was not exercised here.
- No regression on the same boot: multi-queue timeline 214/214, synchronization2 600/421 NS, gpubench 41/41, 0 recoveries, 0 Oops/WARN.
One observation, not a regression: during compute.light at depth 1, the firmware delivered no tag-0x13 records for the whole 6.3 s. utilisation_status was stale and utilisation_samples 0, while the client's drm-engine-compute rose by ~45 ms. So busy_ns did not advance during that sparse load, although pstate 13 was reported. This is the existing utilisation path, and busy_ns inherits it. Worth one sentence in the ABI doc: busy_ns counts only windows the firmware reports, and sparse sub-millisecond work may go uncounted.
Non-blocking:
- Hard-coded top state.
M5_MAX_PSTATE = 13restates the table length. Deriving it fromperf_states.len() - 1would keep the two from drifting. - Device mutex per fdinfo read. On the M5, each fdinfo read now takes the device mutex, which is the submission path's hot lock. That's fine for a top-like tool at 1-2 Hz. A monitor that reads every client's fdinfo in a tight loop would add contention, and the docs already say so.
Build shortcut, for the record: the M5 test kernel was built incrementally with include/linux/dma-mapping.h and include/drm/drm_gem_shmem_helper.h at their old mtimes, so files that only include them weren't recompiled. The hunks are a file-scope #define plus comment, and a same-type, same-position field rename; this head has no green CI compile to stand in for a clean build (below).
CI: kunit, the only full kernel build, is red on this head (https://github.com/aurora-silicon/linux/actions/runs/43125705476/job/130877105315). It failed compiling drivers/pmdomain/apple/pmp-report.c (get_cpu_idle_time undeclared), a file this series doesn't touch: integration/aurora's state when the check ran, fixed since by #1594 (8333109). So there is no green CI compile of this PR yet. A re-run reuses the original merge commit, so it needs a fresh run (a new push, or a re-trigger after the base update). host-tests, source and rust-build are green (rust-build compiles only apple-sep.o).
|
@erikwb kunit fails on pmp-report.c:1704 (get_cpu_idle_time), which comes from the base this branch sits on, not from your change. Please rebase onto current integration/aurora so CI is green. One small thing: frequency() takes the last row of perf_states as the maximum; taking the max over the table, or a comment that the table is sorted, would make that safe for other boards. |
…usy time agx_stats on the M5 reported utilisation only: pstate, the frequencies and busy_ns read "unsupported", so there was no way to tell whether the GPU runs at full clock during a workload. The M5 firmware reports its performance state in its statistics ring: tag 0x1b names the state now in effect (words 1 and 3, an index into the device tree's operating points, 1 = 338 MHz to 13 = 1620 MHz), and tag 0x19 the interval that just ended (start in words 3-4, end in words 1-2, state in word 8). The decoder now takes the state from 0x1b, time stamped by the end of the preceding 0x19, and the reader turns it into achieved_frequency_hz through the board's performance state table. A report whose two state words differ, or that names state 0 or a state beyond the table, is not used; a gate makes the state stale, as before. Its utilisation records (tag 0x13, every 8 ms) carry their own timer timestamps. busy_ns now integrates the device utilisation over the window each record closes. A record after a gap of more than two periods (the GPU was gated or records were lost) or with an earlier timestamp only sets a new baseline, as the legacy integrator does after a power change, so an idle gap is never credited. requested_frequency_hz, power_mw and temperature_raw stay unsupported. Documentation/ABI/testing/sysfs-driver-asahi-agx-stats and Documentation/gpu/asahi-usage-stats.rst describe the M5 values. Checked on the J716c (M5 Max MacBook Pro) with a single-thread loop of dependent integer operations, whose rate is proportional to the clock (54.0 M iterations/s at 1620 MHz): - one 1.4 s job that fills the GPU: every agx_stats read during it (12, 100 ms apart) gave pstate 13, achieved_frequency_hz 1620000000 and 100 % device utilisation. - 60 jobs of 20000 iterations, 30 ms apart: 48 ran in about 1792 us (335 MHz against the 338 MHz of state 1) and 12 in under 600 us. Of the agx_stats reads during the run (1 ms apart) that found a current state, 48 gave pstate 1 with achieved_frequency_hz 338000000 and 11 gave 13 with 1620000000; the others found the GPU gated. - three jobs of 300 ms at full clock, 300 ms apart, twice: busy_ns advanced by 871 and 863 ms for 905 and 919 ms of work, over 1.8 s (one 8 ms window is lost at each power-up). Tests: the telemetry unit tests (g17-telemetry-tests.rs on the host, 9 passed, and KUnit asahi_g17_telemetry) cover the M5 decode from firmware records, the busy windows, gates, gaps and other firmwares; g17-telemetry-integration-tests.py passes and still fails each of its mutations. Developed with Claude Code. Signed-off-by: Erik Bourget <erik@ebourget.net>
Documentation/gpu/drm-usage-stats.rst defines drm-maxfreq-<engine>, the highest frequency of an engine, and drivers that know it also print the frequency the engine runs at as drm-curfreq-<engine>. A Rust driver could only print the busy time. DriverFile::engine_frequency returns the current frequency, if the driver knows it, and the highest, in Hz; show_fdinfo prints them for every engine engine_stats emits. All engines get the same pair, which suits a device whose engines share one clock. The default, None, prints nothing, so drivers that do not implement it are unchanged. Built for arm64; the next commit uses it. Developed with Claude Code. Signed-off-by: Erik Bourget <erik@ebourget.net>
Tools that read a client's fdinfo (gputop, nvtop) see its engine busy time but not the clock the GPU runs at, which agx_stats now reports on the M5. Every engine of a client's fdinfo gets drm-curfreq-<engine>, the achieved frequency of the performance state the firmware reports, while it reports one (it does not while the GPU is gated), and drm-maxfreq-<engine>, the highest frequency in the board's table (1620 MHz on the J716c). One clock drives the vertex, fragment and compute engines. Each read drains the statistics rings under the device mutex, as an agx_stats read does. GPUs without a board performance table (all but the M5) print neither key. Documentation/gpu/asahi-usage-stats.rst said that drm-maxfreq-* was left out on purpose, because the engine times must not be scaled by a frequency. That still holds and drm-cycles-* stays absent: the document now says that the two keys show the device's clock at the time of the read, and that an fdinfo read on the M5 takes the device mutex for them. On the J716c (M5 Max MacBook Pro), the fdinfo of a client running a 1.4 s job that fills the GPU reads, for each of vertex, fragment and compute, drm-curfreq-<engine>: 1620000000 Hz and drm-maxfreq-<engine>: 1620000000 Hz; that of an idle client with the GPU gated has the drm-maxfreq lines only. Developed with Claude Code. Signed-off-by: Erik Bourget <erik@ebourget.net>
f7604d7 to
17eca40
Compare
|
Thanks. Rebased onto current integration/aurora (c868213), which has the idle-accounting change behind the kunit failure; head is now 17eca40.
Rechecked on the new base: each commit builds with the J716c config, KUnit 599 passed / 2 skipped, and a J716c boot with this series (plus #1610 and #1752) gives the same agx_stats and fdinfo results as before; the testing section has the numbers. Separately, |
RyanTheTide
left a comment
There was a problem hiding this comment.
Rebase and the max-over-table change look good, and CI is green. Before merge it needs a check that agx_stats and fdinfo are unchanged on a non-M5 G17 before merging. A separate patch for the perf_states[13] read in GET_PARAMS would be welcome.
DjDeveloperr
left a comment
There was a problem hiding this comment.
Current-head readiness review on @DjDeveloperr’s behalf.
The rebase/max-over-table change is present. frequency() derives the maximum from the board table and returns before read_firmware()/the device lock when no M5 table exists. The default DriverFile hook emits no frequency keys, and M5 decoding remains gated by the variant flag. I found no new source blocker in those changes.
Verified actual green Host43360666695/job131673136521 and SEP43360666812 build/test steps. The hosted QEMU job131673138336 reports294 tests; it is not the author’s separately reported599-pass/2-skip run with the G17 telemetry suite. Keep those scopes distinct. The existing M5 Pro and J716c hardware reports retain their exact composed-kernel limits.
This is still held by Ryan’s5886924588 pre-merge requirement: agx_stats and fdinfo unchanged on a non-M5 G17, with current source/composition identity. Green checks or the M5 rows do not supply that result. The separate GET_PARAMS perf_states[13] cleanup remains nonblocking follow-up. No merge, new hardware run or global GPU/reclaim action performed.
The problem
agx_statson the M5 reported utilisation only:pstate, the frequenciesand
busy_nsread "unsupported", and a client's fdinfo had its engines' busytime but not the clock. There was no way to tell whether the GPU ran at full
clock during a workload, or how long it had been busy.
The change
drm/asahi: g17: M5: report the performance state, its frequency and busy timedecodes two more kinds of record of the M5 firmware's statistics ring:
record 0x19 before it) give
pstate, and the board's performance statetable gives
achieved_frequency_hzfor it. A report whose two state wordsdiffer, or that names a state outside the table, is not used. While the
GPU is gated the state is stale, as before.
and
busy_nsintegrates the device utilisation over the window each recordcloses. A record after a gap of more than two periods only sets a new
baseline, so an idle gap is never credited.
requested_frequency_hz,power_mwandtemperature_rawstayunsupported. Other firmwares are unchanged.
Documentation/gpu/asahi-usage-stats.rst describe the M5 values.
rust: drm: file: print the engines' frequency in fdinfolets a Rust driverprint
drm-maxfreq-<engine>(Documentation/gpu/drm-usage-stats.rst) anddrm-curfreq-<engine>, the key drivers print for the frequency an engineruns at, beside
drm-engine-<engine>. Every engine gets the same pair,which suits a device whose engines share one clock. The default prints
nothing.
drm/asahi: g17: M5: report the GPU frequency in fdinfoprints them on theM5 for the vertex, fragment and compute engines, which one clock drives:
the current frequency while the firmware reports a state, and the highest
frequency in the board's table. An fdinfo read on the M5 now takes the device
mutex once to drain the statistics rings, as a read of
agx_statsdoes.Documentation/gpu/asahi-usage-stats.rst had
drm-maxfreq-*left out onpurpose, because engine times must not be scaled by a frequency. This series
keeps
drm-cycles-*absent and documents the two keys as the device's clockat the time of the read, not as a weight for the engine times.
Testing
Hardware: M5 Max MacBook Pro (J716c).
Build:
integration/aurorac868213 plus this branch, with the J716cconfiguration, at each commit. No warning in the files the branch touches.
KUnit, arm64 with 16K pages in QEMU:
./tools/testing/kunit/kunit.py run --arch=arm64 --kunitconfig=drivers/soc/apple/sep.kunitconfig:Ran 601 tests: passed: 599, skipped: 2, with three new tests inasahi_g17_telemetry(9 of 9).Host tests, as CI runs them:
python3 -I tools/asahi/run-host-tests.py:297/302 passed. The five that fail (apple-dcp,g15-pb-growth-testsand
g15-stats-drain-tests, each of the last two also with--mutations)fail identically on
integration/aurorac868213 on the build host,which lacks the Rust 1.93.1 toolchain the g15 tests ask for.
Among them
g17-telemetry-integration-testswith each of its elevenmutations,
g17-telemetry-tests(9 unit tests, three new) andlegacy-frequency-tests(2, one new).scripts/checkpatch.pl: 0 errors, 0 warnings on the three commits.On the J716c, with a Vulkan test program whose compute jobs are a
single-thread loop of dependent integer operations (54.0 M iterations/s at
1620 MHz), reading
agx_statswhile it runs:agx_statsread every 100 mspstate 13,achieved_frequency_hz 1620000000,util_device_percent 100agx_statsread every 1 mspstate 1,achieved_frequency_hz 338000000and 11 gavepstate 13,achieved_frequency_hz 1620000000. The other reads found the GPU gated (pstate_status stale)busy_nsadvanced by 871 ms and 863 ms for 905 ms and 919 ms of work, over 1.83 s and 1.84 s of wall timeOn the J716c, the fdinfo of the test program during a 1.4 s job that fills
the GPU has, for each of vertex, fragment and compute,
drm-curfreq-<engine>: 1620000000 Hzanddrm-maxfreq-<engine>: 1620000000 Hz. An idle client's fdinfo, with theGPU gated, has the
drm-maxfreqlines only.The J716c runs above were made with this series on the
integration/auroraof 2026-11-09 (e7f996f plus the then-open J716cpull requests), kernel 7.3.0, Mesa
aurora-wipwith the open J716c pullrequests.
On
integration/auroraea10c4d with this series (kernel7.3.0-ewb107+ on the J716c), the same program gave: 12 of 12 reads at
full clock with
pstate 13,achieved_frequency_hz 1620000000andutil_device_percent 100; 59 of the 60 short jobs at about 1792 us andone under 600 us, the reads that found a current state giving
pstate 1,achieved_frequency_hz 338000000(57) orpstate 13,achieved_frequency_hz 1620000000(2);busy_nsadvancing by 848 ms for918 ms of work over 1.85 s of wall time; and the fdinfo lines above.
On
integration/aurorac868213 with this series, drm/asahi: GEM_RESIDENCY, keep an object's memory resident #1610 and drm/asahi: g17: per-queue watchdog allowance for compute commands #1752(kernel 7.3.0-ewb122+ on the J716c): 12 of 12 reads at full clock with
pstate 13andachieved_frequency_hz 1620000000in each of three runs(
util_device_percent 100in the two runs after the desktop hadstarted, 0 in the first run after boot); all 60 short jobs at
about 1792 us, the reads that found a current state giving
pstate 1,achieved_frequency_hz 338000000(49) orpstate 13,achieved_frequency_hz 1620000000(1);busy_nsadvancing by 896 ms for932 ms of work over 1.85 s of wall time; the fdinfo lines above, and
drm-maxfreq-<engine>: 1620000000 Hzonly for the idle compositor. Thekernel log had no Oops, BUG, WARNING or GPU recovery line.
Depends on
Nothing.
Not covered
that runs the M5 command set, but only the J716c was measured.
lowest and the highest.
requested_frequency_hz,power_mwandtemperature_raware notimplemented.
machine has neither installed.
🤖 Generated with Claude Code