Skip to content

soc: apple: clpc, drm/asahi: g17: busy-share memory floor and light effort for T604x - #1089

Closed
AxxzyWasTaken wants to merge 2 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:max-gpu-power
Closed

AxxzyWasTaken wants to merge 2 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:max-gpu-power

Conversation

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor

Lowers power on the M4 Max in everyday use. On T6040/T6041, any reported GPU work holds the top memory and fabric floor, and the GPU runs at full effort for quiet work.

  • CLPC: the GPU driver reports its busy share, and the floor follows a table (DCS 1, 2, 4, 5 and fabric 0, 0, 0, 2 at 160, 500 and 800 permille). Going down waits 400 ms. CPU map quarters now use 4.512 GHz for both clusters, and the T6040 map is lower.
  • G17: a light effort governor, half effort after four quiet windows, with separate entry and exit points. note_submission() no longer skips it.
M4 Max, AC, 120 Hz Old New
Scrolling 24.9 W 22.1 W
Page workload 23.84 W 22.73 W, score -0.9 %

Effort sweep on a 1080p30 clip: full 14.0 W, half 10.4 W.

Tested: this logic has been the daily setup on my M4 Max since 5 Nov (7.3.0), with no GPU recoveries from it. Compared with that tree, I removed the trial debug parameters and moved the defaults to the values the Mac runs. This exact version is compile-tested on integration/aurora and not booted.

Please check:

  • The CLPC table is shared with T6040, and I haven't tested an M4 Pro.
  • The light effort governor is in the shared G17 code, so it applies to other G17 SoCs too. aurora.g17_light_effort=0 turns it off. Tell me if you'd rather have it scoped to T604x.

On T6040 and T6041 any reported GPU work holds the top memory and fabric
floor. At 120 Hz a scrolling browser keeps the GPU reported busy most of
the time for a few percent of real execution, so light desktop use pays
for the top DCS state.

Let the GPU driver report its busy share per window with
apple_clpc_gpu_set_busy() and pick the floor from a table: DCS 1, 2, 4
or 5 and fabric 0, 0, 0 or 2, split at gpu_busy_mid (160), gpu_busy_high
(500) and gpu_busy_top (800) permille, all runtime parameters. A lower
level must last 400 ms, a higher one applies at once. A GPU that reports
no share, and every SoC without the table, keeps the top of the map.

The CPU side had the same problem: a cluster at its own top speed asked
for DCS 5 and fabric 2 even when that was the E cluster. Take the map
quarters from 4.512 GHz for both clusters, and lower the T6040 map to
DCS 1, 1, 3, 4 with fabric 0. On the M4 Max on AC, compared with the old
table: scrolling 24.9 W to 22.1 W at 120 fps with no long frames, a page
workload 23.84 W to 22.73 W for 0.9 % less score.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Sum the engine time every client is charged and report its share of a
200 ms window to CLPC, which then keeps the top memory and fabric floor
for GPU work that really is busy.

The same windows drive a light effort governor. Four quiet windows in a
row (under 35 % busy, no job of 3 ms or more) drop the firmware to half
effort, and 75 % busy or a 7 ms job goes back to full. Light effort runs
the same work about twice as long, so the way out sits at about twice
the way in, or work that fits light effort sticks at full. The share
sent to CLPC is doubled at full effort so one floor table fits both.

note_submission() returned effort_full, so the lockless path skipped the
governor. It now takes the slow path when the governor wants light.

g17_light_effort (65536 is full, 0 turns this off), g17_light_enter,
g17_light_exit, g17_light_job_enter_us and g17_light_job_exit_us are
runtime parameters. Effort sweep on the M4 Max with a 1080p30 clip: full
14.0 W, 32768 10.4 W, and below 50 % gained nothing.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merge-train review (overnight 11-07). Thanks, the busy-share floor reads well and the hysteresis is simple. Before this can merge it needs a J616s (T6040) run, because it changes the M4 Pro too, and not only through the GPU path:

  1. The T6040 CPU map drops. map goes from DCS {1, 2, 5, 5} / SoC {0, 1, 2, 2} to {1, 1, 3, 4} / {0, 0, 0, 0}, and level_khz = 4.512 GHz rescales the quartiles for both clusters. That lowers the memory and fabric floor for CPU-only load on every M4 Pro, which is a separate change from the GPU busy share. We'll benchmark it on our J616s against fleet22: a memory-bound CPU workload (STREAM-style copy/triad, a kernel build), the GPU set (gpubench, glmark2, a dEQP slice) and idle/scroll package power. Perf changes need those numbers before merging, and none may regress.
  2. Module parameters. gpu_busy_top, gpu_busy_high and gpu_busy_mid are runtime tunables (0644). We keep policy in the per-SoC tables, not in knobs. Could they be fields of apple_clpc_hw (next to gpu_busy) with these values for T6040/T6041?
  3. Down-step timing. apple_clpc_gpu_level() only moves down when it is sampled at least 400 ms after the last change, so the effective delay depends on when the CLPC sampler runs. That's fine as a floor, but the commit message should say "at least 400 ms".
  4. apple_clpc_status_show() calls apple_clpc_gpu_level(), so reading debugfs can move the level. It should read gpu->level without updating it.

The J616s is busy until about 05:00 tonight. We'll post the numbers here, and if 2–4 are addressed and nothing regresses, it goes through the train.

@RyanTheTide

Copy link
Copy Markdown
Contributor

Merge train 11-08 follow-up: #1137 (an ANE input for the CLPC on the T8140) just merged into integration/aurora and also edits drivers/soc/apple/clpc.c, so this now conflicts there. When you address the review points (busy-share thresholds as apple_clpc_hw fields instead of module parameters, a side-effect-free debugfs read), please merge integration/aurora into the branch. We'll run the J616s A/B after about 05:00.

@RyanTheTide RyanTheTide left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

host-tests fails on this head in clpc-gpu-demand-tests. The harness extracts apple_clpc_gpu_input_demand(), and that function now calls apple_clpc_gpu_level() and max_t(), which the extracted unit doesn't provide:

fixed.c:233: error: implicit declaration of function 'apple_clpc_gpu_level'
fixed.c:235: error: implicit declaration of function 'max_t'

To fix it, add static unsigned int apple_clpc_gpu_level( to the extracted items in tools/asahi/clpc-gpu-demand-tests.py, along with a max_t macro and the new struct apple_clpc_gpu fields (busy, busy_valid, level, level_since) in its stubs. A few cases for the busy-share table would help too: below gpu_busy_mid, between the thresholds, no share reported, and the 400 ms down-step hold.

The branch also still conflicts with integration/aurora in drivers/soc/apple/clpc.c, so it needs a merge or rebase as well.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks for the review. All points are addressed in #1369 (head 1257b68, same title). The commit message change needed new history, so it's a new branch rebased onto current integration/aurora, and this branch stays as it was.

  1. T6040 CPU map / J616s run: no change. The map and level_khz are the same as here, so your A/B applies to soc: apple: clpc, drm/asahi: g17: busy-share memory floor and light effort for T6041 #1369 as is.
  2. gpu_busy_mid, gpu_busy_high and gpu_busy_top are now fields of struct apple_clpc_hw next to gpu_busy, set to 160, 500 and 800 for T6040/T6041. The module parameters are gone, and apple_clpc_gpu_level() takes the hw table (f4dca10).
  3. The commit message now says the down-step happens at least 400 ms after the last change, at the first CLPC sample past that, so later if the sampler runs late (f4dca10).
  4. apple_clpc_status_show() no longer calls apple_clpc_gpu_level(). It reads busy, busy_valid and level under gpu->lock, so the line shows the level the last sample picked.
  5. clpc-gpu-demand-tests.py now extracts static unsigned int apple_clpc_gpu_level(, has a max_t macro and the busy, busy_valid, level and level_since stub fields, and reads APPLE_CLPC_GPU_DOWN_MS from the driver. New T6040 cases: no share reported (0x25), below gpu_busy_mid (0x01), each threshold edge going up (0x02 / 0x04 / 0x25), the 400 ms hold (still 0x25 at 399 ms, down at 400 ms with fabric kept for HOLD_MS), and a mid-hold sample restarting the hold. New negative controls gpu-down-at-once, gpu-busy-ignored and gpu-no-share-uses-table are each caught. I reproduced the implicit declaration errors on the old head first.
  6. Rebased over Neo ANE: hold DRAM and fabric floors while the engine runs, keep gating off back-to-back (MobileNet 2.3-2.5 -> 1.25-1.31 ms) #1137: struct apple_clpc_hw and the T6040 table keep both the ANE fields and the busy-share fields. I also updated the T6040 comments that said a busy CPU cluster or GPU work gets DCS 5 / fabric 2, since that's no longer true with the new tables. soc: apple: clpc, drm/asahi: g17: busy-share memory floor and light effort for T6041 #1369 shows as mergeable.

Tested on #1369's head: run-host-tests.py --only for clpc, usage, g15-queue-credit, asahi-m4 and g17 all pass (35 runs). clpc.o and g17/clpc.o build with LLVM=1 W=1 with no new warnings (the pmp_agent kernel-doc warning is already in the base), and checkpatch on the CLPC commit is clean. Not tested: the Rust part of drm/asahi wasn't compiled on the new base (my VM's toolchain fails in rust/bindings first), KUnit wasn't run, and nothing was booted.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Replaced by #1369, rebased onto current integration/aurora with the review points addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants