Skip to content

asahi: probe HIGH admission and report effective EGL priority - #31

Closed
satchlj wants to merge 22 commits into
omacom:integration/aurorafrom
aurora-silicon:gpu26/queue-priority-20261103
Closed

satchlj wants to merge 22 commits into
omacom:integration/aurorafrom
aurora-silicon:gpu26/queue-priority-20261103

Conversation

@satchlj

@satchlj satchlj commented Nov 3, 2026 •

Copy link
Copy Markdown

Probe actual queue admission before advertising HIGH; always retain LOW/MEDIUM and stop advertising REALTIME. Report the effective Gallium priority through DRI/EGL after HIGH-to-MEDIUM fallback, including wrappers. The probe creates/destroys a private queue without submitting work and fails closed. Vulkan global-priority semantics are unchanged.

Targets integration/aurora, matching #30. Carries the tested public #22/#29/#30 composition; those dependencies are included in the displayed diff until they land.

Validation: affected libgallium and libEGL_mesa built in the Arch sysroot. Six DRM-shim tests passed: Gallium admission allowed/denied/allowed-once and full GLVND→EGL→DRI→Gallium effective-priority queries for the same cases, including revocation after probing, queue cleanup and shared-cache bypass. Shim checks are offline CPU evidence, not hardware qualification. Metadata rewriting preserved the tested tree.

The M3 desktop can lag under GPU load: the compositor requests HIGH, but the extended T8122 kernel profile rejects it and Mesa silently falls back to MEDIUM. Default clients also use MEDIUM, leaving the desktop without the requested preference.

C13k baseline: three trials in a private headless, hardware-rendered sway session, undocked on battery with power-saver verified before and after (not continuously traced). Unloaded commit-to-present p50/p95/p99/max: 16.181/16.581/17.163/20.673 ms; under explicitly LOW GPU load: 16.205/17.499/22.402/38.332 ms. Loaded maximum presentation gap: 38.386 ms; estimated missed slots: 0.760%, versus 0% unloaded. LOW made progress every second; the bounded journal check found no matching GPU faults. This is a headless pacing proxy, not physical scanout or owner Hyprland latency, and HIGH was denied in this baseline.

REALTIME GPU priority stays rejected. Firmware queue encoding, channels and doorbells are unchanged; this series changes host ordering only and does not establish firmware preemption. Matched power-mode comparisons and physical Hyprland acceptance remain open. See the C13l-r2 hardware results below.

Series: kernel omacom/linux-aurora#346; Mesa #31; Hyprland capability omacom/omarchy-mac-pkgs#88. Kernel supplies admission; Mesa exposes actual permission and effective priority; the capability package authorizes the production compositor.

PR CI was triggered; Mesa CI requires maintainer approval (action_required).

C13l-r2 hardware results

Authorized HIGH is admitted, unauthorized HIGH is denied, and EGL reports the effective priority accurately. Hyprland's capability and first activation were verified. A10 loaded p99 was HIGH 16.53 ms versus MEDIUM 16.61 ms: no material same-boot HIGH advantage was established. Firmware priority encoding is unchanged; only host ordering changes. Visible responsiveness benefit remains unproven pending firmware queue-priority research and physical-display qualification.

Observed all-worker CPU fell from 36.18% to 20.44% of one core, but C13k ran on battery/power-saver and C13l on AC/balanced, so this comparison is confounded. Fence latency is unmeasured because the timing flag was absent. There were zero MMU/PPL/GEM_BIND errors; the camera streamed 1080p at about 30 fps without the opt-in; Quickshell RSS was stable over the bounded observation. The package replacement hook was not exercised as a live transaction.

jgoyvaerts and others added 21 commits October 29, 2026 12:19
agx_create_context tested its flags against PIPE_CONTEXT_PRIORITY_*, the
screen's capability bits, not the PIPE_CONTEXT_*_PRIORITY creation
flags. PIPE_CONTEXT_PREFER_THREADED (1 << 3), which the GL state
tracker sets on nearly every context, is PIPE_CONTEXT_PRIORITY_REALTIME,
so since "asahi: create HIGH-priority contexts as HIGH" every threaded
GL context asked the kernel for HIGH priority.

On bare metal the kernel refuses that without CAP_SYS_NICE and the
context falls back to MEDIUM. Under virtio (muvm) the refused queue
creation never completes: vdrm_host_sync() spins in sched_yield()
forever, so every GL client in the guest (Wine's explorer.exe, glxinfo)
hangs in context creation. Compute-only contexts also came out LOW,
since PIPE_CONTEXT_COMPUTE_ONLY is PIPE_CONTEXT_PRIORITY_LOW.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit b5ccd72)
…first

Under virtio, binds and other ccmds are buffered in the guest until the
next flush, but bo_free closed the GEM handle with a bare GEM_CLOSE that
goes to the host at once. When the closed BO had a VM_BIND still queued,
the host saw the close first and its kernel rejected the bind with
ENOENT. In CS2 under FEX this happened within seconds of creating the
device and ended in VK_ERROR_DEVICE_LOST.

vdrm_bo_close() flushes the buffered ccmds before closing.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit f5685b5)
Under virtio, submits, binds and allocations are asynchronous ccmds: when
the host kernel rejects one (a lost VM rejects every later submit) the
guest never learns of it. The application keeps submitting and waits on
fences that never signal, instead of getting VK_ERROR_DEVICE_LOST.

The host counts the commands it failed in asahi_shmem::async_error. Read
it in hk_check_status and report the device lost once it moves.

The VM status mirror and query are asahi ioctls that can't be issued on
the virtgpu fd, so skip them under virtio.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit 14532ca)
Under virtio every BO is created shareable, so every BO is an external
object of the host VM, and the host kernel's cost per submit grows with
their number. CS2 held 13k-17k BOs, about 10.5k of them idle 128 KiB
command-pool BOs.

Keep the BO cache's idle trim running under virtio, and cap idle Main
command BOs per pool at HK_CMD_POOL_BO_MAX there instead of
HK_G17P_CMD_POOL_MAIN_BO_MAX.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit 2ebb59e)
A cube sample with a register level or bias capped the fragment shader's
register file at 112 halves, so that the level, staged by
agx3_tex_stage_cube_lod in a seven-bit field, could not land above half
127. That cap applied to every value of the shader: material shaders that
sample an environment cube with a computed level or bias spilled, though
they fit in the file. Counter-Strike 2 has 48 such fragment shaders.

The staging form encodes the high bit of the CEM face (f16 bit 7), so only
the level needs a low register. Keep that one value below half 128
through texture_gpr, as the coordinate already is, and give plain cube
samples with a register level or bias the same budget as other cube
samples. Cube arrays stage their face in a seven-bit field and bias with
minimum places a packed word above the allocation; both keep the cap.

Counter-Strike 2's 1969 shaders (fossilize-replay of a capture): spills
11272 -> 9958, fills 15910 -> 13995, scratch -21%, instructions -0.4%;
48 shaders change, none for the worse, typically from ~34 spills to none.
A fragment shader with 18 live vec4s around a register-LOD cube sample
goes from 245 instructions with 7 spills to 226 without, 8-11% faster
per pass, with an identical image; its CEM face lands above half 128.
A third of the dEQP-VK cube texture function, filtering, mipmap and
gather cases (807) give the same results as before.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit 5704b52)
The packer computed every cube sample's CEM in one shared scratch above
the register allocation. A cube array stages CEM's face in a seven-bit
PACK_SMPIR field, so for cube arrays that scratch had to sit below half
128, and the whole register file of any shader sampling a cube array was
capped at 112 halves. Counter-Strike 2's cube-array material shaders hold
far more than that live and spilled heavily.

Before RA, give each cube sample (but bias with minimum and scalar-
coordinate samples) a seventh source: an undefined four-word vector,
defined right before it, which emits no code. RA places it clear of every
value live at the sample, and below half 128 for a cube array
(texture_gpr); the packer computes that sample's CEM there. Cube arrays
lose the 112-half cap and plain cubes the eight halves reserved above the
allocation. Samples with their own scratch get a 176-half budget, which
keeps a heavy shader at 576 threads: Counter-Strike 2's cube-array
material passes ran faster at 177 halves than with the full 184 (185
halves, 512 threads), though they spill more there.

Counter-Strike 2's 1969 shaders (fossilize-replay of a capture): spills
9958 -> 3510, fills 13995 -> 4313, scratch -57%, instructions -1.9%; 181
shaders change, none with more instructions. The 90 cube-array shaders
go from ~240 to ~100 spills at 177 instead of 121 halves (832 -> 576
threads); 91 plain-cube shaders each free eight halves, several gaining
an occupancy step. In de_dust2 (two runs each, kernel job traces), the
median of the 50 longest render jobs fell from 1.60-1.65 ms to
1.00-1.22 ms; average fps is within run-to-run noise.

dEQP-VK cube and cube-array texture, shadow and GLSL texture-function
cases (2615, all cube arrays and a quarter of the rest): same results as
before, 586 pass, 0 fail. A cube-array pressure bench renders the same
image. test-agx3-cube-scratch checks the pass.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit c6f52b5)
A render covers the whole attachment even when its render area is a small
rectangle of it. Draws stay inside the render area and the tiles past its
bottom-right corner keep their contents, so give the render only the
extent up to that corner: the GPU then never visits those tiles.

User interfaces render this way: in the Counter-Strike 2 main menu
(2560x1440) about 1600 render passes a second draw one element into a
full-screen target with a render area under a tenth of it. At that size
the per-pass cost is dominated by fixed overhead and the frame rate moves
within noise; the tile walk shows on larger attachments. On the M4 Pro a
pass with a 128x64 area on an 8192x8192 attachment takes 24 us instead of
125 us (37 against 180 us when the memory clock is low).

Secondary command buffers that continue a render pass do not know the
area; their streams keep the full extent.

tests/vulkan_render_area.c checks that pixels inside and outside small
areas, including areas on the right and bottom edges, end up right (with
a clamp to half the area it fails), and prints the 8192x8192 timing.
dEQP-VK render-area cases and a sample of the renderpass attachment cases
give the same results as before.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit b2ddff3)
A shader that reads members of one record of a uniform buffer array as
separate expressions, as Counter-Strike 2's depth-of-field gather does
with taps[i].xy, taps[i].z and taps[i].w in a 48-tap loop, got a load
for each: once lowered to global loads, their 64-bit addresses
(base + u2u64(48 + 16 * i), + u2u64(56 + 16 * i), ...) no longer show
that they are the same record, so the vectorizer could not merge them.

Before lowering, make component reads of a uniform buffer vector loads
of the whole vector and mark the loads reorderable, as load_ubo always
is: a uniform buffer does not change during a draw or dispatch. CSE then
merges the repeated loads, and the vectorizer, run on the deref offsets,
merges neighbouring ones.

tests/vulkan_ubo_record_loop.c renders that loop twice, once as the game
writes it and once with each record read once, checks that the images
are identical and prints the cost ratio: on the M4 Pro 1.31 before this
change (69 against 64 instructions), 1.18 after (65 against 64). The
rest is the loop-invariant scale read, which stays in the loop: the game
does not enable robustBufferAccess, so it may not be read speculatively.
In a benchmark of the game's shader at 2560x1440 with 48 taps a pass
goes from 5.35 to 4.85 ms. A sample of dEQP-VK.ubo, robustness2 uniform
and binding_model uniform buffer cases (4002) passes as before.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit a044f6b)
A G17P fragment shader whose coverage or exports depend on the sample
count (sample shading, discard or sample-mask writes, a read of the sample
count), compiled without a fixed count -- a dynamic rasterization sample
count or a pipeline library -- builds its multisample variant at the first
draw that needs it (hk_fs_build_sample_variant). By then its shader object
is already in the pipeline cache, which holds the variant as absent, so
every process compiled the variant again, on the thread recording the
draw.

Under FEX that is a stall in the middle of play: Counter-Strike 2 renders
with 2x MSAA and alpha-tested materials, and on a warm cache it still
compiled about 40 such variants per de_dust2 session, 2.2 s of compiling
that landed in its 200-400 ms in-game freezes.

Store each variant that a draw builds in the disk cache, keyed by the
saved source and compile inputs that already describe it (NIR, key,
flags, robustness, features, set layouts) and the sample count, in the
form hk_api_shader_serialize() writes a variant, inline-blend NIR
included. A draw looks the variant up there before compiling it; an entry
that does not deserialize cleanly is dropped and the variant compiled.

On a warm cache Counter-Strike 2 now compiles 5 such variants per session
instead of 40 (0.7 s instead of 2.2 s). tests/vulkan_sample_variant_cache.c
records the same 2x MSAA draw from two instances on an empty cache
directory: 2.4 ms then 0.4 ms with the variant loaded, the same image;
without this change 2.2 ms then 1.5-1.9 ms, compiling it again. 535
dEQP-VK multisample, alpha-to-coverage and dynamic-sample-count cases
(monolithic, library and shader-object pipelines), run twice on a fresh
cache directory: same results as before, cold and warm, 276 pass, 0 fail.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit d4b5dce)
Whether G17P shaders emulate custom border colours is decided at the
program's first sampler that needs it. That sampler lands at a different
point among the pipelines a program compiles on worker threads each time
it runs. Shaders compiled before it leave the emulation out and get an
emulating twin, built on the draw that first binds them (hk_border_twin)
and never cached; and the shader cache key carries the setting, so a
shader compiled on one side of the sampler in one run and on the other
side in the next missed the cache.

Counter-Strike 2 appears to hit this: on a warm cache, each de_dust2
session still compiled ~35 shaders outside hk_compile_shaders() at draw
time (~1.5 s; the path twins take) and recompiled ~45 shader objects
that the previous session had compiled, and those compiles fell in its
300-900 ms in-game freezes. That they are twins and border-key misses is
inferred from the compile paths and timing, not yet confirmed in
Counter-Strike 2 with this change.

When the first sampler of a process turns the emulation on, put a marker
for the process in the disk cache; when a later run of it opens the
cache, start with the emulation on. Its shaders then compile with the
emulation from the start, under one stable cache key, and need no twins.
The emulation is a branch on the bound sampler (agx_nir_lower_custom_border),
so it is correct with or without a custom border. Programs that never
create such a sampler are unaffected.

tests/vulkan_border_emulation_cache.c creates a pipeline, then a sampler
with a custom border colour, then records a draw across the border, from
two instances on an empty cache directory: 2.2-2.5 ms then 0.27-0.29 ms
to record, the same image with the custom border colour; without this
change 2.6-2.8 ms then 1.3-1.4 ms, building the twin again. 876
dEQP-VK sampler border-colour, custom-border and clamp-to-border cases,
run twice on a fresh cache directory: same results as before, cold and
warm, 280 pass, 0 fail.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
(cherry picked from commit 9deef48)
agx3_reserve_cube_scratch gives an AGX3B cube sample a seventh source,
the four-word vector the packer computes CEM in. The validator modelled
that source like any other 32-bit texture source, two halves wide, so a
build with assertions rejected every shader with such a sample ("source
6, expected width 2, got width 8 ... Invalid vectors after cube-array
scratch") and exited. That broke the gallium blit tests on debug builds.
Release builds skip validation, and RA and the packer already size the
scratch from its vector, so the generated code is unchanged.

Read source 6 of a seven-source texture sample as eight halves, and
validate a cube and a cube-array sample before and after the pass in the
cube scratch test.

(cherry picked from commit 4222ab4)
The test records vkCmdPipelineBarrier2 but created its device without the
synchronization2 feature, which the validation layers report as an error
unrelated to what the test checks. Enable it, as
vulkan-border-emulation-cache and the other new tests already do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit f715c9f)
vulkan-sample-variant-cache and vulkan-border-emulation-cache decided that
the second, warm-cache recording compiled a shader when it took more than
a quarter of the first. Scheduling and clock changes can push a recording
that loads from the cache past that ratio, so the tests could fail with
nothing wrong.

Count the shaders compiled while recording instead. With
AGX_MESA_DEBUG=shaderdb, agx_compile_shader_nir() prints one statistics
line for every shader that is not internal. Prologs, epilogs and meta
shaders are built with nir_builder_init_simple_shader(), which marks them
internal. A multisample variant, a border-emulating twin or a G17P
inline-blend program built at draw time comes from the application's
shader and prints a line; loading one from the disk cache prints nothing.
The tests capture stderr around the recording (vulkan_compile_count.h)
and require at least one compile in the first recording and none in the
second. Recording times are still printed, for information.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 32437ae)
The context belongs to ralloc, so release its owner with ralloc_free when
uploader creation fails. Derive the embedded pipe pointer only after
checking allocation success.

Signed-off-by: Satya Benson <satya@benson.earth>
Keep T8122 G15 command streams, Gen4 samplers and the native sampler heap
for both firmware interfaces. The HAL200 profile checks C0 identity and
the reported enabled-core mask without changing physical chip selection.

Keep the scheduled queue feature contract and compiler requirements. Test
profile axes and malformed identities, and allow the no-op shim to present
the additional interface for offline compiler and device-open coverage.

Signed-off-by: Satya Benson <satya@benson.earth>
The synthetic split-query case appends starts directly, bypassing the
production allocator that zeroes new capacity. Teardown visits the entire
capacity to release retained references, so uninitialized spare slots can
crash the test. Zero this all-empty synthetic backing before teardown.

Signed-off-by: Satya Benson <satya@benson.earth>
The G15G HAL200 interface consumes sampler state from the native stream.
Leave the optional firmware-command sampler fields zero in both Gallium
and Honeykrisp while preserving the native heap and shader lowering.

Signed-off-by: Satya Benson <satya@benson.earth>
Signed-off-by: Satya Benson <satya@benson.earth>
Signed-off-by: Satya Benson <satya@benson.earth>
Signed-off-by: Satya Benson <satya@benson.earth>
…rity

Signed-off-by: Satya Benson <satya@benson.earth>
@greptile-apps

greptile-apps Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; all three previous findings are fixed and no new actionable issues remain.

Summary

The PR probes HIGH queue admission and reports the priority actually selected through Gallium, DRI, and EGL.

  • Asahi advertises HIGH only when a caller can create a HIGH queue.
  • T8122 HAL 200 devices use the G15G profile and streamed sampler state.
  • Cube texture samples get scratch space sized for their own register needs.
  • Vulkan shader compilation can combine repeated uniform-buffer reads.
  • Vulkan render commands stop their tile walk at the render area’s edge.
  • Asahi Vulkan reuses saved shader variants and custom-border emulation state.
  • Virtio BO cleanup and host command errors reach the Vulkan device checks.
  • The Zink query test clears unused entries before teardown.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Probe private HIGH queue] --> B{Create and destroy succeed?}
  B -->|Yes| C[Advertise HIGH]
  B -->|No| D[Advertise LOW and MEDIUM only]
  C --> E[Create requested context queue]
  D --> E
  E --> F[Fall back to MEDIUM if HIGH fails]
  F --> G[Report actual priority through Gallium and DRI to EGL]
Loading

Reviews (2) · Last reviewed commit: "Fix shader cache identity, shared queue ..." · Reviewed by Greptile

Comment thread src/asahi/vulkan/hk_physical_device.c Resolved
Comment thread src/gallium/drivers/asahi/agx_pipe.c Resolved
…ring

Signed-off-by: Satya Benson <satya@benson.earth>
@satchlj

satchlj commented Nov 11, 2026

Copy link
Copy Markdown
Author

Claude Fable 5.2 on @satchlj's behalf

Closing in favour of aurora-silicon#162 and its re-land aurora-silicon#177, merged on aurora-wip.

@satchlj satchlj closed this Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants