Skip to content

drm/apple: Fix a GPU fence that never signals wedging the display - #1241

Merged
RyanTheTide merged 2 commits into
aurora-silicon:integration/aurorafrom
rolotrealanis98:pr/apple-drm-bounded-fence-wait
Nov 9, 2026
Merged

RyanTheTide merged 2 commits into
aurora-silicon:integration/aurorafrom
rolotrealanis98:pr/apple-drm-bounded-fence-wait

Conversation

@rolotrealanis98

Copy link
Copy Markdown
Contributor

What this is

One commit on integration/aurora (bdb9994), drivers/gpu/drm/apple only. apple_atomic_commit() replaces drm_atomic_helper_commit as the driver's .atomic_commit: a copy of the helper (setup, prepare, the interruptible pre-swap wait for blocking commits, swap, the worker's wait for dependencies, the tail, cleanup, and the helper's fence deadline hint) whose wait for the planes' in-fences is bounded. All planes of a commit share one second (APPLE_PLANE_FENCE_TIMEOUT_MS); a fence that does not signal in time is dropped with one rate-limited drm_warn naming its driver and timeline, and the buffer is scanned out as it is. Only the helper's self-refresh accounting is left out, which this driver does not use.

Second part: if DCP does not complete a swap within drm_atomic_helper_wait_for_flip_done()'s 10 s, the commit's own pending event is now sent as failed (-ETIMEDOUT on its out-fence) through the new dcp_drm_crtc_flip_lost(), which sends only the event whose flip_done belongs to that commit. The compositor's next commit is then no longer refused with -EBUSY.

The problem

drm_atomic_helper_commit() waits for each plane's in-fence with dma_fence_wait() and no timeout, in the commit worker and, for blocking commits, before the swap. On the J716s (MacBook Pro 16-inch, M5 Pro) the in-fence is the scanout buffer's implicit write fence from the GPU (zink imports its job fence into every exported dma-buf). When a GPU fence never signals, after a GPU recovery for example, the worker sleeps for good: no swap reaches DCP, the flip event never comes, and every later nonblocking commit fails with -EBUSY from stall_checks(), which only logs at debug level. The 10 s flip_done timeout is never reached, so nothing is logged at all.

That is what the machine looked like on 2026-11-08 at about 10:50 UTC: after a GPU recovery killed the compositor (the GPU half is the drm/asahi PR "work a GPU recovery dropped fails spared and within a bound"), every respawned Hyprland set the panel up (3456x2234@120, DPMS on, backlight on) and showed nothing: zero iomfb_swap_complete events, aquamarine logging drm: Cannot commit when a page-flip is awaiting on every commit, no flip_done timed out and no DCP crash line. A reboot was the only way out. Tracker: rolotrealanis98/m4-m5-linux#36; analysis: GPU-RECOVERY-KILLS-COMPOSITOR.md, section 1. The two sysrq task dumps taken during such wedges showed no blocked DRM commit frame, which also fits a flip whose completion never arrived (the second part of this commit).

With this commit a GPU bug costs a stale or torn frame and a warning instead of the panel, and a lost flip costs the compositor one failed event after 10 s instead of a session.

Hardware result (J716s, DCP display, firmware 27.3)

Kernel 7.3.0-g12c445fa5371 (our J716s tree: integration/aurora 33bec40 + #1012 + #1173 + the J716s DCP/board series, with this commit, the two drm/asahi recovery commits and two recovery-diagnostic commits), Hyprland on zink at 3456x2234@120 with VRR:

  • Two 10 min multi-client GPU stress runs with 3 GPU recoveries in the first (RESULTS.md 12:08, 12:24): every frame went through apple_atomic_commit(); swaps continued at 1024 per 10 s after the load; no stuck page flip, no new page-flip is awaiting line (the two per session are at session start, before any load, and the session draws afterwards), no RCU stall.
  • The same kernel has run as the machine's daily since 12:45 UTC, 23/23 in our display verifier at 14:19 UTC (RESULTS.md 14:19).
  • Not observed: neither the fence timeout (no not signalled after 1000 ms warning) nor the lost-flip path fired, because the drm/asahi commits in the same kernel kept every fence signalling. This commit is validated as the frame path for a day of use and by reading; its two failure paths are exercised only by inspection.
  • An OOM at 12:47 UTC on this kernel was first suspected to be a per-frame leak in this commit path. It was a Hyprland 0.57.1 direct-scanout retry storm (519 swapchain reconfigures, 1557 31 MB GBM buffers never released, memory returned when the process died); this commit was read line by line against the helper (no double swap, no skipped cleanup) and the storm did not need a GPU recovery. RESULTS.md 12:43 and 13:15.

Checks

  • Applies to integration/aurora bdb9994 without change (same diff as tested).
  • checkpatch.pl --strict: 0 errors, 0 warnings, 0 checks.
  • Compile: make W=1 drivers/gpu/drm/apple/ (LLVM arm64, defconfig + asahi.config, clang 21) at the base and at the commit, objects rebuilt from scratch: see the Checks line below.

Compile check result: drivers/gpu/drm/asahi/ + drivers/gpu/drm/apple/ with W=1 CLIPPY=1 give the same 68 warnings at the base (bdb9994) and at this commit; none new, none gone (the 68 are pre-existing clippy and deprecation notes in drivers/gpu/drm/asahi, none in drivers/gpu/drm/apple).

Risks

  • The copy of drm_atomic_helper_commit() has to follow the helper if it changes; it is ~120 lines and keeps the helper's order, with comments pointing at each helper step.
  • A fence that signals late (over 1 s) is scanned out early: a torn or stale frame, with one warning per minute at most.
  • apple_fail_lost_flips() skips t8132_ext_crtc() CRTCs (the M4 external path keeps its own event handling) and only sends an event whose flip_done is the commit's own.

Questions

  • Would you prefer the bounded wait as a helper option upstream (drm_atomic_helper_commit() with a fence timeout) rather than a driver copy? We can propose that; this is the smallest change that keeps the panel alive today.
  • Is 1 s the bound you want? It is many frames at the panel's slowest refresh; a GPU job that legitimately takes longer than 1 s to produce a scanout buffer would be torn.

drm_atomic_helper_commit() waits for every plane's in-fence with
dma_fence_wait() and no timeout: in the commit worker before the
driver's commit tail, and, for blocking commits, before the swap. The
in-fence is the scanout buffer's implicit write fence (zink imports its
GPU job's fence into each exported dma-buf) or the compositor's
IN_FENCE_FD. When a GPU fence never signals, after a GPU recovery for
example, the worker sleeps for good: commit_planes() never runs, no swap
reaches DCP, the commit's flip event never comes, and every later
nonblocking commit fails with -EBUSY (stall_checks(), only a debug
message). The 10 s flip_done timeout is never reached, so nothing is
logged. The panel stays black across compositor restarts, which see
"Cannot commit when a page-flip is awaiting", until a reboot.

Commit through a copy of drm_atomic_helper_commit() whose fence wait is
bounded: all of a commit's planes share one second, and a fence that
does not signal in time is dropped with a rate-limited warning naming
its driver and timeline. The buffer is scanned out as it is, so a GPU
bug costs a stale or torn frame instead of the panel. The copy keeps the
helper's sequence (setup, prepare, the interruptible pre-swap wait for
blocking commits, swap, the worker's wait for dependencies, the tail,
cleanup) and its fence deadline hint, and leaves out only the
self-refresh accounting, which this driver does not use.

Once the commit goes on, its flip completes as usual. If DCP then does
not complete the swap within drm_atomic_helper_wait_for_flip_done()'s
10 s, the commit's event stayed pending and a compositor would still
wait for it before committing again. Send that event as failed
(-ETIMEDOUT on its out-fence) after the wait; only the event that
belongs to this commit is sent. The commit then leaves its CRTC's list
at cleanup_done as before, so later nonblocking commits are not held by
it either.

Signed-off-by: Rolando Trevino <rolotrealanis@gmail.com>
Assisted-by: LLM Claude
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-08T15:52:55.371005Z 76b0942 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76b09427c2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread drivers/gpu/drm/apple/dcp.c Resolved
Comment thread drivers/gpu/drm/apple/apple_drv.c Resolved
Comment thread drivers/gpu/drm/apple/apple_drv.c Resolved
Follow-up to the automated review of this pull request, kept as a separate
commit for review and squashed on request.

drm/apple: Fix a GPU fence that never signals wedging the display
drm_atomic_helper_commit() waits for every plane's in-fence with
dma_fence_wait() and no timeout: in the commit worker before the
driver's commit tail, and, for blocking commits, before the swap. The
in-fence is the scanout buffer's implicit write fence (zink imports its
GPU job's fence into each exported dma-buf) or the compositor's
IN_FENCE_FD. When a GPU fence never signals, after a GPU recovery for
example, the worker sleeps for good: commit_planes() never runs, no swap
reaches DCP, the commit's flip event never comes, and every later
nonblocking commit fails with -EBUSY (stall_checks(), only a debug
message). The 10 s flip_done timeout is never reached, so nothing is
logged. The panel stays black across compositor restarts, which see
"Cannot commit when a page-flip is awaiting", until a reboot.
Commit through a copy of drm_atomic_helper_commit() whose fence wait is
bounded: all of a commit's planes share one second, and a fence that
does not signal in time is dropped with a rate-limited warning naming
its driver and timeline. The buffer is scanned out as it is, so a GPU
bug costs a stale or torn frame instead of the panel. The copy keeps the
helper's sequence (setup, prepare, the interruptible pre-swap wait for
blocking commits, swap, the worker's wait for dependencies, the tail,
cleanup) and its fence deadline hint, and leaves out only the
self-refresh accounting, which this driver does not use.
Once the commit goes on, its flip completes as usual. If DCP then does
not complete the swap within drm_atomic_helper_wait_for_flip_done()'s
10 s, the commit's event stayed pending and a compositor would still
wait for it before committing again. Send that event as failed
(-ETIMEDOUT on its out-fence) after the wait; only the event that
belongs to this commit is sent. The commit then leaves its CRTC's list
at cleanup_done as before, so later nonblocking commits are not held by
it either.
The swap of that flip may still complete after the next commit installed
its event. The H17P and desktop 26.x paths match a completion to its
present; on the older firmware (12.3, 13.5, the 14.7 template, H17G) a
completion retires whatever flip is pending, so there the lost swap's
completion is now ignored. The framebuffers the lost swap displaced are
unarmed, as the swap watchdog does for a dropped swap, so that the next
swap re-arms them and its completion releases them; they were otherwise
kept until the display powered off.

Compile-checked with W=1 CLIPPY=1 over drivers/gpu/drm/{asahi,apple}: no new
warnings against the base; checkpatch --strict clean; rustfmt unchanged.

Signed-off-by: Rolando Trevino <rolotrealanis@gmail.com>
Assisted-by: LLM Claude
RyanTheTide added a commit that referenced this pull request Nov 9, 2026
@RyanTheTide
RyanTheTide merged commit 8d9ef3d into aurora-silicon:integration/aurora Nov 9, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants