Skip to content

media: apple: avd: Do not pop an empty mailbox - #640

Closed
AxxzyWasTaken wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:avd-mailbox-guard
Closed

AxxzyWasTaken wants to merge 1 commit into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:avd-mailbox-guard

Conversation

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor

The AVD interrupt handler pops MBOX1_RETRIEVE without checking that the mailbox holds a message. An interrupt that finds it empty pops the previous message again. When that message is a post-processing "done", the driver finishes the frame early and starts the next one while the PP is still busy. That frame's PP never completes, the 2 s watchdog resets the decoder, and the following frames fail with a VP error (H8 error for VP9), so the stream dies. On J616C this hit Chromium/YouTube VP9 (e.g. picture-in-picture while a game ran) every 5-15 minutes.

How it was found

A per-job event trace in the driver (run, submit, every interrupt status, watchdog) caught two real faults. Both had the same shape:

  1. Frame N's PP "done" arrives 8 us after its kick. Normally that takes 400-1000 us; healthy decoding produced 0 such completions in 1400+ frames.
  2. Frame N+1's VP finishes and its PP is kicked, but the PP never reports done.
  3. Watchdog timeout, reset, H8 error on the following frames.

MBOX1_STATUS (+0x5c) reads back the mailbox state: bits 8-11 write pointer, bits 12-15 read pointer, bit 17 empty. In healthy decoding exactly one message is pending at every interrupt.

The fix

Check the empty flag before popping. If it's set, give a late message up to 10 us; if the mailbox stays empty, clear the interrupt and ignore it.

Testing (J616C, M4 Max, 7.2.10)

  • VP9 and H.264 clips with 1080p<->480p switches, alone and 3 streams at once, under all-core CPU load, and under full GPU load plus memory bandwidth load: every frame bit-exact against software decoding (ffmpeg framemd5), 0 timeouts.
  • An empty-mailbox interrupt occurred during those tests and was ignored. The old handler would have re-read the previous message.
  • Before the fix, Brave PiP plus a game failed every 5-15 minutes. A build that read the status register before popping ran ~37,000 frames of the same use without a fault.
  • Built warning-free on integration/aurora (23827fe).

The interrupt handler reads MBOX1_RETRIEVE without checking that the
mailbox holds a message. An interrupt that finds it empty pops the
previous message again. When that is a post-processing "done", the
frame is finished early, the next frame starts while the PP is still
busy, its PP never completes, and the watchdog resets the decoder; the
frames that follow then fail with a VP error.

Seen on a J616C (T6041) with VP9 in Chromium while the system is under
load: a PP "done" 8 us after its kick instead of 400-1000 us, then a
timeout on the next frame.

Check the mailbox's empty flag first, give a late message up to 10 us,
and ignore the interrupt if the mailbox stays empty.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@RyanTheTide

Copy link
Copy Markdown
Contributor

Reviewed for the M4 merge train. The fix is right in principle: reading MBOX1_RETRIEVE without checking the empty flag re-reads the previous message. But it no longer applies. integration/aurora has restructured the AVD interrupt path since this was written: the handler now goes through avd_pm_irq() under queue_mutex, so the conflict in avd-drv.c is a real one.

Could you rebase it onto integration/aurora (19f71a1) and put the empty check where the current path pops the mailbox? The AVD also runs on the J616s (T6040), so after the rebase we'll run a decode no-regression check there.

@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Closing: #725 added the same empty-mailbox check.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants