Skip to content

media: apple: avd: park the decoder with DMA blocked; rest instead of retiring on a reset storm - #755

Closed
RyanTheTide wants to merge 2 commits into
aurora-wipfrom
avd/retire-parks-dma
Closed

RyanTheTide wants to merge 2 commits into
aurora-wipfrom
avd/retire-parks-dma

Conversation

@RyanTheTide

Copy link
Copy Markdown
Contributor

Two follow-ups to #725's recovery policy.

  1. Keep DMA blocked once the decoder is retired. Before this change, the out-of-budget path retired the decoder without quiescing it, and the timed-out frame's buffers went back to userspace while the engine might still be running. Now the decoder is parked first: interrupt off, CM3 stopped, empty IOMMU domain attached and left attached. A reset that fails after DMA was blocked also keeps it blocked. The empty domain is detached at remove.
  2. Rest instead of retiring on a reset storm. A third recovery within a minute used to retire the decoder until a module reload. On the M2, running the H.264 conformance suite retired it within three seconds, because a few vectors use features the hardware rejects. A user with one bad file would lose hardware decode until a reboot. Now the decoder is parked until the budget window ends, and frames fail fast without touching the hardware. The next frame after that resets it with DMA still blocked, restores the domain and boots it. Only a failed reset, restore or boot retires the decoder.

Why parking matters, from the #725 M2 run: after a VP9 H8 error and recover: stop firmware, the DART logged ten translation faults on the empty domain within 2 ms. The engine keeps doing DMA after it has reported an error.

Tests

Built as an external module against 7.3.0-aurora-tt3 with W=1, no warnings. Run on the M2 (T8112) on 2026-11-01 as a module swap. Logs are in reports/engines-20261101/avd/m2-725/.

  • Three forced VP errors within a minute (a bit-flipped H.264 stream decoded through ffmpeg -hwaccel v4l2request): two quiesced recoveries, then decoder resting after repeated resets, new frames fail for up to 60 s.
  • A clean decode during the rest failed fast, with no hardware access and no hang.
  • After the window: recover: wake after rest, recover: done, then a clean decode bit-exact at H.264 1080p 503 fps and HEVC 1080p 848 fps.
  • 40 decodes SIGKILLed at random points with frames in flight: no kernel line, and a clean decode afterwards was bit-exact at 503 fps.
  • No DART fault outside a recovery window.

The M1 Pro (T6000) run follows. This PR stays draft until it passes.

🤖 Generated with Claude Code

When the recovery budget runs out, or a reset fails, the decoder is
retired: its jobs fail without touching the hardware until the module is
reloaded. The frame that triggered it is still finished with an error,
so its buffers go back to userspace while the engine that just timed out
may still be running, which is the DMA-after-free pattern behind the
J616s hangs.

Retire through the same first steps as a recovery: disable the
interrupt, stop the CM3 and attach the empty IOMMU domain, then leave it
attached. A still-running engine can then only fault at the DART. A
reset that fails after DMA was blocked keeps it blocked as well. Detach
the empty domain at remove so a reload starts from the DMA domain.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
… storm

A third recovery within a minute retired the decoder until the module
was reloaded. That is too harsh for a desktop: on an M2 the H.264
conformance suite retired it within three seconds, because a few
vectors use features the hardware rejects with a VP error, and every
later stream failed until a reload. A user with a bad file would lose
hardware decode until a reboot.

Recoveries are now quiesced (DMA blocked at the DART before the reset)
and have held up under forced errors, so the budget only needs to bound
their rate. When it runs out, park the decoder (interrupt off, CM3
stopped, DMA blocked) and fail new frames without touching it until the
budget window ends. The next frame after that resets it with DMA still
blocked, restores the DMA domain and boots the firmware. Only a reset,
restore or boot that fails retires the decoder.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Folded into one combined AVD safety-and-recovery series on branch avd/safety-recovery (same commits, on top of aurora-wip 5067ca4). Per the new one-PR-per-series rule, it opens as a single PR once the M1 Pro run and the ANE strict-flush receipt are in.

@RyanTheTide RyanTheTide closed this Nov 1, 2026
@RyanTheTide
RyanTheTide deleted the avd/retire-parks-dma branch November 1, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant