Repository navigation
media: apple: avd: frame queue, queued open-firmware transport, picture overlap - #713
Merged
Merged
Conversation
Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep the legacy open firmware serialized while moving program storage and completion ownership into tagged frame slots. Leave Borage activation dependent on the register and completion facts absent from the clean specification. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Add deterministic host tests against production helpers and KUnit coverage of slot reuse, ordered retirement, watchdog wrap, inbox retry, command builders and completion parsing. Track the unproved Borage fields explicitly and preserve session generations over firmware reboot. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Remove contexts from the scheduling list before releasing their m2m state, reject already-scheduled jobs during teardown, and keep Apple fp pointers absent on the open backend. Preserve tagged slots over inbox-full retries and avoid periodic watchdog wakeups when only submitted frames remain. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep the CODECS trailer intersection with the SoC capability table when selecting the open backend. Bound firmware copies by the mapped code resource. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Only the H.264 compatibility builder explicitly completes held slice requests during preparation. Continue submitting legacy HEVC requests that retain their capture buffer. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…ation Prevent queued contexts from observing the same free capacity. Run preparation on a context work item because m2m try_schedule can invoke device_run synchronously while the queue mutex is held. Drain that work before freeing the context. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep power through slot retirement and context drains, restore firmware before retrying reserved frames, and defer DMA releases while the DART is inactive. Re-enable a retained Apple MCPU image before mailbox link setup, without uploading it again. Pause and drain the queue for system sleep and final teardown. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Request the mailbox IRQ without automatic enable, open it only after successful firmware boot, and synchronize and close it after backend suspend. Close the host IRQ on teardown even if the final PM transition fails. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Map only DT-declared PIODMA and SRAM resources and replay the shifted SRAM base before every open CM3 boot. Keep reset, empty-domain attach/detach and firmware reboot ordered; refuse to reboot if the DMA domain cannot be restored. Keep host IRQs closed until old slots are invalidated. Add synthetic-RAM boot replay and recovery-gating KUnit coverage. Queued PIODMA use still requires the DT to describe the DMA stream as well as decode. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Replay PIODMA setup in the common boot dispatcher so queued open transports inherit it even with their own mailbox boot implementation. Keep firmware generation handling and initial-only backend fallback in the same dispatcher, and test the real open boot path through it. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Brings in the frame-queue core (avd/rewrite-core) and its runtime and system PM, PIODMA replay and DART recovery (avd/power-idle). Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Each frame in VP to PP flight owns one hardware instruction FIFO, named by EXEC bits 7:4. The hardware has 15 of them on v3 and later tier 1 parts and 7 on T8103 and T8140, so the 16-slot queue must not keep more frames in flight than the variant has FIFOs. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
T6040 decodes HEVC with the v5 header. Add HEVC to its capabilities but keep it hidden unless the loaded open firmware lists h265 in its CODECS trailer, so older firmware without the trailer behaves as before. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
When the VP instruction queue is at least half full the open transport slept a fixed 500-650 us (100-150 us on parts with v5 pacing) before the next segment. Poll the queue level every 10 us instead and continue as soon as it drops below half, keeping the old sleep as the upper bound. The submission order and queue thresholds do not change. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The open transport writes every instruction word to the VP FIFO register from the CPU, sleeps while the FIFO is half full, and kicks the PP from the VP-done interrupt thread, one frame at a time. avd-fw queue ABI 2 moves all of that into the CM3. The host streams each frame program into a word ring in CM3 SRAM, posts a 64-byte descriptor and rings mailbox 0; the firmware feeds the VP FIFO, writes the PP kick (plus one continuation kick per extra segment for HEVC, VP9 and AV1) when the VP is done, and posts a tagged completion on mailbox 1. No PIODMA is used, so the second DART stream that the current device trees lack is not needed. The transport is chosen after every firmware boot from the image info block: ABI 2 with the SRAM ring, and an SRAM aperture matching the node. Anything else keeps the legacy transport. Each frame in flight gets its own instruction FIFO (the slot index), and the queue keeps at most the FIFO count in flight. Frames of one context stay serialized by the queue core, so different streams overlap; within a stream the gain is the removed CPU word writes, pacing sleeps and interrupt round trip. T6040 now loads avd-fw-v5-t2, whose SRAM aperture matches the node, and falls back to v5-t1 when that image is not installed. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
CM3 SRAM keeps its contents across a firmware reload. After an ABI 2 image ran, a later boot of an older image would still show the queue info magic and the driver would pick the queued transport for firmware that does not speak it. Clear the magic before every open-firmware boot so only an image that publishes it again selects queue mode. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
With the firmware chaining VP to PP, the VP of picture N+1 can run while the PP of picture N is still busy, but the legacy H.264 builder kept one set of scratch buffers, one VP to PP instruction FIFO and one slice table per context, and the queue core held the m2m scheduler until a picture retired. Give H.264 two banks of everything the hardware touches while decoding a picture (instruction FIFO, pipe state, above, loop-filter and MV row buffers) and of the slice copies. A new picture takes a free bank and records it in its CAPTURE buffer; retirement frees that bank. Pictures whose last slice never arrived restart in their own bank. The legacy transport keeps one bank, so nothing changes there. The queue core now lets a legacy codec context hold as many pictures as it has banks, carries each frame's FIFO address to the transport, and only keeps the m2m scheduler for slices that hold their CAPTURE buffer, so the next picture is prepared while the previous one decodes. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
… the queue image HEVC gets the same two-bank treatment as H.264 on the queued transport: the VP to PP instruction FIFO, pipe state and all row/column scratch exist once per bank, a picture records its bank in its CAPTURE buffer, and a new done hook frees it at retirement. The VP of picture N+1 then overlaps the PP of picture N. The legacy transport keeps one bank. The per-buffer bank field is now shared by both codecs. Codecs a variant holds back until a newer firmware (fw_caps_required, T6040 HEVC) are now also enabled when the loaded image runs the queue ABI and lists the codec in its info-block codec mask, not only through a CODECS trailer. Falling back to the legacy transport restores the trailer-derived set. Both the transport choice and the codec list come from what the image reports at boot, and clients enumerate formats and size their contexts before a first decode would power the decoder up, so boot the firmware once at probe and let it autosuspend. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
avd_frame gained a DMA address for its instruction FIFO, and the host build of the queue test has no dma_addr_t. Define it in the host shim. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…eue_mutex The threaded interrupt handler takes queue_mutex before it touches the decoder. Queue reset runs from the queue worker with queue_mutex held, and its recovery called disable_irq(), which waits for that handler thread. When an interrupt arrived around a frame timeout, the handler blocked on queue_mutex and the reset blocked on the handler: the worker and the IRQ thread deadlocked with the mutex held. Runtime suspend had a similar cycle with a submission waiting for it under queue_mutex. Close hardware access with a spinlock instead: the handler runs the backend's interrupt code only under irq_lock, with hw_on set and the interrupt generation it sampled before taking queue_mutex unchanged. Turning the IRQ off clears hw_on and bumps the generation under that lock, so an access in progress finishes first and a handler woken by an older firmware instance leaves the hardware alone, and then uses disable_irq_nosync(). Device removal, which holds no queue_mutex, still synchronizes with the handler. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The old driver kicked the PP for every VP DONE message of the open firmware. The legacy shim in the frame queue dropped a VP DONE that came in while no frame was in flight. That happens when a surplus PP DONE of a multi-kick picture completed the next frame early: the next frame's VP DONE then found no active frame, its PP was never started, and it timed out. Kick PP for the last submitting context in that case, as before. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Reading MBOX1_RETRIEVE when the mailbox is empty returns the previous message again. On a spurious or early interrupt that is a repeated PP DONE, which completes the next frame before its PP has run. The next PP then never reports done, the frame times out and the decoder is reset. The legacy shim and the queued transport both popped unconditionally. Check the empty flag in MBOX1_STATUS (bit 17) first, give a message that is not readable yet up to 10 us, and otherwise acknowledge and ignore the interrupt. Both transports share the helper. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
A frame timeout on the J616s was followed by the machine going down with nothing in the journal, so nothing showed how far the recovery got. Log every recovery step (stop firmware, reset, DART flush, firmware boot) before it runs, with the DART, runtime PM and IRQ state at the start, refuse and warn once if recovery is ever entered with the decoder powered off, and name the codec, slot state, frames in flight and transport in the timeout message. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
avd-inst.h builds every instruction word with FIELD_PREP() but relied on an indirect include. The arm64 build gets it through other headers; the UML KUnit build of the avd tests does not and fails. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The recovery trail queried the runtime PM state before the existing inactive-DART check, which the PM KUnit test exercises with no device. Check the DART first and log that case on its own. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…e context
job_ready() admitted at most one context at a time (queued_ctx) and only
while a slot was free. Contexts that queued work while another was
admitted were rejected, and were retried only from the queue worker,
which walks the context list in order: the first context won every time
and later ones could wait for seconds. With four streams the starved
stream's OUTPUT buffer was not returned within FFmpeg's two-second poll
("Failed waiting on OUTPUT buffer") and the run failed, with no decoder
timeout or error.
Drop the reservation and the capacity condition from job_ready() so every
context with work enters the m2m job queue, which runs jobs one at a time
in arrival order. A job that finds no free slot or no transport capacity
is kept as m2m's running job and restarted by the queue worker once a
frame retires; draining its context finishes it.
Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The deferred-dispatch restart in the queue worker called the backend's max_inflight() unconditionally. The PM KUnit fixtures use backends without a transport and crashed there. Treat a missing hook as one frame. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
This was referenced Oct 31, 2026
The J616s hung hard three times during decode benchmarks, each time after frame timeouts and decoder resets, once beyond what the hardware watchdog could recover. A stuck bus access, a reset that cuts DMA in flight, or DMA into memory the driver already gave back can do that. Close each of those paths: - Recovery stops the CM3 firmware, then blocks the decoder's DMA by attaching the empty IOMMU domain and lets in-flight transfers drain into DART faults, and only then asserts the pmgr reset. The domain is restored after the reset and checked before the firmware boots. - A third recovery within a minute, or any failed recovery, retires the decoder: every later frame fails at submission without touching the hardware until the module is reloaded. A stream is stopped at its first frame that fails in hardware. - The legacy transport never writes the VP FIFO register without room in its queue: it checks the level before every word (the old code only looked between segments, and programs exceed the 128-word queue), waits up to 2 ms for room, and otherwise leaves the frame to the watchdog. - A PP DONE completes the active legacy frame only after its VP finished and its PP was kicked. A surplus PP DONE of a multi-kick picture no longer completes the next frame while the engine is still writing it. - The interrupt handler's hardware access lock is now a mutex, so no decoder register is touched with interrupts off. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…tting Closing a stream (STREAMOFF, close or an m2m job abort) while it still owned a slot reset the whole decoder, cancelling every other stream's frames with their DMA in flight. On the J616s both resets logged before the 04:11 hard hang came from this path (no frame had timed out), each when FFmpeg gave up on one of four concurrent streams. Cancel the closing stream's frames that never reached the decoder, then wait for the ones it already owns. They complete normally, or the queue worker times them out and recovers as for any other stuck frame. Only if frames are still owned a second after the frame timeout does the drain fall back to the old reset, and it says so in the log. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The queued open-firmware transport copied each frame's instruction program into the CM3 data SRAM with memcpy_toio(), which on arm64 issues 64-bit stores for aligned runs and byte stores for unaligned ends. Every other access to that SRAM (the info block, the descriptors, the boot time clear) is a 32-bit readl()/writel(), and the firmware consumes the ring as 32-bit words. The first decode through this transport on an M2 hung the machine hard right after the doorbell, before any completion or log line, which is what an unsupported access width on that bus would do. Use __iowrite32_copy() so the ring is written with the same word stores as the rest of the SRAM. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep a queued frame's FIFO and PP continuation metadata independent of later preparation on its context. Encode its FIFO index in every VP continuation and END, check the mailbox before transferring ownership, and split SRAM copies at both the ring and u32 counter boundaries. Never let a reserved mailbox retry be overtaken by a later picture of that context; already-posted pictures and unrelated contexts still overlap. Legacy VP and PP instruction semantics remain unchanged. Drain tagged replies, retain fatal I/O completions until quiesced reset, and add dynamic-debug descriptor, ring-counter and mailbox snapshots at submission, completion and timeout. A sole fatal frame must not retire before recovery merely because no other frame can arm the watchdog. Add host coverage of the production H.264 builder, queued submit, stream, IRQ and T8112 FIFO configuration. Extend the firmware Unicorn harness with reset-vector boot, CM3/NVIC routing and real mailbox status bits. The companion MIT firmware patch fixes the missing queue IRQ routes and the erroneous use of an IRQ-source bit as the outbox EMPTY status bit. The old harness injected both interrupt delivery and the incorrect status, so arbitrary-word transport tests concealed those firmware failures. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Plain job_finish() warns for every HOLD_CAPTURE-capable queue, even when the current source is a final slice. buf_done_and_job_finish() releases OUTPUT and final CAPTURE immediately, so neither API can hand both active buffers to an asynchronous frame slot while releasing the scheduler. Add job_finish_detach() to atomically remove the current pair and finish dispatch without completing either buffer. Document the driver's DMA ownership and eventual completion obligations. Reject partial HOLD slices; they continue to use the existing held-buffer completion API. Use this handoff for AVD's final slices to preserve early dispatch and VP/PP overlap. Return slot-owned buffers only after tagged retirement or quiesced recovery. Reject stale deferred submit work whose context no longer owns the running m2m dispatch. Keep existing HOLD warnings intact. Exercise the production core helpers and AVD submit/send/retire with a strict host scheduler: held/final slices, overlapping pictures, two contexts, capacity and inbox retries, stale work and error paths. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Resolve the legacy driver conflicts at the rewrite's queue and PM layer. Park DMA and close IRQ access before an exhausted budget returns buffers, and leave the empty DART domain attached after a failed reset or boot. Keep a decoder resting until its budget window expires, then reset while still parked before restoring DMA and booting. Runtime resume must not boot resting or retired hardware. Use the unbound watchdog workqueue. Preserve the queued transport, detached dispatch, quiesce-before-reset, per-stream stop and closing-context drain fixes. Retain the upstream AV1 reference bounds fix and all non-AVD updates from the merge parent. Add production recovery/PM host coverage for failed reset/boot, budget exhaustion and parked runtime resume, plus dispatch and KUnit rest cases. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…sses A queued frame must use the FIFO saved by its program builder. Reject a missing address before configuring registers or posting a descriptor, rather than substituting a mutable context FIFO during a retry. Include the saved DMA address in the existing dynamic-debug program record. Exercise the H.264 bank allocator and selector, IDR/P/B address emitters, all valid slots, and the v3/v5 host register layouts. Inject a zero address into each bank-1 buffer and require the host test to fail. Replay the resulting descriptors, programs and MMIO in a routed Unicorn workload matrix, holding decoder DONE events and checking DMA operands and tags. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Brings in #758 follow-ups and #773 (PMC performance floor with the AVD perf-domain consumer). The one conflict is probe: keep the rewrite's avd_pm_enable() and firmware probe boot, and settle the attached domain links to suspended before it, as #773 does for the stock driver. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Give queued VP9 pictures separate DMA banks and retain them until PP retirement. Publish backward probability adaptation from a tagged VP DONE callback, so the next picture can prepare and run VP while PP is active. Forward-only, non-refreshing and error-resilient producers impose no count wait. Preserve the preceding picture's temporal-MV input across banks. Negotiate an optional queue ABI 2 VP notification, with a separate full-tag record and no early release of descriptor or buffer ownership. Include the matching MIT firmware patch and host/Unicorn regressions. Firmware without the feature retains one VP9 picture per stream; legacy programs and saved probability contexts have an independent before/after comparison harness. Document AV1's CDF and segmentation-tail dependencies pending evidence for their visibility and CPU synchronization at VP DONE. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Extract avd_open_capabilities from avd-backend-open.c after the transport split. Update the boundary structures and mutations for the current parser, including its saved firmware capability mask and bounded 64-byte trailer. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Readiness can be checked while STREAMON is still allocating codec storage. Keep ctx->priv unpublished until all VP9 banks and count-pointer tables are initialized, and pair its publication with the readiness callback's acquire load. Failed allocations free their local context without exposing partially initialized banks. Exercise readiness before start, from every bank allocation, and after stop in the production host fixture. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
RyanTheTide
marked this pull request as ready for review
November 1, 2026 07:53
The optional tagged VP DONE event is now committed in avd-fw (bf49a31b, queue feature bit 2), so the patch kept under tests/ is redundant. Remove it and name the firmware commit instead. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The CM3 IRQ routing and FIFO EMPTY bit 17 fixes are committed in avd-fw (50d7f869), so the patch kept under tests/ is redundant. Remove it and name the firmware commit instead. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep two complete AV1 scratch banks on queued firmware advertising tagged VP DONE. Look ahead at the pending request's frame control and follow its actual primary-reference timestamp. Default CDFs and references outside the in-flight set do not wait for an unrelated picture. The open firmware's VP event does not establish CPU visibility of AV1 hardware-written CDF, segmentation or motion-field tails. Retain PP retirement dependencies for those inputs. When frame-end CDF updates or segmentation-map writes are disabled, copy preparation's immutable input snapshots into the consumer bank instead of reading live CAPTURE storage. Only PP retirement releases the saved picture's bank. Preserve the legacy emitters and tail preparation, add production host flag/ownership tests and a routed AV1 Unicorn overlap test, and document the remaining visibility evidence requirement. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
…nputs" This reverts commit 2f9bd00. The AV1 overlap has host and emulator tests but has not run on AV1 hardware (T6040, T8140) yet, and this branch is about to merge with the code that was qualified on the J616s and the M2. The change is kept on avd/rewrite-av1-overlap and comes back once it is bit-exact on hardware. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Contributor
|
@RyanTheTide, a bug report for the queued transport, since issues are off on this repo. J616C (M4 Max): the queued AVD transport decodes VP9 with segmentation wrong Machine and build
Queue mode on T6041 needs the T6040 Result In
No AVD kernel lines, no timeouts and no DART faults in any of these runs. Shortest reproduction: ffmpeg -f lavfi -i testsrc2=size=1920x1080:rate=30:duration=4 -c:v libvpx-vp9 -deadline good -cpu-used 2 \
-row-mt 1 -tile-columns 2 -auto-alt-ref 1 -lag-in-frames 25 -aq-mode 1 -b:v 3M -g 120 -pix_fmt yuv420p seg.webm
ffmpeg -i seg.webm -pix_fmt yuv420p -f framemd5 sw.md5
LIBVA_DRIVER_NAME=v4l2_request ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/card1 -i seg.webm -pix_fmt yuv420p -f framemd5 hw.md5Every clip above has Likely cause (not verified)
Most YouTube VP9 is segmented, so on queue mode this hits most YouTube playback. It probably affects the J616s too. Bench streams without segmentation, like the ones here and in #787/#793 if they were encoded that way, wouldn't catch it. For now this Max stays on |
notmysegv
pushed a commit
to notmysegv/linux
that referenced
this pull request
Nov 4, 2026
…tion/aurora Bring integration/aurora up to aurora-wip b359258, the source of 7.3.0-aurora-fleet14f, promoted on the M1 Pro, M2, Neo and J616s on 2026-11-01 and 11-02. The range also covers the fleet14c, fleet14d and fleet14e sources (f41ac3d, 4759557, 757fc07), each booted on hardware. It adds aurora-silicon#773, aurora-silicon#776, aurora-silicon#777, aurora-silicon#737, aurora-silicon#713, aurora-silicon#785, aurora-silicon#786, aurora-silicon#788, aurora-silicon#789 and aurora-silicon#787. No conflicts. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AVD frame-queue rewrite, integration branch. It carries the queue core and PM, a queued transport for the open firmware, H.264/HEVC/VP9 picture overlap, and fixes for the timeouts seen on the J616s. Based on aurora-wip (f41ac3d merged in, cbbe442), so it carries the PMC SOC floor consumer from #773. The current hardware build is k4m (a1b7685).
J616s (M4 Pro), rewrite + PMC floor: H.264 1080p 1094 fps, against 334 on the stock driver and 2582 on macOS. Every frame was bit-exact. The full tables are under Hardware results.
What is in it
avd/rewrite-coremedia: apple: avd: introduce frame slots and backend transport foundation #701,avd/power-idlemedia: apple: avd: runtime and system PM for the frame queue #707): 16 frame slots with tags, in-order retire per context, per-frame watchdog, queue reset, runtime/system PM, DART empty-domain recovery. Apple MCPU backend present but disabled until its register and reply facts are verified.0x5453xxxxon mailbox 1.avd-fw-v5-t2.bin(its SRAM aperture), falling back to v5-t1. T6040 HEVC is listed when the queue image's codec mask has it.avd/rewrite-integ-mvp2until it is shown bit-exact.v4l2_m2m_job_finish_detach(). With it, a job can leave the m2m queue while its CAPTURE buffer is held (HOLD_CAPTURE_BUF) without the job_finish WARN.disable_irq()for the threaded handler, which waited for queue_mutex.Not in this PR
The current driver got the same safety fixes, without the rewrite, in #725 and #758 (both merged).
Codec
frame_opsbranches (#700, #702, #703, #704) and the Apple MCPU backend activation. The legacy codec builders feed both transports.Tests
ARCH=arm64 LLVM=1 W=1: 0 warnings.tests/run-host.shPASS. VP9 dependency/bank tests run under ASan/UBSan. Legacy VP9 programs and probability contexts are byte-identical to the previous revision.make test, plus a unicorn run of the real CM3 code on all seven images: SRAM ring, wrap, streaming, multi-kick, stray PP DONE, in-order PP, and VP(1) issued before PP(0) DONE.tests/.kunitconfigandavd-pm.kunitconfig38/38.Hardware results
Method:
engines-20261101/avd/{j616s-k4m,m2-k4k}).J616s (T6040), fleet14c with the PMC SOC floor (#773), k4m, 2026-11-01 18:31. Zero DART or AVD kernel lines during the run.
The macOS figures are the 10-31 baseline on the same J616s: VideoToolbox decode with in-memory demux and asynchronous decode, the same shape as avd-bench. HEVC is new on the J616s: the stock firmware does not list it. AV1 is serialized on the rewrite, so it matches the stock path with the floor and gains nothing more.
M2 (T8112), fleet14, k4k (9b125f8), 2026-11-01 16:00, no PMC floor on this SoC. Zero kernel lines.
M2 (T8112), fleet14d, k4m (this branch's driver source), 2026-11-01 19:01. Bench rows were bit-exact with zero kernel lines.
Fluster conformance, M2 fleet14d (rewrite k4m) vs the stock in-tree driver (fleet14b, #725 + #758). conf.py feeds every vector through avd-bench and compares its MD5 with the Fluster reference.
The per-vector results are the same as stock: no new failures and no timeouts.
The decoder rejected some unsupported H.264 vectors with a hardware error. It recovered and was reloaded 9 times, as on stock. Every DART fault in the run is an in-flight DMA hitting the deliberately empty domain while a recovery has DMA blocked, between "recover: block DMA" and "recover: reset". None fall outside a recovery, there were no hangs, and the machine stayed up.
The H.264 fixes on avd/conformance-fixes (68/135 on the stock transport) follow as a separate series.
Not run yet on the rewrite: the M1 Pro and the Neo.
How to run it:
🤖 Generated with Claude Code