Skip to content

media: apple: avd: enable firmware-gated T6040 HEVC and retain request ownership - #698

Closed
RyanTheTide wants to merge 3 commits into
aurora-wipfrom
fix/avd-request-pipeline-hevc
Closed

RyanTheTide wants to merge 3 commits into
aurora-wipfrom
fix/avd-request-pipeline-hevc

Conversation

@RyanTheTide

@RyanTheTide RyanTheTide commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

T6040 match data now admits HEVC with its existing v5 header only when the loaded firmware's CODECS trailer advertises h265. Legacy firmware keeps H.264/VP9/AV1 admission. Other matches and pacing policies retain their behavior. The empty-mailbox protection from #640 is included because repeating a PP completion would invalidate ownership.

The m2m scheduler currently waits for PP completion before another context can prepare a request. This patch retains explicit source/capture buffers and codec state after scheduler release, letting one independent context prepare on the CPU while the previous context owns the hardware. IRQs use that hardware owner; streamoff/release and suspend drain it. Watchdog/PP completion has one retirement winner, partial submission fails explicitly, and reset failure prevents subsequent FIFO reuse and firmware boot on runtime resume.

This is a draft for host-side review and hardware qualification. Hardware remains serialized because PP completions have no request tag. Each context retains one request; H.264 keeps its held-capture scheduling path. This does not claim single-stream acceleration, concurrent firmware requests, correct HEVC pixels, or qualified hardware recovery. The extra retirement workqueue hop also needs performance measurement. The failed-reset host case proves later submissions and firmware resume are rejected; it does not prove a failing physical reset has quiesced DMA. That device-recovery boundary still needs hardware review.

Validation:

  • Four ownership KUnit cases executed through the host shim under ASan/UBSan.
  • Extracted production submission/IRQ/watchdog/reset/retirement/buffer paths pass deterministic two-context, early/duplicate/wrong-slot IRQ, timeout, PM failure, held-capture, streamoff failed-reset and failed-table-cleanup cases. Five deliberate ownership regressions compile and are detected at runtime.
  • Actual OF match data passes; the firmware-tail parser passes 29 cases and detects all three parser mutations.
  • PASS on c7edee0: arm64 Clang 22.1.6, W=1, CONFIG_WERROR=y, AVD and both KUnit translation units. All ten outputs are AArch64 ELF objects, with no warnings or errors in the captured build log. Source, resolved config, compile flags and output hashes were independently verified.

Reproduce host validation from the source root:

python3 tools/asahi/avd-request-tests.py --mutations --output /tmp/avd-requests
python3 tools/asahi/avd-codec-tail-tests.py --mutations --output /tmp/avd-codecs

The compile command is in tools/asahi/avd-request/README.md. The host mocks do not emulate kernel scheduling, DMA or SMP memory ordering. No target was modified for these results.

For the hardware lead: first build and boot the exact candidate with the device's matching config through the controller, retaining the existing one-shot bootguard fallback and original kernel/module/firmware hashes. Keep the installed firmware unchanged. Source, ownership and resource admission must succeed. Use an approved controller upload to place the existing decode kit at /tmp/avd-kit; neither the private kit nor lab records belong in this PR.

Submit the following shell body through hw_request(capability="execute", device="j616s", mode="exclusive", task=TASK, timeout="30m", args=[SCRIPT]), where SCRIPT is the complete block and TASK is the live qualification task. This is a command for the lead, not an execution performed by this PR. Keep hardware protection active. It opens no audio output. Retain full logs/failures privately and publish only scrubbed summaries.

#!/usr/bin/env bash
# Draft command body for an approved controller execute request; not executed here.
set -euo pipefail
KIT=/tmp/avd-kit
OUT=/tmp/avd-validation
mkdir "$OUT"
mkdir /tmp/bench.lock
trap 'rmdir /tmp/bench.lock' EXIT
(cd "$KIT/samples" && sha256sum -c SHA256SUMS)
v4l2-ctl -d /dev/video0 --list-formats-out > "$OUT/formats.txt"
gst-inspect-1.0 v4l2slh265dec > "$OUT/hevc-element.txt"

validate_row() {
    local codec=$1 res=$2 label=$3 parser element pix w h caps dir
    local sample="$KIT/samples/$codec-$res.mp4"
    dir="$OUT/$label"
    mkdir "$dir"
    case "$codec" in
        hevc) parser=h265parse; element=v4l2slh265dec ;;
        *) parser="${codec}parse"; element="v4l2sl${codec}dec" ;;
    esac
    w=$(ffprobe -v error -select_streams v:0 -show_entries stream=width -of csv=p=0 "$sample")
    h=$(ffprobe -v error -select_streams v:0 -show_entries stream=height -of csv=p=0 "$sample")
    pix=$(ffprobe -v error -select_streams v:0 -show_entries stream=pix_fmt -of csv=p=0 "$sample")
    # These kit geometries have no I420 row padding. Reject other layouts.
    test $((w % 8)) = 0 && test $((h % 2)) = 0
    case "$pix" in
        yuv420p) caps=I420 ;;
        yuv420p10le) caps=I420_10LE ;;
        *) printf 'unhandled reference layout: %s\n' "$pix" >&2; return 1 ;;
    esac
    ffmpeg -v error -threads 1 -i "$sample" -map 0:v:0 -an -sn -dn \
        -vsync 0 -pix_fmt "$pix" -f framemd5 -hash sha256 "$dir/software.sha256"
    timeout 90s gst-launch-1.0 -q filesrc location="$sample" ! qtdemux \
        ! "$parser" ! "$element" ! videoconvert ! "video/x-raw,format=$caps" \
        ! fdsink fd=1 sync=false 2> "$dir/gstreamer.log" \
      | ffmpeg -v error -f rawvideo -pixel_format "$pix" -video_size "${w}x${h}" \
          -i pipe:0 -vsync 0 -f framemd5 -hash sha256 "$dir/hardware.sha256"
    # Compare frame size, hash and count, excluding timestamps and metadata.
    for mode in software hardware; do
        awk -F, '!/^#/ { gsub(/[[:space:]]/, "", $5); gsub(/[[:space:]]/, "", $6); print $5 "," $6 }' \
            "$dir/$mode.sha256" > "$dir/$mode.frames"
    done
    test -s "$dir/software.frames"
    cmp "$dir/software.frames" "$dir/hardware.frames"
    wc -l "$dir/hardware.frames" > "$dir/frame-count.txt"
}

for codec in hevc h264 vp9 av1; do
    for res in 1080 4k; do
        validate_row "$codec" "$res" "$codec-$res"
    done
done

# Independent contexts: both must still produce every reference frame.
validate_row hevc 1080 concurrent-hevc & p1=$!
validate_row vp9 1080 concurrent-vp9 & p2=$!
rc=0
wait "$p1" || rc=$?
wait "$p2" || rc=$?
test "$rc" = 0

# Close a decoder while a timed stream is active, then verify a clean restart.
rc=0
timeout --signal=TERM --kill-after=3s 0.2s gst-launch-1.0 -q \
    filesrc location="$KIT/samples/hevc-1080.mp4" ! qtdemux ! h265parse \
    ! v4l2slh265dec ! fakesink sync=true > "$OUT/teardown.log" 2>&1 || rc=$?
printf '%s\n' "$rc" > "$OUT/teardown.rc"
test "$rc" = 0 || test "$rc" = 124
validate_row h264 1080 recovery-h264
validate_row hevc 1080 restart-hevc

rmdir /tmp/bench.lock
trap - EXIT

# The profiling kit probes the completion helper renamed by this patch.
python3 - "$KIT/profile_avd.py" "$OUT/profile_avd.py" <<'ADAPT'
import pathlib, sys
text = pathlib.Path(sys.argv[1]).read_text()
text = text.replace('apple_avd:avd_job_finish', 'apple_avd:avd_finish_buffers')
loaded = {line.split()[0] for line in pathlib.Path('/proc/modules').read_text().splitlines()}
for module in ('apple_avd', 'videobuf2_dma_contig'):
    if module not in loaded:
        text = text.replace(module + ':', '')
pathlib.Path(sys.argv[2]).write_text(text)
ADAPT
python3 "$OUT/profile_avd.py" "$KIT/samples" "$OUT/throughput" --profile \
    | tee "$OUT/throughput.jsonl"

The correctness pass streams normalized 8/10-bit planar frames into hashes and compares every frame size/hash/count against software decoding of the identical compressed bytes. The geometry checks deliberately reject layouts with unhandled row padding. Concurrent correctness and teardown/restart run outside throughput timing. Unsupported hardware elements fail instead of using software fallback.

After any actual timeout or VP fault, preserve the controller result and kernel fault log, then rerun the validate_row h264 1080 recovery-h264 command in a fresh output directory to check continued decoding. A reset failure requires the device's approved bootguard rollback/recovery operation; do not keep submitting frames. The teardown test does not force a hardware watchdog timeout: that hardware fault/recovery case still needs separate qualification. Compare baseline/candidate with identical samples, count completed frames, keep wall and PLAYING-to-EOS timing separate, and reject lossy traces. The adapter also handles built-in versus module probe symbols. The old trace analyzer cannot assign overlapping host runs to requests, so do not interpret its per-request latencies when it flags ambiguity.

…ership

Let an independent context prepare its next decode request while one context
owns the hardware. Retain explicit buffers and per-context codec state after
releasing the m2m scheduler, route IRQs through the hardware owner, and drain
released ownership before streamoff/free. Keep one hardware owner because PP
completion is untagged, one request per context for VP9/AV1 dependencies, and
the existing H.264 held-capture scheduling contract.

Serialize retirement with watchdog cancellation and reset/IRQ/mailbox drain.
Fail closed after reset failure and fail partial submission explicitly. Include
the empty-mailbox guard from the public fix proposed in #640 so a repeated PP
word cannot retire a new owner. Preserve PM references until retirement and
pause/drain requests across system suspend.

Enable HEVC in T6040 match data with its existing v5 header, requiring firmware
CODECS:h265 support for HEVC while retaining legacy admission of other codecs.
No pixel, firmware-overlap or throughput qualification is claimed.

Add KUnit ownership transitions and host ASan/UBSan tests executing production
decode paths, OF match data and firmware capability parsing with negative
controls. Hardware validation remains a separate step.

Link: #640
Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Do not boot firmware on runtime resume after reset failure has marked the
device broken. Submission admission already rejects this state; keep the
same boundary when an autosuspended device is accessed again.

Execute the production runtime-resume function in the host fixture and
check that broken admission performs no firmware boot, while normal
resume retains its boot path.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide RyanTheTide changed the title media: apple: avd: retain request ownership for CPU pipelining and gate T6040 HEVC on firmware media: apple: avd: enable firmware-gated T6040 HEVC and retain request ownership Oct 31, 2026
A submission error can leave a CPU instruction table referencing buffers
that the codec's error completion then drops. Free and clear that table
before error completion so a following H.264 continuation cannot reuse
stale instructions. Successful held-capture requests keep their table.

Remove the fixture's implicit table cleanup so it exercises production
retirement. Check validation and partial-submission failures, successful
held capture, and a mutation that retains the failed table.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
@RyanTheTide

Copy link
Copy Markdown
Contributor Author

Closing per the one-PR-per-series rule. The branch stays and gets integrated into the AVD rewrite, which lands as one PR once it works on hardware.

@RyanTheTide RyanTheTide closed this Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant