Skip to content

ci: run every host test on each push and pull request (aurora-wip) - #625

Merged
RyanTheTide merged 13 commits into
aurora-wipfrom
ci/host-tests-wip
Oct 30, 2026
Merged

RyanTheTide merged 13 commits into
aurora-wipfrom
ci/host-tests-wip

Conversation

@RyanTheTide

Copy link
Copy Markdown
Contributor

The host tests are a merge gate, but no workflow ran them, so they rotted. Seven were fixed earlier today (#617 to #623). Running the whole suite with every test's real arguments found ten more that no longer built or passed. This PR repairs those ten and adds a workflow that runs everything on each push and pull request to integration/aurora and aurora-wip.

Runner tools/asahi/run-host-tests.py

  • Every tools/asahi test under python3 -I, each with its own output directory, plus its mutation and old-code controls (133 runs in all).
  • A mutation counts as killed only when the built control reports it (assertion, abort or panic). A mutant that just stops compiling fails the run.
  • Also runs the tools/asahi Rust and shell tests and the apple-dcp (80 cases), apple-afkv2 (gcc and clang, ASan/UBSan), asahi-g15 and asahi-m4 selftests.
  • A tools/asahi test that is missing from the runner fails it, so new tests have to be registered.

Workflow .github/workflows/host-tests.yml: ubuntu-24.04, Python 3.14, rustc 1.96.0, clang 18, gcc 13, a treeless full-history checkout for the old-code controls, and a cached Rust toolchain. It fails on any failure and uploads the logs.

Fixture repairs (test-only, no driver changes): bcm4377-resume, bcm4377-id, dma-heap-fd, g15-input-error, compute-preempt, t6030-pmp-graph, avd-av1-tail, mt7932-association-events, neo-stability and apple-afkv2. Each commit names the driver change that broke it. Stale mutations were rewritten against the current code, never dropped, and some gained new ones.

Evidence: on the buildbox (gcc 16, clang 22) the runner passes 133/133 on this branch and on the aurora-wip twin. Before and after logs for each repaired test are kept outside the repo.

apple-dcp ordered-rx (test 29) passes on both tips (f60ee6e, df1e208), and 16 parallel runs under load passed too. 14b5515 (#598) still holds.

🤖 Generated with Claude Code

RyanTheTide and others added 13 commits October 30, 2026 17:53
…4377-resume test

The bcm4377-resume host test stopped compiling after two driver changes.
Commit 0877cd7 ("arm64: apple: enable J713 Bluetooth with scoped
radio DMA") made the hardware variant table name its context register
through BCM4377_BAR2_CONTEXT_ADDR_HI and BCM4388_BAR2_CONTEXT_ADDR_HI.
The test compiles that table unchanged but never had those macros.
Commit afd5117 ("Bluetooth: bcm4377: abort suspend if wake IRQ
arming fails") made bcm4377_suspend() call bcm4377_resume() to undo a
failed arming, through a forward declaration ahead of suspend. The test
pastes suspend before resume, so the call became an implicit declaration
that clashed with the later static definition.

Take every #define the variant table refers to from the driver source,
and take the forward declaration of bcm4377_resume() from the driver as
well, so both follow the code instead of being restated in the fixture.
The controls, assertions and the five mutations are unchanged, and each
mutation is still killed by its control.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The bcm4377-id host test compiles the gate from before commit
8935b9f ("Bluetooth: bcm4377: match the hardware table ID for J493
recovery") against the current hardware tables, and it required the
hardware struct and the variant table to be byte for byte the same in
the old and the current driver. Commit 0877cd7 ("arm64: apple:
enable J713 Bluetooth with scoped radio DMA") added a context register
field to both, so that check failed with any pre-fix ref. The new table
entries also name BCM4377_BAR2_CONTEXT_ADDR_HI and
BCM4388_BAR2_CONTEXT_ADDR_HI, which the test never defined, so even the
positive build stopped compiling.

Keep the exact match for the chip enum, the PCI ID table, the probe
selection and the reset registration, which is what the old gate is
paired with. For the struct and the table, check what the gate reads
instead, the id field and the same chip to ID mapping in both versions.
Take every #define the variant table refers to from the driver source.
All five builds keep their expected results, and with the pre-fix ref the
original, decimal ID, other chip and board bypass gates still abort on
the probe registration check.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dma-heap-fd host test compiles the real dma_heap_ioctl() against a
small user space harness that defines max() but not max3(). Commit
d59676b ("dma-buf: heaps: use max3 in dma_heap_ioctl") came in with
the 7.3 base and changed the ioctl to size its buffer with max3(), so the
positive build stopped compiling with an implicit declaration error.

Define max3() in the harness the same way the kernel composes it from
max(). The bodies under test, the old ref build and all five mutations
are unchanged. With the parent of 30d0aff ("dma-buf: dma-heap:
don't publish fd before copy_to_user() succeeds") as the old ref, the
positive build passes at both page sizes and the original and every
mutation still abort on their assertions.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…test

The test copies the real VmStatus::report_failure body into a host
prelude. Commit f227c76 ("drm/asahi: log the first GPU work failure
reported to each G17 VM") made that body use a warned flag, and commit
769e0b0 ("drm/asahi: g17: log a closed client's teardown failures at
debug level") made it call unobserved_teardown and mod_pr_debug. The
prelude had none of these, so the control no longer built.

Give the host VmStatus the client_closed and warned fields, add ENOENT
and a mod_pr_debug stub next to pr_warn, and extract the real
unobserved_teardown body with the other status methods. The control and
all nine mutations run as before, and the removed-fix control still
reproduces the lost input error with --ref ffe8011^.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… test

The test extracts the priority policy from WorkQueue::new as the text
between the PRIORITY lookup and the WorkQueueInner literal. Commit
ab30807 ("drm/asahi: keep work-queue firmware structures until the
context is invalidated") moved the QueueInfo allocation out of that
literal to just after the policy, so the extracted body pulled in
try_init!, alloc, gpu_context and DebugFlags::VerboseFaults and no
longer built.

End the extracted body at the close of the compute priority override,
which is the last statement of the policy in the current driver and in
the older ones. All four mutations are unchanged and still fail,
and --old-ref c8aed72^ still fails the same assertions.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…h test

The PMP report entry probe gained an apple,irq-safe opt-in that sets
GENPD_FLAG_IRQ_SAFE (540f52f). The host fixture had no such flag,
so the extracted probe stopped compiling.

Define the flag, let the fake OF node carry apple,irq-safe and read it
from the overlay alongside the other entry properties. Check that each
entry gets the ACTIVE_WAKEUP and IRQ_SAFE flags its overlay node asks
for, and add a case where only one entry opts in. Two new mutations
drop the flag and set it without the property, and both are killed.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The AV1 capture tail gained a per-frame segmentation map below the CDFs
(57b0e7c), and the preamble now refuses frames larger than the
capture format and out-of-range primary references (9fb5f79). The
host test did not extract avd_av1_seg_size() and its fixture had no
decoded format or s8, so it stopped compiling.

Extract avd_av1_seg_size() and avd_av1_set_seg(), give the fixture
buffers a DMA address, and give the context a decoded format. The
reservation check now counts the map and places it between the
compressed data and the CDFs, and the preamble check covers the map
address. New cases cover the size and reference index guards, and the
map that set_seg zeroes, copies from a reference with the same MiCols
and MiRows, or leaves for the AVD, with the CPU and DMA offsets agreeing.

Every mutation now names the text it rewrites and is refused if the
driver no longer contains it. Four new ones break the map CPU offset,
the MiCols match, the reference index bound and the frame width guard,
and all six are killed.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The association capability event gained a dynamic debug line for the
generated HT capabilities (b23d29d). It looks the element up with
cfg80211_find_elem(), which the host fixture did not provide, so the
extracted mt_link_event() stopped compiling.

Define WLAN_EID_HT_CAPABILITY, give struct element its real id and
length header, and add a cfg80211_find_elem() that walks only the IEs it
is given. The capability event had no case before, so add one where a
generated HT element is exported and the lookup reads the driver's copy,
and one where a malformed element fails the attempt without any lookup.
The four existing mutations are unchanged and still killed.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eo stability test

Several driver changes left this harness behind. The MT7932 6 GHz
station support (12010e0) made channels band-qualified, added
channels6, PSD and VLP handling to the regulatory notifier and a band
argument to mt7932_policy_permits(), so the harness stopped compiling.
The DWC3 glue gained a runtime PM reference (c8372a0), a host-only
role mark (a50abc4) and a set_role that can fail (3949e36).
DockChannel no longer makes the receive IRQ a wake IRQ and requests it
with IRQF_NO_SUSPEND instead (f98122b), so the wake case and its
mutation tested behavior the driver no longer has.

Give the fixture the nl80211 band, 6 GHz channels with frequency, PSD
and VLP flags, a recording cfg80211 rule lookup and the missing helpers.
The regulatory case now checks band-qualified permission where channel
numbers overlap, that only 6 GHz channels consult the rule, that the
VLP-client channel stays out of the domain and the others lose NO_IR,
that the policy pass lifts NO_IR only under a rule without it, and that
the disable bitmap spans every channel.

The USB case checks that the runtime PM reference is dropped exactly
once per stop and kept while the reset assert fails, that a refused PHY
role stops the core before any host or gadget start, and that only a
host-only host start marks the host role before the core comes up.

The wake case now checks that the device is a wakeup source without a
wake IRQ, and runs the RX IRQ request statement from dockchannel_init()
to check its flags. The wake-irq mutation now restores the old wake IRQ,
and new rx-suspend, usb-rpm, vlp-client and six-band mutations join it.
Every mutation is refused if it no longer matches the driver. All seven
mutations and both old-code controls are killed.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…HPD, sender and EDID tests

Since the HPD reply check (e8f48a6), driver work reached this tree
that the extracted units never followed, so three of them stopped
compiling. The 14.7 external role closes its AV client on an HPD low
(cadaaeb). Compact EPIC on desktop 26.x endpoints (8e82151,
9987082) gave the sender a compact path, the HPD request a status
word reply and the legacy EDID reader a 2 KiB request. The sender gained
a legacy reply-length interface on top of afk_send_command_common()
(e6c1062), and the legacy EDID reader now bounds the EDID by the
reported reply length (48cbf95).

HPD: count dcpavserv_v14_close() calls and check that only an HPD low
closes the AV client, before the request goes out. On compact EPIC
check that a zero status is accepted without an echo and that any
nonzero status is refused.

Sender: extract afk_send_command_common(), the reply-length wrapper and
the compact status allocator. Legacy reply lengths go through the real
completer, and the test checks the bounded copy with zero fill, the
oversized refusal, the U32 overflow and NULL guards, and that AFK-v2
refuses the interface without sending. Compact requests check the
status echo mapping and that no DMA or tag slot is used.

EDID: model the reply-length call and the compact buffer size. The
legacy reader keeps its checksum tolerance but now refuses a truncated
reply, a wrong max_size echo, partial blocks and a short block count.
Its compact request asks for 2 KiB and expects that size echoed.

Add EOVERFLOW and EOPNOTSUPP to the host errno header.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The host tests extract production code from this tree and build it on
the host. Nothing ran them automatically, so a driver change could break
one while the kernel build stayed green, and by 2026-10-30 seventeen of
them had stopped passing.

Add tools/asahi/run-host-tests.py. It runs every tools/asahi test under
python3 -I in its own directory, together with its mutation and old-code
controls, plus the Rust and shell tests there and the apple-dcp, apple-
afkv2 (gcc and clang, with sanitizers), asahi-g15 and asahi-m4
selftests. A mutation only counts as killed when the built control
reports the failure, so a mutant that just stops compiling fails the
run. A tools/asahi test that is not registered in the runner also fails
it, so new tests cannot be skipped by accident.

Add a workflow that runs it on ubuntu-24.04 with pinned Python 3.14,
rustc 1.96.0, clang 18 and gcc 13, for every push and pull request on
integration/aurora and aurora-wip. The checkout keeps the commit history
without trees so the old-code controls can read their revisions.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ut probe

The C layout probe includes the tree's asahi_drm.h, which pulls in
drm_mode.h and so <linux/bits.h>. That header only exists in recent
kernel UAPI exports. On a host with older ones, such as Ubuntu 24.04's
6.8 linux-libc-dev, the probe stopped compiling and test-uapi failed.

Search the tree's include/uapi after the system headers. Headers the
host already has still come from the host, and only the missing ones
come from the tree. The measured layout is unchanged.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first run on ubuntu-24.04 found two gaps in the toolchain setup. The
apple-dcp usb-kconfig case configures the tree with LLVM=1, which needs
ld.lld. The g15-pb-growth and g15-stats-drain tests ask rustup for their
own toolchain unless HOSTRUSTC names a compiler.

Install lld 18 next to clang 18, and point HOSTRUSTC at the pinned rustc.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RyanTheTide
RyanTheTide merged commit ddfca99 into aurora-wip Oct 30, 2026
1 check passed
@RyanTheTide
RyanTheTide deleted the ci/host-tests-wip branch October 30, 2026 07:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant