[DjDeveloperr]

Deep sleep (suspend-to-RAM) for the MacBook Neo (J700, T8140) with Linux at EL2, as 72 commits on integration/aurora. The order is dt-bindings first, then the shared infrastructure, the sleep controller, each subsystem's suspend and wake, and the device tree last. SoC facts live in the DT and in match data wherever shared code would otherwise hard-code the J700, so M4 and later machines need their own tables, not code changes.

Needs from aura

• The EL2 return path, kboot: let an OS at EL2 return from its own sleep aura#117 (open). iBoot locks every core's reset vector to the boot object, so a core that powers off comes back inside aura. aura#117 handles any OS it starts at EL2 where a secondary's locked RVBAR leads back into aura. It sends a reset secondary back through its spin-table release address, sends the boot core to a resume mailbox, and publishes that mailbox as /chosen/aurora,s2r-mailbox. The CPUs keep spin-table as their enable method. The mailbox is the only signal. With it, Linux gives spin-table CPUs the parking ops for power-off and return. Without it nothing in this series binds, the CPUs stay on plain spin-table, and s2idle stays the default.
• The APCIe tables for ANS's wake, Publish APCIe ANS recovery tables with atomic packed-DT export aura#112 (open). ANS's NAND link runs on the shared PCIe block, and a real sleep powers that block off. The sleep controller reads apple,tunable-axi2af, -cio3pll, -clkgen, -phy, -phy-ip-pll and -phy-ip-auspma on the PCIe node, and pcie-apple reads apple,tunable-port, -rc, -rc-gen3 and -rc-gen4 on pci@0,0. Without the PCIe node's tables the sleep controller refuses every real sleep with ANS stopped ("no PCIe tunables for ANS"). aura#112 has to publish all ten.

What a system needs

• apple_sep loaded before the first sleep. A SEP that iBoot started and no driver has bound refuses every sleep ("sep runs with no driver bound"). The aurora-sep package in the Fedora, ALARM and NixOS trees does this. It blacklists the module's alias, and aurora-sep.timer (1 s after boot) starts aurora-sep.service, which runs modprobe apple_sep. Fedora and ALARM enable the timer through a systemd preset that their image builds apply, and NixOS through modules/sep.nix, part of the default module set. A system without that package, or with the service masked, refuses every deep sleep.
• Nothing else. Once the sleep controller binds, deep is the default sleep state, set through the kernel's usual suspend_set_ops() path, so systemctl suspend takes it. The stock mem_sleep_default=s2idle parameter still keeps s2idle. Without aura#117's mailbox the controller logs that once and doesn't bind, so s2idle is the only sleep. Without ANS's APCIe tables every real sleep would refuse, so the controller says so once and leaves s2idle as the default. Distributions need nothing beyond the aurora-sep timer they already ship.

How it works

The boot object opts in with /chosen/aurora,s2r-mailbox and nothing else. CPUs whose enable method is spin-table then get a parking spin table, so the device tree keeps the method every kernel knows, and a kernel without this series ignores the property. The older aurora,rvbar-spin-table method selects the same ops. Until a platform driver registers its power ops, the parked CPUs refuse hotplug as plain spin-table does, and kexec refuses the load up front instead of failing later with the CPUs still online.

• The sleep controller runs macOS's last 14 ms in macOS's order: WD2, the PMU's boot stage, perf states, DCS, memory controller, power words, CLLT, DVFM, AMCC error enables, complex words, the commit, AIC masked. On the way back it restores every power word to its value at the commit, before any driver resumes. The cluster power-gate enable, set at boot, is what lets the boot CPU power-gate at all. Without it the sleep engine never started, and the watchdog reset the SoC after 5 s.
• Refusals come before anything stops. Both DCPs, the PMP and SIO must have stopped, and an enclave that runs unbound refuses. The driver doesn't read a display CPU whose domain is off, and it lowers or refuses on what XNU panics on at the last step (AVD, VENC0/1, GFX, JPG). In .enter the display goes off first, before the GPU, as on macOS. Then come each coprocessor's last check, such as NVMe's shutdown notification, and then the enclave's sleep request. That request is the very last refusal, because a gated enclave comes back only through a real sleep. After it nothing can refuse, and from the GPU's stop on, a failure goes on to the commit.
• No commit with a CPU awake. Right after the first checks, before the display goes off, every parked CPU answers a hold and waits. Nothing is written to the hardware yet. The sleep refuses right there, before anything that can't be undone, if a CPU doesn't answer within 100 ms, or if a CPU is offline but neither held nor powered off, like one whose start failed after a wake. Every CPU then comes back as after any refusal, and a later refusal releases the hold the same way. At the commit only held CPUs stop, at the same point as before. Without this, measured on a J700 with a test switch, a core that missed its stop kept the sleep engine waiting until the watchdog reset the SoC.
• Every coprocessor macOS stops is stopped the same way and brought back with a warm restart, with firmware and rings kept in DRAM. That's DCP and DCPEXT, PMP, SIO, GFX/GFX1, ANS and the SEP. The SMC, MTP and AOP keep running and get the AP power notices.
  • An idled firmware doesn't wake with its power kept, so the PMP's and the GPU's idle and stop run in the sleep's last step, after every refusal.
  • After a refused sleep, the DCPs come back from a power cycle, with DCPEXT's DISPEXT0 raised first if the sleep had lowered it. SIO comes back from the sleep engine's own reset of its domain.
  • If a DCP's stop fails after its CPU stopped, the driver starts it again before returning the error. A crashed DCP no longer blocks every later sleep. The driver stops it cold and the wake's power cycle restarts it, which is how macOS recovers one.
• OSLOG counters are echoed for every T8140 RTKit coprocessor except the GPU, as macOS does. A firmware with one unanswered counter takes no AP power notice. The polled idle in the last step echoes them too.
• ANS comes back from power-off. The sleep controller lowers its PCIe block with its domains and brings it back in macOS's order with the PHY's PLL and lane tables. It puts the 212 AP pins back and writes the SART again right before ANS's run bit.
• Radios stay bound and powered. The radio's link goes through L2 and PERST# and is retrained at the wake.
  • Wi-Fi keeps its firmware (CID 0xf6 suspend/resume), and can stay associated and wake on a magic packet.
  • Bluetooth enters the controller's host sleep as macOS does, keeps page-scanning, and wakes the SoC through the SMC. The driver takes the wake event before HCI resumes, and logs it.
  • A radio that didn't survive restarts from cold.
• The J700's USB host sleeps in place. The core goes through hibernation, and the PIPE is parked at noirq rather than USB3 being taken down.
  • The core has come back without its registers after every sleep so far, so the glue sets it up again in place and xHCI restarts in place. The hub and keyboard reset-resume and keep their input devices.
  • The wake sets the PHY up again in its old mode, USB2 or USB3, and the core's re-init routes the PIPE to it before xHCI starts. SuperSpeed needs no mode change or port cycle afterwards.
  • The glue checks the fixed hub after every sleep.
  • The port controller arms USB wake only when a device behind its connector can wake, meaning wakeup is enabled at or below the fixed hub port that the connector's DT graph leads to.
• A lid closed during a sleep is a dark wake, as on macOS. The whole system resumes with the panel and DCPEXT's output kept dark, then goes straight back to sleep. The lid and power button stay armed through the dark wake, so opening the lid or pressing the button ends it.
• An attached USB-C display is released for the sleep. The pipe goes off, HPD goes low and the crossbar output is parked, and the wake sets the DP mode up again. The Type-C route and the DRM connector stay, so the compositor keeps its output.
• Wake sources are the RTC alarm, USB, the SMC (power button, lid, AC, Wi-Fi, Bluetooth, USB keyboard) and the MTP (internal keyboard and trackpad).
  • The SMC gets its sleep warning with no fixed wait after it.
  • The debug link is cut before the SMC's AP idle notice, so its events don't wake the SoC.
  • Opening the lid or pressing a key while the machine goes down ends the suspend.
  • The log names the SMC's wake reason.
• Smaller pieces.
  • The ISP's firmware stops the way macOS stops it, and a stream that ran across the sleep restarts once tasks thaw.
  • The mic array's power rail, SMC pcIO output 25 through the existing apple,pcio-templates, switches on and off around each capture.
  • DART tunables and address filters come back after a power-off. DARTs keep the normal sleep phase on every other SoC and for every other kind of sleep.
  • The system level cache is written back before the sleep.
  • The Touch ID sensor gets its sleep command and is powered off for a system sleep. See Testing.

Fitting into integration/aurora

• t8140-peripherals.dtsi already describes the SIO, its DART and its mailbox, so the series only names the SIO's register blocks (asc, auxiliary, asc-tail) for the sleep controller.
• The placeholder for the DCP's exclave mailbox (apple,t8140-secure-mailbox) becomes the secure RTBuddy port that the DCP driver polls (apple,t8140-secure-rtbuddy-proxy).
• The J700's mic array rail uses the SMC GPIO driver's apple,pcio-templates, so the GPIO driver needs no change for it.
• macaudio's sleep handling stays as its speaker protection lease has it.
• RTKit's wait during a sleep uses the per-instance handshake timeout.

Measured on a J700

These numbers come from test kernels carrying this code. Those kernels also carried test-only switches and debug code, and the later ones two MTP commits that aren't in this series.

• 30 of 30 real deep sleeps over two boots, with ANS stopped and restarted each time. With Plasma up and USB down, 10 of 10, and 30 of 30 with the panel DPMS'd off before every sleep.
• Wakes from the RTC alarm, the power button, the internal keyboard and trackpad, a USB keyboard, a Bluetooth keyboard, the lid, and a Wi-Fi magic packet, still associated after each wake.
• A key or the lid during the entry brings it straight back. An external display is released and restored.
• On battery with nothing plugged in, 16 of 16 systemd suspends returned, 15 on the RTC alarm and 1 on a magic packet. Bluetooth, USB, NVMe, audio and the panel were back each time, with no crash lines. Entry took 0.25 s median (0.22-0.26), and the wake to "PM: suspend exit" 1.02 s median (0.99-1.04).
• With the USB changes, 12 of 12 systemd sleeps and 3 of 3 magic-packet wakes. The SuperSpeed side was already in USB3 at every wake. With nothing on the ports USB wake isn't armed, and a USB keyboard wakes it.
• With the review fixes below, 8 of 8 sleeps, including 2 of 2 magic-packet wakes, and every device check passed.
  • A forced refusal at the latest point a sleep can still refuse, after the display and the last checks and before the enclave, returned EBUSY. The enclave was never told.
  • The GPU, the SEP with its keystore open, both DCPs, SIO, the AOP and NVMe at 1.4 GB/s all came back, and a real sleep on the same boot then worked.
• From the SMC's charge counter on battery, a 300 s sleep averaged 11.3 mA, 0.049 W at 4.375 V, entry and wake included. A fit over four sleeps gives about 8.6 mA asleep. macOS under aura measured 0.109 W on this machine with the same counter.

Not in this series

• Test and debug code: the wake write log, PMU scratch-byte reporting, power-word and register dumps, the no-sleep restart tests and their helpers, every sysfs or debugfs test hook, wake_reason in sysfs, the SMC boot-record log, PM_DEBUG in asahi.config, timing lines, and the test-only switches keep_debug_link, refuse_late and ignore_stop_cpu.
• The suspend_again dark wake experiment.
• The sleep-DDR vote for the USB host. macOS uses it, but from Linux it resets the machine at the wake.
• The MTP's in-place CPU reset and rtkit-helper's own MTP start and stop. They wait for their boot-loader side.

Open

• A core that answers the hold and then doesn't take its stop at the commit would still keep the sleep engine waiting until the watchdog resets the SoC. To get there it would have to stop running a WFE loop with interrupts masked. macOS doesn't handle that case either.
• The sleep controller still keeps T8140 offsets in the driver: power-word lists, domain offsets, CLLT, CPM and mux words. Moving them to the DT and a per-SoC table is the next cleanup, and it needs its own hardware run.
• USB's host loses its registers in every sleep, so the hub and devices reset-resume after each wake.
• The sleep controller shares its unit address with pmgr, and W=1 dtbs warns about it. Moving it means reordering its binding's ranges and the tree together, as a separate change.
• With the PMP built as a module, removing it leaves every later sleep refused until a reboot. The built-in default has no unbind on T8140.
• The DCP's secure port has no interrupts in its binding, so the driver polls it, quickly only around its messages.
• The GPU's OSLOG counters stay unechoed until a soak shows echoing them is safe.
• The ATC PHY doesn't acknowledge the PIPE lock at most sleep entries, logs it, and parks the PIPE unlocked. USB came back after every one of those sleeps.
• One ping right after the sleep that followed the forced refusal went unanswered. The Wi-Fi driver logged nothing, NetworkManager's dispatcher ran at about that moment, and Wi-Fi was fine a minute later. I haven't chased it further.

Review

Two reviews of an earlier version found 25 issues. 24 are fixed in code, each folded into the commit it belongs to or added where it's new. The 25th, the DockChannel host-state order, was right, and only its comment was wrong. The fixes:

• The display goes off and NVMe's shutdown notification runs before the GPU's last stop. The enclave's request is the last refusal, and last stops can't refuse.
• DARTs keep the normal sleep phase except in a firmware-driven sleep. The PCIe-C reset table has its earlier value back. Coprocessor iommus are parsed by #iommu-cells.
• Only the debug port's controller answers the debug link cut. The sleep driver publishes itself only once its probe can't fail.
• DCP raises DISPEXT0 before a refused cycle, and restarts the firmware when a stop fails after the CPU stopped. A crashed DCP sleeps and restarts, a refused idle is undone, and DCPEXT stays down through a dark wake.
• An ISP stream across a deep sleep restarts, and T8140's cold STOP is its own command.
• A radio host that a failed wake left down keeps its saved state. The Wi-Fi module is held while its radio stop runs.
• An AOP call checks for the sleep again once it has its turn.
• The PMP has no sysfs unbind on T8140.
• A core still parked after the wake starts at its reset vector. The hold now catches a core that misses its stop before the commit, so this path shouldn't run any more.
• The display power words got a binding and their own address.

Testing

• Build. The tip builds Image modules dtbs with the asahi profile, Rust on and no compiler warnings, on a build guest that verified clean before and after. Wherever this rebase changed a commit's code, or a later commit changes that code again, I also compiled the changed objects at that commit with W=1 and the tip's config. That's #16, #24, #30, #31, #33, #41, #48, #50, #54, #56, #57, #58, #62, and the J700 DTB at #70. All pass. The one W=1 note is t8140_sleep_lower_io() unused at #30 and #31, until its caller arrives in #32. Making deep the default touched #30 and #32 again, and t8140-sleep.o compiles at both with W=1, with that same note only.
• checkpatch --strict. Nothing new against the series before the rebase. It reports 29 errors, 16 warnings and 7 checks, all of known kinds: a minified host-test fixture, MAINTAINERS reminders on new files, udelay in atomic sections, arm64-only includes in the spin table, and a false positive on function-pointer locals.
• Host tests and DT checks. tools/asahi/run-host-tests.py passes 195 of 195 at the tip, and dt_binding_check and the DTB schema checks pass. Where a commit changes code that a host test compiles from source (the DCP, ATC and dwc3 slices of the apple-dcp selftests, the PMP bridge in display26-tests, mt7932-association-events-tests and rtkit-oslog-tests), the same commit updates that test's stubs. Each of those suites also passes at the commit that changes it.
• KUnit. The ATC PHY's suites passed under QEMU, 39 of 39, before the rebase. The base doesn't change that driver.
• Bluetooth. This base builds mt7932_bt_pcie again, so the series' Bluetooth changes build here too.
• Touch ID sensor. Its sleep commit does nothing without the sensor, and it hasn't run on hardware. Its KUnit suite compiles with KUnit built in, and its cases pass in a host build of the shim's code.
• Hardware. This tree, plus three test-only switches that do nothing unless set, on a J700 booted through aura with aura#117 and all ten APCIe tables, apple_sep loaded and deep selected:
  • 6 of 6 sleeps on a 20 s RTC alarm, with keep_debug_link set so the debug cable's events didn't wake it. Each lasted the full 21 s and woke on the PMU (NUB.SPMI0.SW3), with an empty SMC wake record. Entry took 0.41-1.10 s and resume 1.14-1.17 s. After every wake Wi-Fi pinged, Bluetooth was powered, the USB hub and keyboard were back, NVMe was live, both audio cards were there and eDP was connected, with no crash lines.
  • With one CPU set to ignore the hold, the sleep refused with -ETIMEDOUT ("CPU1 did not answer the hold for a suspend") before the display went off. CPUs 0-5 were online after, and the next plain sleep returned on the RTC.
  • With a refusal forced at the latest point, the sleep returned -EBUSY. The PMP, SIO, both DCPs (DCPEXT's DISPEXT0 raised for its cycle) and the GPU came back, the device checks passed with CPUs 0-5 online, and the next plain sleep returned. One ping right after that wake went unanswered; Wi-Fi was fine a minute later.
  • On battery with nothing plugged in, the debug cable out and no test switch set, over ssh on Wi-Fi: 2 of 2 sleeps on a 60 s RTC alarm returned after 61 and 60 s. Entry took 0.21-0.23 s and resume 0.89 s, and every device check passed.
  • Those runs selected deep by hand. The change that makes deep the default came after them. On a boot of that tree with no mem_sleep_default on the command line, /sys/power/mem_sleep read s2idle [deep] untouched, and the controller logged "deep sleep is the default". One plain systemctl suspend on a 30 s RTC alarm entered "PM: suspend entry (deep)" and returned 30.4 s later, woken by the PMU with an empty SMC wake record.
  • After the rebase onto 009c87801b, which brings in upstream's G17 GPU changes, the rebased tree with the same three switches booted on stage2-14. mem_sleep read s2idle [deep] with no mem_sleep_default on the command line, and apple_sep was bound. 6 of 6 sleeps on a 20 s RTC alarm, with keep_debug_link set, returned after 20-21 s, woken by the PMU with an empty SMC wake record. Entry took 0.88-1.14 s and resume 0.86-0.88 s. The GPU firmware restarted at every wake with no fault lines, and every device check passed: Wi-Fi ping, Bluetooth, the USB hub, NVMe, both audio cards and eDP, with no crash lines.
  • A key-press wake from the internal keyboard on this kernel was inconclusive, because the timing of the press against the 300 s backup alarm isn't known. Keyboard wake passed on earlier kernels with this code; a timed row on this kernel will follow.

🤖 Generated with Claude Code

[DjDeveloperr]

Superseded by #1445 (merged as 2b05915); its changes are in integration/aurora. #1445's d9b438c carries this stack's final diff. The sleep driver is now drivers/soc/apple/sleep.c (#1334 renamed t8140-sleep.c).