Skip to content

soc: apple: suspend-to-RAM on the M4 Max (T6041, J616c) - #1664

Draft
AxxzyWasTaken wants to merge 33 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:t6041-s2r
Draft

AxxzyWasTaken wants to merge 33 commits into
aurora-silicon:integration/aurorafrom
AxxzyWasTaken:t6041-s2r

Conversation

@AxxzyWasTaken

@AxxzyWasTaken AxxzyWasTaken commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Real suspend-to-RAM on the MacBook Pro 16" M4 Max (T6041, J616c) under Linux at EL2: entry, wake and a full resume of the running system, from the lid or systemctl suspend. It uses the sleep core from #1445; the T6041 side has its own platform driver (t6041-sleep.c), because the M4 Max commits through aura's S2R mailbox rather than the T8140 path.

Draft: the sleep works on my machine and the bring-up diagnostics are gone; see "Not working" for what is left.

Tested on

One machine: J616c, 26.6.2 firmware, booted at EL2 through aura.

This series, on #1445 as it was on 11 Nov (b8f5b1e7f750), 12 Nov:

  • 3 deep sleeps in a row in one boot (RTC alarm, 20 s each), each one back on its own: Wi-Fi (the SSH session I ran the test from stayed up through all three), NVMe, Bluetooth, no GPU errors. Kernel return path 16 ms.
  • Earlier versions of the same series, before the SPMI fix below: 4 more wakes (the RTC driver had to be rebound between them).
  • Rebased onto current integration/aurora (9219565939723cd5) for this PR; it rebased without conflicts, and two new AOP configs (T8152, T8142) needed the new ed0p field. With the J614C fixes and the cleanup below, 12 Nov evening: 3 deep sleeps in a row (RTC alarm, 20 s each), each one back on its own: Wi-Fi with the SSH session up, NVMe, Bluetooth, suspend_stats 3 ok and 0 failed. With the display gate fix below as well: the internal panel on appledrm from boot, then a lid close and open, deep sleep, and the picture back after the wake.

The same T6041 code on my older bring-up tree (7.3 daily kernel, 10 to 12 Nov):

  • 7 sleeps in a row, lid close and open, key wake, with a Hyprland session open.
  • Back after a wake: internal panel and backlight, GPU (firmware warm restart), NVMe, Wi-Fi, SD reader, Bluetooth (the driver rebinds itself), login and logout.
  • Timing: about 1.7 s from suspend entry to the sleep commit; panel lit about 0.3 to 0.5 s after the kernel gets control back.
  • Power: one unplugged deep sleep of 8 h 27 min (12 Nov, lid closed, then a lid wake) cost 7 mAh on the battery gauge (7124 to 7117 mAh of 7517, 95% before and after). That is close to what the gauge can resolve, so I'd read it as "well under 1% of the battery over a working day" rather than a watt figure.

Found while porting onto #1445

  • Parked CPUs: Apple SoC suspend-to-RAM on T8140 #1445's spin table only stops a parked CPU that answered spin_table_hold_parked() first. The T6041 commit called spin_table_stop_parked() without the hold, so the other cores stayed on and the watchdog reset the SoC 5 s after the commit. The T6041 enter now asks for the hold while it can still refuse.
  • SPMI replies at the wake: aura writes the PMU on the same SPMI queue at the wake, and its reply was still waiting when Linux came back. The RTC's acknowledge of the alarm took it as its own ("unexpected SPMI response 0x8b30"), and the RTC then refused every later suspend. spmi-apple-controller now drops replies left over at a firmware-driven resume, and the T6041 driver drains the FIFO around its own PMU writes. The SPMI change applies to any firmware-driven suspend, so T8140 too; it would be good to check it there.

Found on a J614C (joske's test, 12 Nov)

  • SEP DART sleep lists: with the in-tree device tree the suspend hung in the noirq phase. The SEP is told to sleep at device suspend, and its DART is saved first only if the DART's node has apple,sleep-restore; the J616C's bring-up tree has the lists, the in-tree one didn't. Without them the DART's noirq suspend read its registers after the SEP's domain went off. t6041.dtsi now has the lists (the same writes as T8140's), and the SEP refuses that sleep with a message instead of hanging when they're missing.
  • 40-core GPU: CONFIG_25G83 in hw/t6041.rs had no sleep_restart, so the GPU refused every sleep there; it has it now.
  • The J614C and J616C trees now have the apple,t6041-sleep node, and the driver doesn't bind without aura's S2R mailbox (before, it set its suspend ops and refused every sleep).
  • Booted on my J616C (the sleeps above), which already slept with the SEP DART lists; not yet run on a J614C.

Found on the J616C with integration/aurora

  • Display gate with the bring-up EL2 tree: that tree gives the external chains' DARTs, mailbox and DCP power-domains = <&dispextN_fe>, disabled like the PMP report entries above it; the in-tree trees describe no DISPEXT domain. The M4 Max gate enabled the chain nodes only, so they waited for domain providers that never came, failed at the deferred probe timeout (the mailboxes with -110), and the display subsystem never bound: the panel stayed on simpledrm. soc: apple: display-gate: M4 Max: chains from a tree that describes DISPEXT drops the property where the domain stays disabled; nothing changes with the in-tree trees.
  • Freeze at logout after a wake: logging out, or restarting sddm, after a deep sleep stopped the machine without a word in the log. Breadcrumbs in PMU scratch named the last access: a TLB flush on iommu@3a2300000, the HDMI chain's display DART. That chain's DCP fails its probe here, so it is not bound to the DCP driver and the sleep's stop never marked its DART dark, while the power word step powers the chain off. drm/apple: T6041 S2R: the external chains' DARTs dark by node marks both external chains' display and DCP DARTs dark by node before their power words go down. After it: 2 deep sleeps, each followed by an sddm restart, in one boot, with the machine and the panel fine. Then on my daily build of this series (latest integration/aurora + this PR + soc: apple: clpc, drm/asahi: g17: busy-share memory floor and light effort for T6041 #1369), a lid close and open followed by a desktop logout: fine.
  • With the external DCPs bound, dcp_platform_suspend() warns twice (workqueue.c:4409, flush_work and cancel_delayed_work_sync on work that was never set up), apparently for the external DCP that is skipped as unusable. Harmless. That code isn't from this series, the gate fix only makes it reachable here; I left it alone.

Not working / not tested

  • External displays: I have no external monitor to test with. The current code shuts the external display controllers down at the first sleep and they stay down until a reboot (as J616s (M4 Pro) at EL2: hibernation and suspend-to-RAM #1477 does).
    @astaff, you are the other person doing M4 Max work: if you have a display and want to try or fix this part, that would be very welcome.

  • Touch ID after a wake, bring-up tree: works (a lid close and open, then a fingerprint unlock at the lock screen). A deep sleep is refused, cleanly, while the enclave is not confirmed asleep (soc: apple: sep: refuse a deep sleep at once over a stuck enclave).

  • Speakers after a wake: the amplifiers keep their configuration and report no fault; playback not checked by ear.

  • Camera: its firmware is stopped for the sleep and is unavailable until the next boot.

  • Touch ID after a wake on this tree: works (a lid close and open, then an unlock at the lock screen). One earlier try failed 5 times and locked Touch ID; the integration driver keeps that lockout across restarts until SEP_BIO_REARM gets the password, which nothing on my install calls yet.

  • pm_test=platform and the display: the display stop assumes a real sleep follows. Under pm_test the firmware is stopped but never power-cycled, the restart expects a power cycle, and the internal panel stays dark (joske, J614c). Not supported yet.

  • Wi-Fi WoWLAN: with WoWLAN enabled, brcmfmac refuses deep with -95 (brcmf_pcie_pm_enter_D3) and the suspend is aborted; WoWLAN is s2idle only. With deep as the default, that is worth knowing (joske).

  • More than two external DCPs: the display stop used to handle only the first two external DCPs; drm/apple: T6041 S2R: every external DCP, not only the first two handles every enabled apple,t6041-dcpext. Build-tested only.

Notes for review

  • Boot requirements: EL2 through aura (it passes the S2R mailbox). The in-tree J614C and J616C trees now have the apple,t6041-sleep node and the SEP DART lists; I have only slept with the bring-up EL2 DTB on the J616C so far.
  • T6041's SEP gets SLEEP at device suspend, seconds before the commit, which is the order that has been tested on this machine. On T8140, Apple SoC suspend-to-RAM on T8140 #1445's commit-time SLEEP is unchanged; sep_pm_platform_commit() picks the path.
  • Overlap with J616s (M4 Pro) at EL2: hibernation and suspend-to-RAM #1477: the AIC save/restore, NVMe sleep flags, PMP SRAM backup, SMC WKTP/pmVC and conventional PCIe port re-setup commits do the same thing as J616s (M4 Pro) at EL2: hibernation and suspend-to-RAM #1477. Whichever lands first, the other can drop them.
  • Outside T6041-only code: spmi: apple: drop replies left by firmware at a firmware-driven resume (above), pinctrl: apple: ack a level interrupt again after unmasking it (all Apple SoCs; it stopped an interrupt storm from the speaker amps after a wake here), PCI: apple, nvme-apple: resume asynchronously, and the coprocessor stop helpers in sleep.c.
  • The bring-up diagnostics are gone: the last 3 commits drop them from t6041-sleep.c (the dry, diag and serror_go parameters, PMU scratch marks and notes, register snapshots), the DCP driver (the s2r_* parameters, the step-by-step test hook, S2RSTOP/S2RSTART prints, the panic dumper) and apple_dart_display_keep() (its register dump). The commit and wake do the same hardware writes, delays and polls as before; the defaults the parameters had are now fixed. They are separate commits so the change is easy to review, and can be folded into the commits they clean up. The 4 BRING-UP ONLY commits are dropped.

… its own

The T6041 sleep driver runs its own commit and needs the coprocessor
stop the T8140 path does: every ready coprocessor's last stop and its
run bit off, those registered first before the rest. Export that as
apple_sleep_coprocs_stop_all(), with apple_sleep_coprocs_ans_stopped()
to tell whether ANS (and so the storage PCIe link) was stopped and
apple_sleep_coprocs_count() for the driver's own report.

Unlike apple_sleep_coprocs_stop(), a negative return of a last stop
ends the stop, as the T6041 driver expects.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
A system sleep driver for T6041 with the kernel at EL2 and no monitor
below, binding to an "apple,t6041-sleep" node. It runs macOS 26.6.2's
last steps before the S2R commit in macOS's order and with its values,
as traced from macOS sleeping under aura's hypervisor, commits through
aura's S2R mailbox, and on the wake restores what the sleep took:

- power-state words compared with and restored toward macOS's values at
  its commit, parents before children (t6041-ps-order.h, generated
  from t6040-pmgr's power-domains);
- ATC, AXI2AF, MCC/AMCC and AVE power-down steps;
- the ANS storage-PCIe tunables reapplied on the wake
  (t6041-apcie-tunables.h, from the J616C ADT /arm-io/apcie0);
- the coprocessors stopped through the sleep core's
  apple_sleep_coprocs_stop_all().

t6041-macos-regs.h holds the register values macOS has at its S2R
commit (generated from a trace of macOS 26.6.2). The driver needs the
kernel at EL2 (CTXR window) and the "aurora,rvbar-spin-table" CPU
enable method, so the other cores can leave the kernel.

The driver still carries its bring-up diagnostics (dry and diag
parameters, PMU scratch notes, register compare snapshots); the hooks
that feed it from the PM core and the SError handler are separate
commits at the end of this series.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…uspend

The T6041 (M4 Max) has the AICv3 of T8122 and now a suspend-to-RAM
driver of its own. Give "apple,t6041-aic3" its own info, the T8122 one
with fw_suspend, so the AIC state the sleep loses is saved and put
back as on T8140. Nothing else in the driver looks at fw_suspend.

Also in aurora-silicon#1477 (T6040 sleep), which admits apple,t6041-aic3 the same
way; drop this one when that lands.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
apple_nvme_t6040_hw is only matched by "apple,t6041-nvme-ans2". Give it
the sleep fields T8140 uses: register ANS with the sleep core
(sleep_traced), flush namespace 1 only, the OS's own, and send the
full shutdown notification at suspend. The T6041 sleep driver stops
ANS through the sleep core's coprocessor stop.

On T6041 the sleep core's last check (the shutdown notification has
completed) does not run: the T6041 driver calls only the coprocessor
stop.

Also in aurora-silicon#1477, which sets the same fields with sleep_untraced on the
same struct; drop this one when that lands.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The PMP's SRAM loses its contents in a real sleep. On the J616C's
26.6.2 firmware its __DATA segment is at 0x500761000, 0x7d000 bytes:
back up (0x61000, 0x7d000) of the SRAM, as T8140 does for its own.

Also in aurora-silicon#1477, whose HW_CFG_T6041 has the same range; drop this one
when that lands.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
On apple,t6041, write the keys macOS writes last before MBSE slpa in a
trace of its sleep: WKTP = 2 (AppleSMCEmbedded, going to sleep), then
pmVC. The value written is 0x101, as the bring-up kernel runs it; the
comment, taken over from it, says the trace has 0x100. That difference
is open.

Also in aurora-silicon#1477, which writes WKTP through a sleep_wktp parameter;
drop this one when that lands.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Give T6041 a configuration of its own, T6040's with:

- the AFK sleep and wake order macOS 26.6.2 uses on the J616C (trace,
  8 Nov 2026): sleep to 0x2b, 0x2a, 0x24, 0x20, 0x27, 0x26, 0x28, 0x25,
  0x21, 0x29, 0x23, 0x22, then the AP idle notice, and wake in the
  reverse order;
- the ED0p boot argument at ec0p + 0x20000, where the 27.2 firmware
  adds a segment, and that segment accepted at its translated IOVA by
  the memory-region check.

Other SoCs keep their configuration; ED0p stays the firmware's own
value there.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
A conventional host (internal ports with the radio and the SD reader)
loses its power in a real sleep: the endpoints sit unpowered or in
reset behind a link that is down. On a firmware suspend, save what
only the port registers hold; on the noirq resume, note which ports
lost their link and keep a copy of their functions saved state before
the PCI core spends it on the down link; in the normal resume phase
(after the SMC, which owns the PWREN GPIOs) bring each such port up
again as probe does, then restore its functions.

Only for hosts that are not tunneled, have no managed SIDs, no PIO DMA
supplier and more than a root bus; s2r_conventional turns it off. The
suspend-to-idle paths (domain hold, adopted links) are unchanged.

Also in aurora-silicon#1477 (0ab1a66), which this follows; drop this one when that
lands.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Let the T6041 25G83 profile (the J616C) restart the GPU after a real
sleep, as T8140 does: sleep_restart is set for it only.

Find the GPU's power-state word through a PMP report entry too: where
the GPU's first power domain is an "apple,t6000-pmp-v2-report-entry"
node, as in the J616C device tree the bring-up runs, the word is that
entry's own power domain. With a first power domain that is the word
itself, as this tree's t6041.dtsi has (ps_gfx), nothing changes.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
A display DART is locked, so the driver neither saves nor restores its
TCR/TTBR. After a real sleep the panel stayed black with everything
else up. apple_dart_display_keep() saves the TCR/TTBR of the given
streams and the whole register block before the sleep; after it, it
logs which register words differ and writes back the TCR/TTBR values
that changed.

Bring-up, test code, not for upstream as is.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Add dcp_t6041_s2r_all(), called from the platform suspend (firmware
sleep only) and resume, to stop every DCP before a real sleep and
restart and recommit it after, with tunable settle/stop/down waits as
module parameters, step marks kept in nvmem, the display DART state
kept via apple_dart_display_keep(), and a panic kmsg dumper.

After the restart, dcp_backlight_s2r_restore() clears a request left
in flight by the stop and sends the user's level again, since the
backlight class device may resume before or after the restart.

For the 26.6 desktop firmware, add A500 and callbacks 583 and 597
(purpose unknown, answered with zero).

Bring-up, not for upstream as is.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Three changes to the T6041 display stop and restart:

- At the first stop of an external pipe, mark its display DART dark
  for good while its block is still on. The stop forces the external
  display domains off and nothing raises them again, so a later
  framebuffer unmap flushed a DART behind a dead block and the CPU
  doing it never returned (login or sddm restart after a wake).
- At restart, set bit 0 of bridge 4's two words only with the new
  s2r_bridge_arm parameter (default off). Boot leaves the bit clear,
  and with it set the driver's display power-off after a wake froze
  the machine. Print the bridge words at restart.
- Keep the internal pipe's GAPF word as boot left it, and write it
  back after the restart only with the new s2r_gapf_restore parameter
  (default off): writing it back after a wake froze the machine.

Bring-up, not for upstream as is.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Lower the default settle/words/stop/down waits of the T6041 display
stop and restart from 1000/500/2000/1000 ms to 200/100/300/100 ms,
the values used in the 11 Nov wake runs. They stay module
parameters.

Bring-up, not for upstream as is.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
When the resume comes via firmware and no DMA window is set, the
function lost its power and its firmware is gone; every resume command
would only time out before the rebind anyway. Skip them and queue the
rebind directly, so probe boots the firmware from scratch.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
After a real sleep, setting the PCIe ports up again takes about
0.75 s, and the NVMe resume waits 0.2 to 0.9 s on its coprocessor.
Enable async suspend/resume on both platform devices so they do not
hold up the other devices.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
On T6041 a level interrupt fired again at once every time it was
unmasked although its line was inactive: the J616c speaker amps on
pin 55 (mode 3, data 1) raised about 140 interrupts a second from boot
until the core disabled the line as unhandled. Switching the pin's
mode back from "off" leaves its status bit set. Clear it once more in
the unmask of a level interrupt; a level that really is active sets it
again.

Untested bring-up: not yet run on hardware.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Refuse a firmware suspend in sleep_enter() with one line, -EBUSY, when
the enclave's mailbox stays full for 100 ms: it isn't reading, and the
commit's own check would only find out after its send wait with
interrupts off. A Lost enclave (one that did not come back from a sleep
earlier this boot) is refused with -EBUSY and a log line too, instead
of a silent -EIO. Mailbox gains send_ready().

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…it sends SLEEP

The sleep core stages the enclave's SLEEP for its commit, which sends
it with interrupts off; only the T8140 commit does that. T6041 has a
sleep driver of its own and no such commit, so the staged request was
refused there, and a commit-time send the T6041 driver ran itself made
the J616C reset in its first test.

Where apple_sleep_commit_stops_coprocs() is false on a firmware
suspend, put the enclave to sleep as the T6041 bring-up kernel does,
an order that has passed many sleeps on the J616C: at device suspend,
seconds before the commit, save its DART, send SLEEP (mode byte 0x02,
as op_sleep() has it), wait for the answer, hold its receive side,
and let suspend_noirq wait up to 5 s for it to gate its domain. On a
refused sleep, an enclave that gated stays asleep until the next real
sleep, and one that didn't is asked whether it still answers.

A deep sleep is refused at once, -EBUSY with one line, over an enclave
that did not answer a sleep request or a wake earlier this boot, that
did not finish coming up, or whose mailbox stays full: a commit over a
running enclave resets the SoC.

T8140 keeps the commit-time SLEEP unchanged.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The probe line and the header comment still said "test, entry only"
and "no resume yet". The driver resumes: the commit writes cpu_resume
into aura's S2R mailbox, and the wake comes back through it. No code
change.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The spin table only stops a parked CPU that answered a hold first:
smp_rvbar_spin_table_cpu_die() leaves the park for the stop only while
spin_table_hold_parked() has asked for it. The T6041 commit called
spin_table_stop_parked() without the hold, so every other core stayed
awake in the park, with its power state still on. The sleep engine then
waited for the cores, and the watchdog reset the SoC about 5 s after
the commit.

Ask for the hold in .enter while the suspend can still be refused, as
the sleep core does for T8140, and refuse the sleep if a parked CPU
does not answer.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The driver writes the PMU over the SPMI controller by itself and gave
each command's reply a fixed 50 us before draining the FIFO. A reply
that arrived later stayed there. The controller keeps its power
through the sleep, so after the wake the SPMI driver's next command,
the RTC's acknowledge of the alarm that woke the machine, read that
reply ("unexpected SPMI response 0x8b30"), the RTC marked its I/O as
failed and refused every later suspend.

Poll for the reply for up to 1 ms, then drain the FIFO.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
While Linux is away the queue keeps being used: aura's wake path and
the platform's sleep code write the PMU through it. Every command the
driver sent was answered before the suspend, so a reply still waiting
at the resume is someone else's, and the next command takes it as its
own. On the M4 Max that was the RTC's acknowledge of the alarm that
woke the machine ("unexpected SPMI response 0x8b30"), after which the
RTC refused every later suspend.

Read such replies out at resume_noirq after a firmware-driven suspend.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
aura writes the PMU on the same queue at the wake. Waiting for the
first reply after a write could take one of its replies for ours and
leave ours behind. Empty the FIFO before the write as well.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@joske

joske commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Tried this on a J614c (MacBook Pro 14" M4 Max, 40-core GPU), os-fw 26.6.2, EL2 through aura main d5e180ba. The kernel was integration/aurora 9219565 with this PR merged on top, plus an apple,t6041-sleep node in t6041-j614c.dts (the J614c's ADT has the same pmgr and apcie0 as the J616c's, so the driver's tables should fit). It probes: "T6041 deep sleep at EL2 through aura's S2R mailbox", and mem_sleep offers deep.

GPU: the 40-core part runs CONFIG_25G83 in hw/t6041.rs, which has no sleep_restart, so the first try was refused ("no firmware restart on this chip, refusing sleep"). Setting sleep_restart: true there, as t6041_25g83::CONFIG has, got past that.

Then it hangs:

  • dry=1 + rtcwake -m mem -s 20: never comes back; only a forced power-off helps.
  • pm_test=devices: comes back fine.
  • pm_test=platform: hangs. So it's in a late or noirq callback, before .enter.

PMU notes (diag=1, read after the forced reset from /sys/bus/nvmem/devices/spmi_nvmem0/nvmem at 0xa200; the 0x8308 mark is 0x10 both times):

  • With everything bound: last device 0000:01:00.0 (brcmfmac), callback count 0x195.
  • With 0000:01:00.0 and 0000:01:00.1 removed from the bus: last device 404c80000.iommu (the SEP's DART), count 0x18b.

The count drops by 10, the two removed devices' callbacks, so the hang comes at the same point in the sequence. The noted device looks like whatever is next, not the cause. My guess: something is already stuck by then, possibly around the SEP going to sleep at device suspend.

One more J614c data point, from #1477's hibernation dry runs on this machine: at every pm_test level from platform up, apple-pmgr-pwrstate ...power-controller@c00: PS sep: Failed to reach power state 0xf (requested 0xf, now: 0x1000030f) on the way back. I can run more tests or provide a different log if it helps.

… lists

On T6041 the enclave is told to sleep at device suspend. Its DART is
saved first with apple_dart_sleep_prepare(), which only saves a DART
whose node has apple,sleep-restore, and only such a DART is then left
alone by its own callbacks. Without the property nothing was saved, the
enclave slept, its power domain went off, and the DART's noirq suspend
read its registers: the suspend hung there. This was seen on a J614C
booted with the in-tree device tree, which has no lists for that DART.

Refuse the sleep at suspend instead when the DART has no list.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Without the mailbox the driver set its suspend ops anyway and refused
every sleep in .enter. Deep sleep is then the default, so a kernel
booted at EL2 by another loader could not suspend at all. Don't bind
instead.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
Give the SEP's DART the writes around a sleep that T8140's has, as the
J616C's bring-up device tree does: the SEP driver saves the DART with
them before the enclave sleeps at suspend, and refuses that sleep
without them.

Add the apple,t6041-sleep node to the J614C and J616C, so the in-tree
device tree is enough to test suspend-to-RAM there. The driver stays out
unless aura passes its S2R mailbox.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The 40-core J614C on 25G83 firmware runs CONFIG_25G83, which had no
sleep_restart, so its GPU refused every deep sleep. Set it as the
32-core profile has.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks for trying it on the J614c, this helps a lot.

I think the hang is the SEP's DART. On my J616c the bring-up device tree gives iommu@404c80000 three sleep lists (apple,sleep-prepare, apple,sleep-restore, apple,sleep-restore-late), and the in-tree t604x.dtsi has none. On T6041 the SEP is told to sleep at device suspend, and the SEP driver saves its DART with those lists first, after which the DART's own callbacks leave it alone. Without them nothing is saved, the enclave sleeps, its power domain drops (your 0x1000030f), and then the DART's noirq suspend reads its registers and the bus hangs. That fits both your runs: the hang is at the same point in the order, and the noted device is just whatever started last.

I've pushed four commits for it:

  • the lists on &sep_dart in t6041.dtsi, and the apple,t6041-sleep node in t6041-j614c-j616c.dtsi, so the in-tree tree should be enough now
  • the SEP refuses the sleep with a message, instead of hanging, when its DART has no lists
  • sleep_restart: true for CONFIG_25G83, as you did
  • the driver doesn't bind without aura's mailbox

To try it on your current build, this in your DTS should be enough:

&sep_dart {
	apple,sleep-prepare = <0x80 0xa102>, <0x704 0xffffffff>;
	apple,sleep-restore = <0x210 0x0>, <0x20c 0x800005f5>,
			      <0x220 0xf0f0f>, <0x224 0x80808>,
			      <0x300 0x1>, <0x318 0x1>;
	apple,sleep-restore-late = <0x80 0x8000>, <0x228 0x3f3f3f3f>,
				   <0x22c 0x1010101>;
};

dry=1 only stops at .enter, so it can't catch anything earlier. If it gets further and then resets or hangs, the 0x8308 mark after the forced reset says how far it got (6: it reached the commit, 0x67: Linux got all the way back), and the notes at 0xa200 work as before. The new commits are only build-tested; my J616c already slept with the lists, so nothing changes on it.

Remove what was only there to find the sleep's problems on hardware:
the dry, diag and serror_go parameters, the progress marks and notes in
PMU scratch and SRAM, the register snapshot against the macOS table at
the commit, the before and after windows with their S2RDIFF and S2RAMCC
lines, the console, die and panic notes, and code nothing called any
more (the MCC stop, the AXI2AF teardown, the core snapshot).

The writes, delays and polls of the commit and of the wake are the same
as before. The AMCC cache words and the display's PMC votes, which the
wake puts back, are saved on their own now instead of through the
comparison windows. The clkgen write-back that worked from those windows
only ran with diag set, and goes with them. What the wake could not
bring back is reported at .end.

apple_sleep_coprocs_count() had no other user and goes too.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
apple_dart_display_keep() read the whole register block of a display
DART before a sleep and printed what differed after the wake, with each
stream's TCR and TTBR. That was only there to look for what the wake
took. Keep the TCR and TTBR save and write-back, and declare the helper
in <linux/soc/apple/dart.h> instead of in its caller.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The display stop and restart for T6041's deep sleep ran as test steps
first: an s2r_stop parameter took them one at a time, and each step
wrote progress marks to PMU scratch, logged S2RSTOP, S2RSTART and
S2RTIME lines and kept result words. A kmsg dumper put a panic's call
trace into PMU scratch, and s2r_bl showed the backlight state.

Remove all of that, and the parameters with it. The waits are constants
with the values that ran (settle 200 ms, words 100 ms, stop 300 ms,
down 100 ms), and each switch keeps the behaviour it had by default: the
display power-off without the sleep transition (s2r_hard = 2), bridge 4
left as boot has it, the GAPF word not written back, the backlight and
the boot power words restored, the steps run from the display's own
suspend and resume. What fails is still reported. The power word step
now unmaps what it maps.

dcp_sleep() has no caller again and gets its __maybe_unused back.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
…ISPEXT

The M4 Max gate enables each external display chain as the in-tree
trees describe it: no DISPEXT power domain, the domains are on as iBoot
left them. The J616c's bring-up EL2 tree does describe them: the chains'
DARTs, mailbox and DCP point at dispextN_fe, which is disabled like the
PMP report entries above it. Enabled as they were, those devices waited
for power domain providers that never came and failed at the deferred
probe timeout (the mailboxes with -110), and the display subsystem never
bound: the internal panel stayed on simpledrm.

Drop the power-domains property of a chain node whose domain stays
disabled, so such a tree ends up as the in-tree ones do. Nothing changes
with the in-tree trees.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
The power word step forces both external display chains down, and the
display subsystem maps every framebuffer through their display DARTs.
Only the DART of an external DCP the stop visited was marked dark, but
a DCP whose probe failed (the display subsystem skips it as unusable)
is not bound to the DCP driver, and the stop never saw it. After the
wake, an sddm restart or a logout unmapped a framebuffer, flushed that
DART behind its dead block, and the machine stopped: on the J616c the
HDMI chain's display DART at 0x3a2300000.

Mark the external chains' display and DCP DARTs dark by device tree
node right before their power words go down, while they can still be
read, whether or not their DCP bound.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@joske

joske commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Thanks, the SEP DART lists were it: the hang at pm_test=platform is gone on the J614c. A real sleep now gets to the commit, but the Mac resets instead of waking.

Same J614c (MacBook Pro 14" M4 Max, 40-core GPU), os-fw 26.6.2, aura 1.0.0b12 + aura#166 + a local dcpext3 entry. Kernels: integration/aurora fa59399 + #1667/#1684 and a few J614c DT patches, with this PR merged at two points:

  • 35.7: PR head 128743e, no local sleep patches. The in-tree apple,t6041-sleep node probes ("T6041 deep sleep at EL2 through aura's S2R mailbox") and mem_sleep defaults to deep.
  • 35.6: PR at fe530fa, to still have diag and the PMU marks.

pm_test=platform (35.7): comes back. The PMP refuses ("firmware never stopped"), and so do the SIO and the GPU. The SEP logs "the enclave stays asleep until the next real sleep or a reboot". Two problems after it:

  • The internal panel stays black. The system is fine over SSH and GNOME is running, but apple-panel-bl reads brightness 54 and actual_brightness 0. Writing a new brightness, or display power-save off and on through Mutter, gets no reaction from the driver (nothing in the log). On the way down, dcp@402e00000 logged Endpoint 0x37 is not running, cannot send message, iomfb_poweroff_v26_6_desktop: clear swap timed out and setPowerState(0) timeout 1000 ms.
  • The next reboot hangs after SSH is gone; only a forced power-off helped.

Real sleep, rtcwake -m mem -s 20: the Mac resets (35.7, and 35.6 with diag=1). The journal ends at "PM: suspend entry (deep)".

PMU scratch after the 35.6 reset (diag=1; the 0x8308 mark was 0x10 before the run):

  • 0x8308 = 0x06: it reached the commit, but not 0x60, so Linux never ran again.
  • Bit 11 counts all 0: 0x8314, 0x831c-0x831f and 0x8311; 0x8316-0x8318 and 0x8384+ are 0xff.
  • 0x830a = 0x10, 0x830b = 0x0f, 0x830c = 0x4d.
  • 0x830d = 0x80: no USB-C port domains left on.
  • 0x830f = 0x18, 0x8312 = 0x98, 0x8315 = 0x80.
  • 0x8310 = 0xc0: all 64 keep-alive words.
  • 0x8319 = 0x80, 0x831a/b = 0x4f/0xff.
  • 0x8342/3 = 0x06/0x03, 0x8344/5 = 0/0.
  • SEP: 0x8346/7 = 0x07/0x07, 0x8359/5a = 0x3f/0x00, 0x835b = 2 ms.
  • 0x8348 = 0x80, 0x8349/4a = 0xff/0x81, 0x834b/c = 0x4f/0x01.
  • 0x8350/1 = 3 non-PS and 52 PS rows differing from macOS. The PS bytes from 0x83a2: ff, 13 × 4f, 15 × 00, 14 × 4f, 0f ff ff ff 00 00 ff 00 00.
  • 0xa200 note: 500700000.pmp, count bytes ba 02.

So the commit looks clean from here: nothing busy, the SEP asleep, the USB-C domains down. The failure is after it: the RTC wake doesn't come, or the wake path resets before it reaches Linux.

One more thing: with Wi-Fi WoWLAN enabled (iw phy phy0 wowlan enable magic-packet, or whatever re-enabled it at boot here), brcmfmac refuses deep with -95 (brcmf_pcie_pm_enter_D3), so the suspend is aborted. That's by design, WoWLAN being s2idle only, but it may be worth a hint when deep is the default.

I'm happy to run more: a different wake source, a debug build, or other PMU offsets.

The display stop for a T6041 sleep kept one "stopped already" slot per
external DCP by name: dcp@3a2e00000 got one, every other external DCP
shared the second. With more than two external chains enabled (dcpext2,
dcpext3) the second external DCP to be stopped marked that slot, and
any after it was skipped: its firmware still ran at the commit. The
DARTs made dark before the power word step were also only those of the
first two chains.

Keep the mark in each DCP, and make the display and DCP DARTs of every
enabled apple,t6041-dcpext node dark, found through the device tree.

Signed-off-by: AxxzyWasTaken <axxzytte@gmail.com>
@AxxzyWasTaken

Copy link
Copy Markdown
Contributor Author

Thanks, that's a big step: the commit really is clean now. Your 35.6 record matches the commit record of my J616c when it wakes fine almost byte for byte: 3 non-PS and 52 PS rows differing from macOS, the same 0x8342-0x834c, the same PS bytes, nothing busy, the SEP asleep, the USB-C domains down. Only the first differing PS row differs: ff on yours, 0f on mine.

My best guess for the reset: your kernel has #1667, #1684 and a local dcpext3 entry, so it runs more external DCPs than mine. The display stop before the sleep kept one "already stopped" slot per external DCP by name: dcp@3a2e00000 got one, and every other external DCP shared the second. So with three or four external DCPs, the second one stopped took that slot and the rest were skipped, their firmware still running at the commit. Only the first two chains' DARTs were made dark, too. I've pushed drm/apple: T6041 S2R: every external DCP, not only the first two, which keeps the mark in each DCP and makes the DARTs of every enabled apple,t6041-dcpext node dark.

Two things that would tell us more:

The pm_test=platform black panel is a real gap: the display stop assumes a real sleep follows. Under pm_test the firmware is stopped but never power-cycled, and the restart path expects a power cycle. So pm_test=platform with the display isn't supported yet. I'll put that in the PR text, along with your WoWLAN note (brcmfmac refuses deep with -95 while WoWLAN is enabled).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants