Skip to content

Keep the USB host through deep sleep on the MacBook Neo (T8140) - #1217

Closed
DjDeveloperr wants to merge 22 commits into
feat/t8140-s2rfrom
feat/t8140-s2r-usb-keep
Closed

DjDeveloperr wants to merge 22 commits into
feat/t8140-s2rfrom
feat/t8140-s2r-usb-keep

Conversation

@DjDeveloperr

Copy link
Copy Markdown
Contributor

Stacked on #1208 (base feat/t8140-s2r), which goes in first.

With #1208, every deep sleep powers the USB host down, so at each wake the hub resets and USB devices enumerate again. macOS on the same machine keeps the host. Its USB host votes for sleep DDR, so the sleep stops at SLP_DDR, and DRAM, the memory controller and the host's always-on side stay powered. This series does the same on Linux.

What it changes

  • DART: a sleep-prepare list can stop the streams, and the consumer can report that it lost power (binding, driver, T8140 tree).
  • t8140-sleep, pmgr-pwrstate: a driver can take its own power domain down for a sleep, and pmgr-pwrstate leaves that domain to it.
  • ATC PHY: takes the USB host's DARTs and power domains down for a sleep, as macOS does. It also closes the fabric bridge's sleep gate (new apple,axi2af-gate) and keeps the always-on side at the wake.
  • xhci-plat, dwc3-apple: xHCI restores its saved state when the always-on side was kept, and starts over when it can't.
  • The vote: dwc3-apple votes for sleep DDR while the host is up (new apple,sleep-ddr), and t8140-sleep writes it to the sleep-DDR word.
  • J700: no USBw for the port controllers, as on macOS.

The first-sleep rule

After a Linux boot, a sleep with the sleep-DDR word set stalls in the sleep engine's wake, and the watchdog resets the machine about 5 s after the alarm. After one deep sleep with the word clear in the same boot, it returns every time.

So t8140-sleep keeps the word clear, whatever the host votes, until a wake shows that the deep path ran. AP_TMM is on only after C2S, and only that path runs C2S. That wake applies the held vote, and every later sleep keeps the USB host. The first sleep of each boot costs one hub reset, the same as every sleep without this series.

We don't know yet which boot-time state the deep sleep rebuilds. Once we do, the kernel should set it at boot, and this rule can go.

Testing

J700 (MacBook Neo), one boot, six sleeps, each with a +30 s PMU RTC alarm. Nothing wrote the sleep-DDR word by hand.

Sleep Word before the sleep USB host at the wake Hub resets Back after the alarm
1 0x0 always-on side lost; the deep path ran and the vote was applied 3 1.6 s
2 0x1 always-on side kept, xHCI state restored 0 1.2 s
3 0x1 always-on side kept, xHCI state restored 0 1.2 s
4 0x1 always-on side kept, xHCI state restored 0 1.2 s
5 0x1 always-on side kept, xHCI state restored 0 1.2 s
6 0x1 always-on side kept, xHCI state restored 0 1.2 s
  • The hub kept the same device number through all six sleeps. Every wake came from the PMU RTC alarm.
  • An earlier debug build with the same rule logged the power states. Sleep 1 took the deep path (AP_TMM 0x300 to 0xff). Sleeps 2-6 stopped at SLP_DDR (AP_TMM stayed 0x300, USB always-on state 0x3ff kept). Six of six returned.
  • The run used these 16 patches on Deep sleep (suspend-to-RAM) for the MacBook Neo (T8140) at EL2 #1208's previous head, bab0092. Between that head and the current one, the only change is that deep is now the default sleep state. The patches rebase unchanged: git range-diff shows all 16 identical.
  • The test kernel also had a test-only switch that keeps the debug link up for the test rig's console. That switch isn't part of this series.
  • Every commit builds with W=1 (the objects it touches, users of the headers it changes, and t8140-j700.dtb). dt-doc-validate passes on the three bindings.

🤖 Generated with Claude Code

…eams

A DART's apple,sleep-prepare writes run after its state is saved, so a
list may stop the DART's streams before its power goes for a sleep, as
macOS does for the USB host's DARTs. Say so, and that the restore starts
those streams again.

Signed-off-by: Diljit Singh <hey@djdev.me>
…mer say it lost power

macOS stops the USB host's DARTs before their power goes for a sleep:
the exception words, the block errors, every stream disabled, then a
flush (measured). A consumer's apple,sleep-prepare list can say that,
but the DART's state and stream enables were read after the list ran,
so the restore would bring the streams back disabled. Read them first.
The SEP DART's list stops no stream and doesn't touch what is read, so
it behaves as before.

A consumer that cuts its DART's power before the commit knows the state
is gone even when the sleep is refused, where pm_resume_via_firmware()
is false. apple_dart_sleep_restore_lost() takes that from the caller,
and starts streams the prepare list stopped when nothing was lost.
apple_dart_sleep_listed() tells a consumer whether the DART's own
callbacks leave it alone once prepared.

While a consumer holds a DART whose list stopped its streams, TLB
commands wait for the restore: the list flushed it, no stream is left
to fill it, and its power may be off. A restore with nothing lost
writes the late list too, which undoes the rest of such a prepare list.

Signed-off-by: Diljit Singh <hey@djdev.me>
What macOS writes to both of the USB host's DARTs when the host sleeps
and keeps its power (measured): at entry the exception words, the block
errors, every stream stopped and a flush; at the wake the config words,
the translations, then 0x228, 0x22c and the exception word, and the
streams. Linux enables all 16 streams, so the prepare list stops all
16. 0x104 is the error mask Linux's own reset clears.

With these the host's PHY can save the DARTs before their power goes
and put them back once it is on.

Signed-off-by: Diljit Singh <hey@djdev.me>
…for a sleep

macOS's USB host lowers ATC0_USB and then ATC0_COMMON in its own
suspend, 0.7 to 1.7 s before the CPUs stop, and the commit only reads
them (measured). Linux lowered both at the commit, about 4 ms after the
other cores stopped.

apple_sleep_domain_set() lowers or raises a power word for the driver
behind it, the way the commit does, with the word found from the power
controller's node: its reg in the PMGR block "ps" maps. A lower waits
for the word to read off. The commit now lowers the two ATC0 words only
if they are still up.

apple_sleep_domain_check() tells the driver beforehand whether the word
is one of the sleep driver's; apple_sleep_domain_is_down() tells the
PMGR's driver at the wake that a driver took the domain down and raises
it itself. A raise of a word that is on already sets AUTO_ENABLE, as
macOS's does.

Signed-off-by: Diljit Singh <hey@djdev.me>
…or the sleep to that driver

macOS's USB host raises ATC0_COMMON, opens the fabric bridge, then
raises ATC0_USB at the wake (measured). genpd still has the host's
domains on, so the resync after a real wake raised them first,
ATC0_USB included, before the PHY could open the bridge. Skip a domain
whose driver took it down with apple_sleep_domain_set() and hasn't
raised it again: that driver raises it itself, in its order. Only the
PHY, the controller and its DARTs live in those domains, and their
callbacks leave them to the PHY until then.

Signed-off-by: Diljit Singh <hey@djdev.me>
The ATC fabric bridge bits macOS clears before the PHY's common power
domain goes off for a system sleep and sets again once it is back, as
offset and bit pairs in the PHY's axi2af region.

Signed-off-by: Diljit Singh <hey@djdev.me>
macOS clears bit 0 of the ATC fabric bridge's words 0x104 and 0x108
after ATC0_USB is off and before ATC0_COMMON goes off, and sets them
again right after ATC0_COMMON is on, at every such change, its PMGR's
restore at the wake included (measured over four sleeps). Linux never
touched them. Give ATC0's PHY those bits, so it can gate the bridge
around the common domain when it takes the controller's domains down
for a sleep.

Signed-off-by: Diljit Singh <hey@djdev.me>
… a sleep, as macOS does

Right after the controller's hibernation, 0.7 to 1.7 s before the CPUs
stop, macOS's controller driver stops and flushes both DARTs, lowers
ATC0_USB, clears the fabric bridge bits and lowers ATC0_COMMON.
ATC0_USB_AON is never written. At the wake it raises ATC0_COMMON, sets
the bridge bits, raises ATC0_USB and reprograms the DARTs before the
controller's D0 (measured over four sleeps). Linux left the DARTs to
their own callbacks, never touched the bridge, and lowered both domains
at the commit, about 4 ms after the other cores stopped.

Do it as macOS does, from the device tree: the DARTs are the
controller's iommus, saved and restored with their sleep lists; the
domains are the PHY's power domain and its parent; the bridge bits are
apple,axi2af-gate. The bridge gets back the bits that were set before
the sleep, and the log says which were. The DARTs' own callbacks leave
them alone in between, so nothing touches them while their domain is
off, a refused sleep included. Since the PHY now takes ATC0_COMMON down
itself, it sets the common block up again after every sleep it
hibernated for. If the power-down fails the controller comes straight
back and stops for later sleeps; without all the pieces in the device
tree it stops for every sleep, as before.

If the domains don't come back on, the PHY refuses every operation that
could reach them until a reboot.

Signed-off-by: Diljit Singh <hey@djdev.me>
…the wake

When the controller's always-on side keeps power through a sleep,
macOS's wake leaves the PHY's core alone in USB2 mode, holds pwrctl's
word 0x18 (acknowledged in 0x1c) around the core's USB3 setup, and
writes nothing to the secondary eUSB2 bank (measured). The USB2 PHY
then still holds the hub's link in U3.

Tell a kept always-on side from a lost one by the power control's
status, which reads all zeros after a power loss, before the common
block is set up again. With it kept: in USB2 only forget the common
block's state, so the next mode change sets it up; in USB3 restore it
under that lock; leave the eUSB2 bank alone. With it lost everything
goes as before. The result is APPLE_ATCPHY_SLEEP_POWER_LOST for the
controller's driver, which still starts xHCI over for it as before,
and a pending wake event is logged.

Signed-off-by: Diljit Singh <hey@djdev.me>
…state

xhci_resume() reinitializes a controller whose restore ends with a save
or restore error, but one whose Restore State never finishes, or that
never gets ready, fails the resume and leaves the host dead. A resume
quirk that readied a controller for its restore is the case where that
can happen on a working machine: the Apple DWC3 core loses its registers
in a deep sleep and is set up again before xHCI restores.

Try once more as a controller that lost its state, so its devices are
reset-resumed instead of left without a host.

Signed-off-by: Diljit Singh <hey@djdev.me>
…the always-on side

macOS's controller loses its core's registers in every sleep too: at
the wake its GUCTL reads the reset value, and its driver writes the
globals and the tunables again. What survives is the always-on side,
which reads its sleep state after the wake and gives D0_ON in about
1 ms, and xHCI's saved state. macOS then writes the operational
registers back, sets Controller Restore State, and resumes the root
port from U3 to U0 with no port reset, so the hub and the keyboard
behind it stay enumerated (measured).

Linux set the core up again and told xHCI it had lost its state every
time, so every wake reset the hub and reset-resumed its devices, 0.4 to
0.65 s on the wake's critical path. Tell it so only when the PHY says
the always-on side lost power too. A restore that fails falls back to
the reset in xHCI itself, and the hub check still restarts a host whose
hub isn't back.

Signed-off-by: Diljit Singh <hey@djdev.me>
While its USB host runs, macOS votes USB_SLP_DDR for the controller,
and a system sleep then keeps DRAM up for the controller's always-on
side. Describe the controllers whose host does so.

Signed-off-by: Diljit Singh <hey@djdev.me>
…ost is up, as macOS does

macOS sets bit 0 of the wake block's word 0 from its USB host's start to
its Off: the host raises USB_SLP_DDR-V at every power-up and drops it
only on the Off path, and the SoC tuner turns that vote into the bit. It
reads 1 before and after every macOS sleep, nothing writes it at a sleep
or a wake, and the host's always-on side keeps its power through the
sleep (measured). Linux cleared it before every commit, and its host's
power control read all zeros after each wake.

Take the vote while a controller with apple,sleep-ddr runs as a host. A
host stopped for a sleep drops it first. The commit leaves the word
alone. No device tree sets the property yet.

Signed-off-by: Diljit Singh <hey@djdev.me>
…eep path

With the sleep-DDR word set, the T8140 sleep stops at SLP_DDR and the
wake runs D->M->A on the SoC-side state the system left. After a Linux
boot that wake stalls in the sleep engine, and the watchdog resets the
machine about 5 s after the alarm. After one deep sleep it gets through
every time: in one boot, a deep sleep, then five sleeps with the word
set, each back 0.4 s after its alarm with the USB host's always-on side
kept and no hub reset (measured).

So keep the word clear, whatever the USB host votes, until a wake shows
the deep path ran: AP_TMM is on only after C2S, which only that path
runs. That wake writes the held vote, so every later sleep keeps the
USB host. The first sleep of each boot costs one hub reset, as every
sleep did before the vote.

Which boot-time state the deep sleep rebuilds isn't known yet. Once it
is, set it at boot and drop this.

Signed-off-by: Diljit Singh <hey@djdev.me>
On the J700, Apple's device tree gives usb-drd the USB_SLP_DDR-V clock
gate, which macOS raises while its USB host runs, so its sleeps keep
DRAM up for the host's always-on side. Mark the controller with
apple,sleep-ddr so Linux's host takes the same vote.

Signed-off-by: Diljit Singh <hey@djdev.me>
…n macOS

macOS sends both port controllers SSPS 3 at a sleep's entry and SSPS 0
at the wake, and nothing else: no USBw, as the J700's Apple device tree
has no dock-remote-wake. A key press on a USB keyboard woke macOS
through ATC0_PMGR_USB_WAKEUP, the host's own wake source 70, after its
power control's wake event (measured).

The USBw arm stood in for that while the host lost its always-on side
in every sleep. With that side kept, leave it out.

Signed-off-by: Diljit Singh <hey@djdev.me>
Carry the integration/aurora reconciliation into the USB sleep-DDR stack without rewriting its 16 commits. Keep the first sleep on the deep path and apply the held USB vote only after that wake.
Keep the driver-owned domain skip at the wake next to #1283's no-recovery
rule during system sleep; take the J700 JPEG1, watchdog IRQ and #1304 as is.
Brings in integration/aurora 15f88e1 through #1208. The conflicts are
the glue's kernel-doc and the neo-stability host test's mock: this
branch's sleep_ddr is kept next to integration's drd_regs and usb2_phy.

Signed-off-by: Diljit Singh <hey@djdev.me>
Brings in #1208's aop-bootargs host test fix. No conflicts.

Signed-off-by: Diljit Singh <hey@djdev.me>
Signed-off-by: Diljit Singh <hey@djdev.me>
Brings in integration/aurora ac11649 (#1311, PCIe multi power domain
support) through #1208. No conflicts.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr

Copy link
Copy Markdown
Contributor Author

Superseded by #1445 (merged as 2b05915); its changes are in integration/aurora. The kept USB host is in drivers/phy/apple/atc.c and drivers/soc/apple/sleep.c. The rule that sleeps stay deep until one deep path has run was reworked later in the stack (#1393) and is now sleep_slp_ddr_deep_once.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant