Repository navigation
pmdomain: apple: clear a latched bit 11 with a PS reset on power-on (Neo ANE after sleep) - #1255
Merged
Merged
Conversation
On the Neo (T8140) ANE_SYS sometimes fails to come back on when genpd powers it on in the noirq resume of a system sleep: PS ane_sys: Failed to reach power state 0xf (requested 0x200f, now: 0x2b4f) Target 0xf, actual 0x4 (clock-gated), bit 11 set. genpd ignores the error there, the ANE finds ANE_SYS off at PM_POST_SUSPEND and quarantines itself until reboot, and every later power-on of the domain stops the same way. It happens in about one boot in four, within the first ten sleeps. Bit 11 is not a parent that comes up late here. A test kernel that kept watching the word saw no change for 20 ms; the fabric domains above ANE_SYS (0x1a8, 0x1e8, 0x228) and its siblings were all on. Gated again, the domain reports actual 0 at once, but bit 11 stays set (0x2b00), and the next power-on stops at 0x2b4f again. Writing bit 11 back does nothing. A PS reset pulse while the domain is off clears it (0x3300, then 0x2300), and the power-on then reaches 0x23ff in 8 us. So when a power-on times out with bit 11 set and the actual state short of the target, gate the domain, pulse the PS reset, release it and request the on state again, each step with the usual 100 us budget. The devices of the domain lose their state while it is off anyway. Domains whose bit 11 means a parent that is not up yet (wait_parent, T6000 GFX), strict-state and externally clocked domains are left as they were. A latch the reset does not clear still fails and logs. KUnit: four new cases (latch cleared, cleared in the noirq phase, a latch the reset does not clear, no reset for wait_parent domains or a plain stall). Signed-off-by: Ryan Murray <81541646+RyanTheTide@users.noreply.github.com> Assisted-by: LLM
2 tasks done
rolotrealanis98
pushed a commit
to rolotrealanis98/linux
that referenced
this pull request
Nov 10, 2026
A power-on that stops with bit 11 latched is retried after a PS reset pulse while the domain is gated (commit a10defe). The genpd noirq resume runs that power-on with the domain's DARTs and engine still in their noirq phase, and on the T8140 the recovery there hangs the machine. In a suspend loop on the J700 (12 systemd and 12 direct pm_test=platform suspends per boot, from a fresh boot), kernels with the recovery hung on 3 of 3 boots. The hw watchdog brought each one back, and no "cleared by a PS reset" line ever reached the disk. Kernels without it (fleet36, and the same tree with the recovery reverted) hung on none of 6 boots. Two of those boots latched ANE_SYS, quarantined the ANE and kept going. The freezer and devices test levels, which never power a domain off, hung on none of 6 boots with the recovery in place. This is the J700 "hang on suspend entry" seen in fleet43 suites and on aneres2. Skip the recovery while the domain is in a system sleep transition (from its suspend_late to its resume_early). The power-on then fails as before aurora-silicon#1255. genpd ignores the error in that phase and keeps the domain requested on, and the ANE quarantines at PM_POST_SUSPEND as it did before. Runtime power-ons, including apple_pmgr_ps_cycle(), keep the recovery. Signed-off-by: Ryan Murray <81541646+RyanTheTide@users.noreply.github.com> Assisted-by: LLM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
On the Neo (J700, T8140) the ANE sometimes does not come back from system sleep:
After that, every later power-on of ANE_SYS stops the same way until reboot. This kept fleet38/fleet39 off the Neo.
The cause is a latched bit 11 in the ANE_SYS power-state word. A PS reset pulse while the domain is gated clears it. This PR makes
apple_pmgr_ps_set()do that: when a power-on times out with bit 11 set and the actual state short of the target, it gates the domain, pulses PS_RESET, releases it and asks for the on state again. Each step gets the usual 100 us budget. Domains where bit 11 means "parent not up yet" (wait_parent, T6000 GFX), strict-state domains and externally clocked domains are left alone. If the reset does not clear the latch, the power-on still fails and logs as before.One commit,
drivers/pmdomain/apple/pmgr-pwrstate.c+ its KUnit file.How it was found (Neo, 2026-11-09)
Kernel journals of every bisect boot: all failures are the noirq-resume power-on of ANE_SYS stopping at
0x2b4f(target 0xf, actual 0x4, bit 11). genpd ignores that error. The ANE then finds ANE_SYS off at PM_POST_SUSPEND and quarantines.It is not tied to thunderbolt: xdomain: do not redo lane bonding on a link that is still bonded (J616s s2idle, #1230) #1244/media: apple: jpeg: complete the T6000 reset image, enable M1 Pro JPEG #1237/media: apple: ave: speed over quality for HEVC Main 10 too #1239. fleet37 failed the same way once (02:59). Five fleet38 boots with about 250 resumes were clean. The failure hits roughly 1 boot in 4, always within the first ~10 sleeps, and then it is permanent. The bisect (one run per kernel) was noise.
Diagnostic test kernel
aneres1(fleet39 + a debug patch, not in this PR) reproduced it on the third fresh suite boot:Gated again, the word read
0x2b00at once: actual 0, bit 11 still set. The next power-on stopped at0x2b4fagain. So this is not a slow parent.On the wedged word, through /dev/mem (only 0x290 written):
0x2000(off)0x2b000x2800(bit 11 back)0x2b000x200f(on)0x2b4f0x3000(off + PS_RESET)0x33000x2000(release)0x23000x200f(on)0x23ffafter 8 usTests
apple-pmgr-pwrstate(arm64,.kunitconfig.pmgr-pwrstate): 14/14, run twice. Four new cases: latch cleared; cleared in the noirq phase; a latch the reset cannot clear still fails and is requested off; no reset forwait_parentdomains or for a plain stall. Negative control (new tests, old driver): 11/14, all three latch cases fail.7.3.0-aurora-aneres2= integration eb24190 + this commit, CTXR, own release string, one-shot boots only. Eight fresh boots, each running the full FLEET regression suite (6 systemd + 6 direct pm_test sleeps, 3 rtcwake s2idle, every sleep with an RTC alarm armed).ane_resume15/0 in each, so 105/105 ANE resumes. 0resume FAILED, 0ane_sysfailures, 0 kernel error-level lines. MobileNet top-1 16/16 after sleep.resume ok; the hardware watchdog reset it. That boot has nocleared by a PS resetline, so this change did not run there. It is the only boot of the last 45 Neo boots that ended this way, and it is being tracked separately.cleared by a PS reset.Evidence:
traces/kernel/20261109-aneres-ane-sys-latch(S500, not tainted: our own captures).Related: #1139 (T604x ANE_SYS parent wait, draft) shows the same
0x0b4fsignature on the J616s, and the J616s ANE_SYS also gets this recovery. A Daybreak task checks how macOS treats a latched bit 11 (follow-up only).🤖 Generated with Claude Code