Skip to content

perf: apple: A9 memory port byte counters - #123

Closed
brkosebay wants to merge 40 commits into
OrchardLinux:mainfrom
brkosebay:j71t/amc-pmu
Closed

brkosebay wants to merge 40 commits into
OrchardLinux:mainfrom
brkosebay:j71t/amc-pmu

Conversation

@brkosebay

Copy link
Copy Markdown
Contributor

Part 17 of 23 of the iPad 5 (J71t) series in this repository; merge after #122.

Problem

There was no way to measure memory bandwidth per port on A9, which is needed to tell GPU work from memory stalls.

Change

Add a binding, DT node and perf driver for the A9 AMC port monitor, exposing CPU and GFX read/write byte counters, and enable it for apple-16k.

Testing

  • Hardware: iPad 5 Wi-Fi (J71t). Hands-on testing on 2026-11-11 with an integrated build of this series, made before the final failure-path fixes, covered the counters under GPU load (GFX writes match the expected traffic within a few percent).
  • Not covered on hardware: CPU write calibration, which reads low.
  • 2026-11-12: no further hardware test of this change beyond the above. The top of this series applies to the same kernel tree as the release build tested on J71t that day (20+ warm restarts without a freeze).
  • Host: the Linux test suite passes at this branch's head, and the kernel and m1n1 patch series apply on apple-16k and apple-4k. At the top of the series the applied kernel tree is identical to the one in the built release image.
  • Other boards that share these drivers or compatibles have not been tested.

Dependencies

Depends on #122.

This branch is stacked on the PRs it depends on and carries their commits; the change of this PR is its last commit, "perf: apple: A9 memory port byte counters".

🤖 Generated with Claude Code

Add the A9 (S8003) G5P render driver used by the merged PowerVR/G5P Mesa
winsys as kernel patches, enable it on J71t in the device tree, set
CONFIG_DRM_APPLE_G5P=y for apple-16k, and add an install-time recipe that
extracts the original META LDR (apple/agx-g5p-c0.ldr) from the device's own
iOS kernelcache into /lib/firmware. No firmware is committed.

The driver provides TYPED_V3 rendering, GEM buffers, VM/arena mappings,
PRIME import/export, syncobj, bounded retirement and recovery. It does not
expose the old compute or fixed-fixture ABI.

Hardware: the exact kernel this derives from (brkosebay/linux a9/gpu-pr1
407b7bf2cbbe) booted on a J71t with the merged Mesa, rendered labwc on the
GPU with KMS scanout at 60 Hz, and passed the G5P device rungs with zero GPU
events. This Orchard adaptation itself has not yet been built by orchard or
run on hardware.

Assisted-by: LLM
kernel/configs/apple-16k/config:
CONFIG_DRM_APPLE_G5P=y
These options provide g5p and its Kconfig dependencies on both page sizes.

Assisted-by: LLM
kernel/configs/apple-16k/config:
CONFIG_APPLE_AOP_GYRO=m
CONFIG_APPLE_AOP_PRESSURE=m
These options provide aop and its Kconfig dependencies on both page sizes.

Assisted-by: LLM
kernel/configs/apple-16k/config:
# CONFIG_NVME_APPLE_ANS_EARLY_USB is not set
These options provide warm-storage and its Kconfig dependencies on both page sizes.

kernel/configs/apple-4k/config:
# CONFIG_NVME_APPLE_ANS_EARLY_USB is not set
These options provide warm-storage and its Kconfig dependencies on both page sizes.

Assisted-by: LLM
kernel/configs/apple-16k/config:
CONFIG_APPLE_AOP_GYRO=m
CONFIG_APPLE_AOP_PRESSURE=m
These options provide aop and its Kconfig dependencies on both page sizes.

Assisted-by: LLM
kernel/configs/apple-16k/config:
# CONFIG_NVME_APPLE_ANS_EARLY_USB is not set
These options provide warm-storage and its Kconfig dependencies on both page sizes.

kernel/configs/apple-4k/config:
# CONFIG_NVME_APPLE_ANS_EARLY_USB is not set
These options provide warm-storage and its Kconfig dependencies on both page sizes.

Assisted-by: LLM
Bring this branch onto current main. The merge keeps the other side's
tree; the next commit carries the reworked version of this branch's
change.
A7-A9 have no EL2 and boot without any PSCI firmware, so Linux on an
iPad 5 could not bring secondary CPUs down and up again, and a kexec had
nowhere to park them. CPU hotplug, kexec and a clean poweroff all need a
PSCI implementation that stays resident after m1n1 hands over.

Carry m1n1 patches that keep a small EL3 PSCI implementation resident:
CPU_ON/CPU_OFF with a firmware rendezvous (EL1 WFI/WFE trap to EL3),
SYSTEM_OFF, and a guarded SYSTEM_RESET that boots the default entry
again. The loader marks retained hardware in the device tree after a
PSCI re-entry. The resident PSCI is on by default on these SoCs; the
shipped boot framing keeps psci_reset=0, so SYSTEM_RESET stays off
unless a loader enables it. Adds boot/m1n1/psci-loader and its tests.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
The A9 AOP hosts the accelerometer, gyroscope, pressure and
ambient-light sensors. Its firmware needs boot arguments and per-device
calibration that m1n1 did not pass on S8003, and a reloaded AOP (after
kexec or sleep) lost its sensor firmware state, so the sensors did not
come back.

m1n1 passes the A9 AOP boot arguments and sensor calibration. The kernel
selects a certified AOP reload path through device-tree match data
(bindings, DT, mailbox, pmdomain, IIO and AOP driver patches), keeps the
AOP power domain and firmware state for that reload, and refuses a
suspend that would interrupt the firmware's boot. The J71t firmware
recipe gains the AOP pieces, the gyroscope and pressure drivers are
enabled, and J71t's accelerometer and gyroscope move from missing to
working, and its light sensor from untested to working.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
The TriStar charger subscribed to the PMU's RTC event 16 for source
identification. An acknowledgement there could take the RTC's wake
event, and the polled cover inputs had no driver enabled.

Give RTC event 16 to the RTC alone and keep the PMU and RTC event split
(DT, RTC and USB patches), so one consumer's acknowledgement cannot take
the other's wake event. Enable KEYBOARD_GPIO_POLLED for the polled cover
inputs.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
On J71t the NVMe controller (ANS) sits behind the H9P PCIe port, and
resetting that port loses the controller until a cold boot. A kexec
therefore has to hand over a stopped but powered ANS, with its DART
translations retained, or the next kernel loses its root filesystem. The
first version of this handoff panicked when a step failed, which with
panic=0 is an indefinite hang.

Keep the certified ANS kexec handoff: DT and bindings describe it, the
DART, H9P and NVMe drivers stop queues and translations in order and the
next kernel picks them up, and core/printk export what the modules need.
Every failure on the outgoing side now ends in a native cold reset
through apple_cold_reset() instead of a panic, with explicit returns
after it. The port is handed over powered, with PERST# asserted and the
link stopped, and the incoming kernel brings it up with the cold-boot
sequence (port power cycle, NVMe mode, port enable, then link training).
Retraining the powered port by PERST# alone, as iOS never does, left the
root port in Detect.Quiet and cost a timeout on every warm boot.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
After a kexec the next kernel found the S8000 GPIO controller with
interrupt modes armed by the previous kernel, the PIO SPI controller
with queued transfers and an asserted chip select, and the touch
controller still running. An orphaned GPIO interrupt fires forever and
touch did not come back.

Retire inherited GPIO interrupt modes before the chained parent
interrupts are enabled, drain and idle the S8000 PIO SPI controller on
shutdown and at probe, and stop the Apple Z2 touch controller before its
SPI parent shuts down.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
A Mac grants an iPad more than the standard USB current, but the charger
took the gadget's reported budget unchecked and the composite gadget
could draw the extra current while the bus was suspended.

Cap and log the USB host budget in the TriStar charger, draw no extra
Apple current while suspended, and let the initramfs and orchard USB
helpers opt J71t into the Mac current profile. Documented in
docs/bring-up/charging/j71t-mac-current.md, with tests.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
The USB serial number changed between installed boots and RAM sessions,
so the host saw a new device every boot and could not keep its SSH
known_hosts entry or serial console name.

Share one BOARD/hashed-ID serial format between installed boots and RAM
sessions (usb-identity), and get the per-boot SSH identity through the
existing ACM console. Documented in docs/device-protocol.md, with tests.

Assisted-by: LLM
Bring this branch onto current main and the series it depends on. The
merge keeps the other side's tree; the next commit carries the reworked
version of this branch's change.
A warm restart through kexec needs the next kernel to see the pristine
device tree from boot, the pstore region, the display state and the
power domains that must not be cut while the next kernel starts.

DT, OF, DRM, pmdomain and pstore patches retain the board-selected warm
handoff state, so probe and shutdown keep what the next kernel needs:
the pristine boot device tree, the pstore region, the display state and
the Apple pmgr warm hold. The Orchard service file learns the retained
state.

Assisted-by: LLM
Each render job paid a fixed host cost (serial submission, polling after
events, uninterruptible VM binds), which capped frame rates well below
the GPU's capacity.

Account per-job GPU and host time, reduce serial job overhead, make VM
binds killable and tune post-event polling, record joint latency,
pipeline two render jobs deep, honour fragment-only waits, start the
next job early during the 3D wait, gate TA PM records on dirty state,
and default to the measured pipeline configuration.

Assisted-by: LLM
The GPU blocked every system sleep, and an idle GPU kept its power on
because the firmware was never allowed to park itself.

Let the firmware self-park after bring-up and ring a parked firmware
without the GPC settle wait. Implement a bounded retained system sleep:
register the reservation as a suspend admission, hold clients across it,
answer the readiness query, and refuse sleep with a named reason while
work, a kexec power hold or pmgr's gfx hold is active. pmdomain ends the
warm boot hold at sync_state, the gfx gate is cycled with auto-PM like
pmgr-pwrstate, and the other admission users name their refusals too.

Assisted-by: LLM
The VXD decoder only exposed H.264, so HEVC video was decoded on the
CPU.

Prepare the driver for a second coded format, add the HEVC
command-stream builder and the stateless HEVC decoder, enable it on the
S8000 and T8010 variants, and require a proven firmware stop before
kexec. Adds the video_decode_hevc feature (working on J71t, untested on
the other boards that have the driver, missing elsewhere).

Assisted-by: LLM
S8000/S8003 left the SoC and memory performance state at the boot
default, which bounds GPU throughput by memory bandwidth.

Describe the SoC performance state in the binding and DT, run the SoC
and memory at the high state while active, and restore the low state for
sleep.

Assisted-by: LLM
There was no way to measure memory bandwidth per port on A9, which is
needed to tell GPU work from memory stalls.

Add a binding, DT node and perf driver for the A9 AMC port monitor,
exposing CPU and GFX read/write byte counters, and enable it for
apple-16k.

Assisted-by: LLM
@henrikklee

Copy link
Copy Markdown
Contributor

Thanks for this. We're not taking the memory port counters for now: they're a developer measurement tool with no user-facing feature behind them yet, A9-only, and the CPU write counts are still uncalibrated. The code is kept as tag contrib/brkosebay-j71t-amc-pmu; when the A9 GPU work needs bandwidth numbers, it can come back from there.

@henrikklee henrikklee closed this Nov 12, 2026
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in drivers/usb are renumbered to follow
main's.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, drivers/base and drivers/usb
are renumbered to follow main's.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, drivers/base and drivers/usb
are renumbered to follow main's.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, drivers/base,
drivers/pmdomain and drivers/usb are renumbered to follow main's.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, drivers/base,
drivers/pmdomain and drivers/usb are renumbered to follow main's.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, devices/t8010, drivers/base,
drivers/media, drivers/pmdomain, drivers/power and drivers/usb are
renumbered to follow main's.

Main's versions apply first now, so 0011 in drivers/media keep their
change with a diff regenerated for the new context.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, devices/s8003,
devices/t8010, drivers/base, drivers/media, drivers/pmdomain,
drivers/power and drivers/usb are renumbered to follow main's.

Main's versions apply first now, so 0011 in drivers/media keep their
change with a diff regenerated for the new context.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, devices/s8003,
devices/t8010, drivers/base, drivers/media, drivers/pmdomain,
drivers/power and drivers/usb are renumbered to follow main's.

Main's versions apply first now, so 0011 in drivers/media keep their
change with a diff regenerated for the new context.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, devices/s8003,
devices/t8010, drivers/base, drivers/media, drivers/pmdomain,
drivers/power, drivers/soc-apple and drivers/usb are renumbered to
follow main's.

Main's versions apply first now, so 0011 in drivers/media keep their
change with a diff regenerated for the new context.
brkosebay added a commit to brkosebay/hardware that referenced this pull request Nov 13, 2026
Bring this branch onto current main. Main has since merged OrchardLinux#124, OrchardLinux#122,
OrchardLinux#118, OrchardLinux#121, OrchardLinux#110, OrchardLinux#127, OrchardLinux#101, OrchardLinux#126, OrchardLinux#108 and OrchardLinux#111, several in part or
reworked, and OrchardLinux#123 and OrchardLinux#128 were closed. Where main carries a topic,
main's version is kept; the left-out parts that warm restart and
suspend need come back in OrchardLinux#107 and OrchardLinux#117.

This branch's own patches in devices/s8000, devices/s8003,
devices/t8010, drivers/base, drivers/media, drivers/pmdomain,
drivers/power, drivers/soc-apple and drivers/usb are renumbered to
follow main's.

Main's versions apply first now, so 0011 in drivers/media keep their
change with a diff regenerated for the new context.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants