Repository navigation
fix(apic): stay in xAPIC mode on bare metal unless the firmware handed over x2APIC - #797
Merged
Merged
Conversation
apic_choose_mode now names the mode and its reason from what CPUID and IA32_APIC_BASE report, and the bootstrap processor's choice is applied by every processor. It still keeps today's rule, x2APIC whenever offered, so the new checks fail on a bare-metal bench: - interrupt_controllers.bare_metal_keeps_xapic_unless_the_firmware_handed_over_x2apic walks the eight platforms three bits make; - interrupt_controllers.the_bootstrap_processor_runs_the_mode_this_machine_gets compares the mode with the hypervisor bit; - interprocessor_interrupts.every_processor_runs_the_mode_the_bootstrap_processor_chose asks each application processor for its own IA32_APIC_BASE. Refs #795
…d over x2APIC Without interrupt remapping, no device interrupt reaches a local APIC in x2APIC mode on bare metal (Intel SDM vol. 3, 325384-093US, 13.12.6 and 13.12.7), and the MSI code refuses to aim one there. The bootstrap processor now keeps the x2APIC mode its firmware handed over, turns it on only under a hypervisor, and otherwise stays in xAPIC mode; every application processor follows it. Each prints the mode and its reason. Refs #795
In xAPIC mode a destination holds 8 bits and 0xFF broadcasts, so a processor whose APIC ID is 0xFF or more cannot be started. Its start-up used to fail three times with nothing sent. It is now refused before any interrupt, with one line that names it and why, and the smp record counts it as ap_unaddressable; it stays offline, so the record still fails. Refs #795
…ocessor's APIC ID on its line The [LPLTLM] machine record gains apic_reason, the name apic_mode_reason_name gives the mode apic_get_mode returns, so the reason is readable by a tool and apic_get_mode has a caller outside the tests. Each processor's line now ends with its APIC ID, so a boot shows which processor ran which mode. The documentation of the modes says what Linux v6.17 decides instead of paraphrasing it: x2apic_disable leaves the firmware's x2APIC mode unless the bootstrap processor's ID is 255 or more or the mode is locked, and Linux trusts only a hypervisor whose guest support sets x86_init.hyper.x2apic_available, which this kernel assumes of every one. Refs #795
…enumeration's value The enumeration's value changes meaning when its order does; whether the firmware handed over x2APIC mode does not. The smp counter's note is shortened so that its declaration stays on one line. Refs #795
# Conflicts: # kernel/arch/i386/cpu/apic.c # kernel/include/kernel/cpu/apic.h
…APICs run The boot chooses the model it tells firmware before the local APICs are initialised, from the mode apic_runs_x2apic_mode predicts; the test computes that mode's delivery apart, from the mode they now run and the hypervisor bit.
…h left the MSI module #796 moved message_signaled_interrupt_allocate and _release to device_interrupt_allocate and _release; the test of every processor's mode still called the old names.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #795
What changed
On bare metal (CPUID.1:ECX[31] clear) the local APIC stays in xAPIC mode unless the firmware already set IA32_APIC_BASE.EXTD; under a hypervisor x2APIC is turned on as before. The bootstrap processor chooses (
apic_choose_mode), every application processor applies that mode, and each says why:CPU APIC mode=xapic-active, reason=bare-metal-without-interrupt-remapping, apic_id=1, andapic_reason=in[LPLTLM] machine. An AP the mode cannot address (0xFF and up in xAPIC) is refused by name and counted asap_unaddressablein[LPLTLM] smp.Why
Without interrupt remapping no device interrupt reaches x2APIC mode on bare metal (Intel SDM vol. 3, 325384-093US, §13.12.6-7), and
device_interrupt_platform_deliversrefuses to aim one there.With main
Since #799 the boot chooses the interrupt model it tells firmware before the local APIC is initialised, from
apic_runs_x2apic_mode. That prediction now follows this rule: beforeapic_initialize_on_cpu, the modeapic_choose_modewould choose; after it, the chosen one.the_interrupt_model_follows_the_mode_the_local_apics_runchecks the model against the mode the local APICs run, computed apart; the old prediction fails it under-cpu max,-hypervisor. Merged with main, every boot ends with 0 fail:-cpu max,-hypervisor; pc 854;How to verify
./build.sh --server, an image frompython3 -I tools/nvme-test-image.py nvme.img, thenKTAP
# Totals, the same command with the row's-machine/-cpu:-cpu max,-hypervisoreach_queue_interrupts_the_core_its_entry_names)-cpu max-hypervisorAt 7f0e459, which keeps today's rule,
bare_metal_keeps_xapic_unless_the_firmware_handed_over_x2apicandthe_bootstrap_processor_runs_the_mode_this_machine_getsfail;every_processor_runs_the_mode_the_bootstrap_processor_chosefails once an AP keeps the mode it woke in. Under qboot (-bios /usr/share/qemu/qboot.rom), which leaves xAPIC with IDs 0xFF and 0x100, both APs are refused by name.Not in it
-cpu maxor the default CPU; not seen in 30 boots of the branch, not investigated.