Conversation
Snapshot sections, device suspend callbacks and several device implementations were already in place, but nothing walked a machine to drive them. rvvm_machine_snapshot() serializes RAM, every hart and every device in a fixed section order, in whichever direction the snapshot was opened. Hart state covers registers, FPU and vector state, CSRs, translation and privilege mode, LR/SC reservation, AIA register files, timer comparators and raised interrupts. Address translation and compiled blocks are caches over guest memory and are dropped on resume. RAM size, hart count and XLEN are stored and checked, so a snapshot only resumes into the machine layout it was taken from. Legacy MMIO devices gain the suspend callback the region device API already had, forwarded through the region wrapper, which lets existing device implementations participate.
Command, status, interrupt line, bridge windows, power management, MSI and MSI-X state of every function, and the address each BAR is mapped at, so that regions are placed back where the guest expects them on resume.
Submission and completion queues, controller configuration, interrupt mask and temperature threshold. IO workers are drained first, so queues are not read or overwritten while commands are in flight. The backing block device is not part of a snapshot and must be restored to a matching state by the caller.
Resume a machine from a snapshot before it is started, and write one out after it shuts down.
Resuming skips the machine reset, which is the only place the timer frequency is set up. Harts then waited in WFI for a timer that never advanced, leaving the guest frozen with its display intact. The timebase now travels in the snapshot, and the machine section carries a version so that a snapshot of an older layout is rejected rather than misread.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The snapshot container and the device suspend callback were already in
place, but nothing walked a machine to drive them, so snapshots couldn't
actually be taken.
rvvm_machine_snapshot() serializes RAM, the harts and every device, in
whichever direction the snapshot was opened. RAM size, hart count, XLEN and
the timebase are stored and checked, so a snapshot only resumes into a
machine it fits. TLB and compiled blocks are dropped on resume since
they're just caches over guest memory.
Legacy MMIO devices gained the suspend callback the region API already had,
so the existing implementations take part. I added suspend for three more
that a typical machine needs: PCI config state including BAR placement,
NVMe queues, and the OpenCores I2C controller. -loadsnap and -savesnap make
it usable from the CLI.
Tested by booting a Linux guest with a framebuffer, NVMe disks and PCIe,
saving it, rebuilding an identical machine and restoring: the guest carries
on from where it left off.