Repository navigation
Put virtio-serial on the O2 - #424
Draft
jamesbraid wants to merge 13 commits into
Draft
jamesbraid wants to merge 13 commits into
jamesbraid wants to merge 13 commits into
Conversation
Fuel uses two independent SCSI buses behind one ISP12160 function. Reuse the ISP1040 command engine and let the part trait own the firmware, register and channel differences. A bus reset reaches every attached target as unit attention rather than a media change. Keep the reset attention on the shared target interface so a controller can notify each device through the same bus. The ISP12160 is its only caller: the WD33C93A, AIC-7880 and ISP1040 models do not call it, so their targets behave as before. A bus reset restores each disk's write cache bit per channel: a reset of one channel leaves the other's disks alone, and a chip or soft reset restores both. The engine changes reach the Octane's ISP1040: mailbox commands now run with dispatch held, and the target interface gains the bus reset. The Octane's IRIX 6.5.30 SCSI disk (Octane2, R10000, 256 MiB, V12, jitv2, release build, headless, NAT) boots to the login prompt and runs commands on this branch's tip and on main c6dea8c: login after 126 s on the branch and 109 s on main in one run each (host load makes the times indicative only), the same `hinv` listing of both integral controllers and the disk, and the same command output.
MacePci::attach and Bridge::attach each repeated the same four steps: hand the function its way to memory, read its interrupt pin, build the line for that pin if there is one, and connect it. They differ only in the DMA object and in what a pin is wired to. pci::plug takes those two as arguments and does the rest, so a bus that gains a way to place a function at a device number does it in one place. No behaviour change: the same objects are connected in the same order, and a function with no pin is still left unconnected.
MacePci kept each pin's level and ORed the pins routed to one CRIME input in a private router whose routing was a fn pointer baked in at construction. The Fuel and Tezro buses carry copies of the same logic. IntxTable is that logic as a type in pci.rs, with the routes held as data (IntxTable::route) instead of a function. A bus can then wire a function at a device number it learns about after construction without changing the bridge. MacePci fills the table from its existing routing function, so the O2 routes exactly what it did: slot 1 INTA to CRIME 8, slot 2 INTA to CRIME 9, everything else dropped. Levels, the snapshot "pins" array, the pci status line and the order in which a shared input is driven are unchanged. The table also drops the per-change scan that called the routing function for all 128 lines; the scan now reads the stored routes.
PCI functions need different writable and write-one-to-clear header fields. Describe those fields in one construction policy while keeping the current header defaults for existing functions. Forward narrow Bridge configuration writes with their byte enables so write-one-to-clear bits are handled correctly when a function supplies them. Existing Octane functions do not use that status policy, so their behavior stays unchanged.
Which device numbers on a machine's PCI bus take an add-in card is a fact about the chassis, not something a configuration should be able to invent. Describe it once, as a const table per bus: each device number is either an onboard function the machine wires itself, or an add-in slot with how well its number is known and the public manual or driver it comes from. Only an established or observed number can be filled; the rest can be listed with what would settle them. The O2 has one add-in slot, device 3, which Linux and OpenBSD agree on and the hardware guides describe as a single PCI board position. The Octane's Bridge has four onboard devices and no add-in slot: its PCI boards hang off an XIO port behind a Bridge of their own, and neither the port nor the device numbers are established, so that table says so instead of guessing. The O2's `pci slots` monitor command prints the table with what is fitted and where each add-in pin goes. Nothing routes slot 3 yet, so it reports INTA to INTD unrouted.
A fitted card needs more from a machine than a slot: the SCSI buses it brings, a worker to start and stop, state to save. Give every card the same small interface and list the kinds in one static table, each entry building a fresh card, so a machine can fit several of the same kind. Nothing here depends on a particular bus. The first card is a dual-channel PCI SCSI adapter built on the QLogic ISP12160. It reuses the ISP12160 model added just before it in this series and brings two wide SCSI buses. It takes no options: nothing about the part is configurable on the board.
Turn a card entry into a placement or a message that says what to fix. An entry names a bus (optional on a machine with one), a slot, a card and the card's own options. The slot is a PCI device number or the label printed on the chassis; labels are resolved here and nowhere else, so the buses, snapshots and the monitor only ever see device numbers. Checks run in the order a reader would fix them: the machine has PCI, the bus exists, the slot is an add-in slot and its number is known, the slot is free, the card exists, its options are ones it takes. Onboard devices are named in the message instead of reported as missing, because they are real and cannot be replaced. Several cards of one kind are allowed; their SCSI buses are numbered after the machine's own, in entry order. The command-line form is `[BUS:]SLOT=CARD[,KEY=VALUE]...`. An entry on the command line replaces a configuration entry for the same slot.
Let a configuration fit a card in an add-in slot: one `[[pci]]` table per card with `slot`, `card` and the card's own options inline, or `--pci [BUS:]SLOT=CARD[,KEY=VALUE]...` on the command line. Validation runs with the rest of the configuration, so a mistake stops the run before any machine is built. The card's SCSI buses become controllers after the machine's own, so `[scsi.N]` can put a disk on them: controllers 2 and 3 for the first ISP12160 on an O2. Their ids go to 15, since the card's buses are wide; the O2's own buses stay at 7. Without a card the controller limits and their message are as before.
Wire configured cards into the O2. The bridge routes the slot's INTA to CRIME input 10 and INTB to INTD to the shared inputs 13 to 15, as the MACE specification's PCI interrupt tables have them, and Linux, OpenBSD and NetBSD agree; a shared input is the OR of every pin routed to it. Whether the board's connector wires INTB to INTD is not known. This is the only change a machine without a card sees, and nothing answers in the slot then. The machine builds the cards before it attaches `[scsi.N]` disks, so a disk can sit on a card's bus (controllers 2 and 3 for an ISP12160), then hands them to the bus, which attaches them and owns their lifetime: they start and stop with the onboard controllers, their disks are flushed and synced with the others, their copy-on-write overlays are saved by controller number, and their state is part of the bus's saved state. Restoring state saved with a different number of cards is refused.
A snapshot taken with a card in the slot holds the card's registers and its ring state. Restored into a machine without the card, that state has nowhere to go; restored the other way round, the card would sit at power-on under a kernel that has been driving it. Neither is a state the machine was ever in, so list the fitted cards in the manifest, by bus, device number and card, and compare before touching anything. The O2 refuses with a line per difference. Device numbers, not chassis labels, are recorded, so relabelling a slot later cannot break an old save. The key is optional and left out when nothing is fitted, so snapshots without cards read and write as before and the schema version does not change; an older build restoring a save with cards ignores the key and fails later on the saved slot instead. The check does not honour IRIS_SNAPSHOT_SKIP_CHECK.
The O2's IRIX has no driver for the ISP12160: it finds the card on the PCI bus and lists it as an unclaimed adapter, but has nothing to bind, so no controller or disk shows in `hinv`. The Adaptec AIC-7880 is the chip the O2 already drives onboard, and it also came on PCI cards, so the same driver claims one in the add-in slot and the guest exercises the slot end to end: configuration, BARs, DMA and the interrupt on CRIME input 10. The card is the existing controller model with one bus, numbered as the next controller after the machine's, which is why a card is now built knowing its first controller number.
Describe `[[pci]]`, `--pci`, the O2's one slot and its interrupt wiring, the two cards and how a card's SCSI buses are numbered, the `pci slots` monitor command and the snapshot rule. Say which card IRIX 6.5.30 drives: the AIC-7880 shows in `hinv` with its disk, the ISP12160 is found on the bus but has no driver.
`[virtio_serial]` was refused on the O2 because the MACE bridge had no interrupt route for a spare device. The MACE specification gives device 4 its INTA on CRIME input 11, and a stock board leaves that device empty, so the paravirtual device goes there and its pin is routed. It is iris's own device, not a slot: it is not in the add-in slot table, so `pci slots` does not list it and `[[pci]]` cannot name it, and it coexists with a card in device 3. The Octane's version of the device is the same function on its Bridge; the O2's starts and stops with the other devices and resets with the machine. Only INTA is routed: the other pins of device 4 are disputed between the kernels. A snapshot taken with the device loads only into a machine that has it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #422 (the O2 PCI add-in slot): the diff against it is one commit,
ip32: put virtio-serial at PCI device 4. Review that one; rebase onto main once #422 lands.[virtio_serial]was refused on the O2 because the MACE bridge had no interrupt route for a spare device. The MACE specification gives device 4 its INTA on CRIME input 11, and a stock board leaves device 4 empty, so the paravirtual device goes there, as it sits on the Octane's Bridge, and its INTA is routed. Only INTA: the MACE specification gives device 4's other pins too, but Linux and NetBSD route them differently, and the device needs only INTA.It is iris's own device, not a slot. It is not in the add-in slot table, so
pci slotsdoesn't list it and[[pci]]can't name it, and it sits beside a card in the real slot (device 3). It starts and stops with the other devices and resets with the machine. A snapshot loads only into a machine that matches it: one taken with the device is refused by a machine without it, and one taken without it by a machine with it. The device itself keeps no state across a restore, as on the Octane.The refusal in
config.rsis gone, anddocs/virtio-serial.md, README.md, FEATURES.md and the changelog are updated.Tested: device 4's INTA reaches CRIME 11 (and a card's INTA on CRIME 10 is unaffected); the device enumerates at device 4 with a card at device 3 and both coexist; the snapshot rule in both directions; the machine starts and stops the device, and a cold reset resets it (each fails if its call is removed); release and debug lib tests. IRIX 6.5.30 boots on an O2 with
[virtio_serial]enabled and an AIC-7880 card, lists the device as "PCI Adapter ID (vendor 0x1af4, device 0x1043) PCI slot 4", andhalt -ppowers off.Not guest-tested: no Linux O2 image is available, so a driver has never talked to this device on the O2. The transport is the one the Octane uses.
Performance
No code changes for an O2 without
[virtio_serial].