multikernel: add isolated SR-IOV VF assignment

[nickolaev]

Depends on #5.

Summary

This 15-commit series adds host-controlled SR-IOV VF assignment to multikernel instances while keeping device ownership and privileged hardware programming in the host kernel.

• Carries assigned PCI resource metadata into each instance and enumerates only its assigned functions.
• Filters raw configuration access by exact BDF and mediates supported configuration operations through the host.
• Adds exclusive VF leases and makes resource reservation atomic.
• Constrains VF DMA with a host-owned IOMMU domain.
• Quiesces and resets VFs before release or respawn, then restores the host driver when ownership returns.
• Serializes CPU ownership changes and pins the control route across migration, restart, and teardown.
• Makes the shared IPI transport ordered and recoverable.
• Moves assigned MSI/MSI-X and FLR control into the host process-context control plane.
• Delivers interrupts through preallocated lossless pending mailboxes with masking and coalescing.
• Exposes per-instance transport reliability statistics.

The lifecycle fails closed when assignment, DMA isolation, reset, interrupt programming, or teardown cannot be completed safely.

Validation

• Strict per-commit checkpatch review; only the generic new-file MAINTAINERS warning remains.
• Full multikernel build plus !MULTIKERNEL, PCI_MSI=n, ACPI=n/PCI_MMCONFIG=n, and PCI=n build lanes.
• 102 harness and transport-model unit tests.
• Two fresh sequential QEMU SR-IOV runs covering DMA isolation, concurrent leases, FLR/rebind, restart datapath, repeated lease cycles, hostile lifecycle operations, interrupt delivery, and reliability counters.

[congwang-mk]

Hi @nickolaev

Nice work, thanks for the PR! I finally have time to look into it.

A few high-level comments:

1. If I understand it correctly, it only works with IOMMU off? This looks a big limitation.
2. VF lifetime is coupled to the PF driver with no coordination
3. Exposed bridges are shared and mutable?
4. For the timer fix, it would be better if we could use existing x86 quirks.

[nickolaev]

@congwang-mk thanks for the high-level review. Indeed, this is a naive implementation of "let's make ICMP pass"; It is very fragile as it is. The questions you asked are setting a proper direction. I have some ideas; let me try to clean things up and will ping you once I have a better patch series.

[nickolaev]

@congwang-mk I have updated the commit history with the revised work and the PR message reflects the new state.
Please take a look when you can.

[congwang-mk]

@congwang-mk I have updated the commit history with the revised work and the PR message reflects the new state. Please take a look when you can.

Thank you for doing it. I will take a look during the weekend, since I am still working on the refactoring to prepare for ARM/RISCV (which is also why you got merge conflicts).

[congwang-mk]

Thanks for the update! The layering is a clear improvement: synthetic roots instead of physical bridges, manifest-owned BAR/identity metadata, pci_set_dev_assigned() for the SR-IOV interlock, and prepare/commit/rollback. Main things I think need an answer before this lands:

1. Interrupts are not in the ownership model. The spawn kernel calls disable_acpi(), so no DMAR, no IR, no IO-APIC: it composes compatibility-format MSIs. But mk_pci_iommu_validate_group() requires iommu_group_has_isolated_msi(), i.e. host IR is on, and Linux blocks compat-format interrupts in that case. Nothing allocates an IRTE for the VF. Who owns interrupt remapping for an assigned VF?

2. Fail-closed on IOMMU faults does not work on Intel. report_iommu_fault() is only called by amd/arm-smmu and the embedded drivers; VT-d faults go through dmar_fault(), which just logs. mk_pci_iommu_fault() is dead code on the platform you tested.

3. Teardown deadlock. mk_pci_release_assignment() calls cancel_work_sync(&failure_work) on a path holding mk_instance_mutex (via mk_instance_destroy()), while the work reaches mk_instance_find() through mk_send_message() and takes that same mutex.

4. No VF reset across instance restart. Quiesce runs only at lease release, so a force-halted and restarted instance gets a VF with bus mastering possibly still on and DMA in flight into memory the new boot reuses.

5. The early fixup fires on the host. mk_pci_restore_resources() is PCI_ANY_ID and gated only on root_instance->dtb_data, which baseline sets on the host too. resources_valid is set at commit and never cleared on release, so after one assign/release cycle a host rescan of that VF gets non_compliant_bars = true and stale BARs.

6. Filter is one layer too high. raw_pci_ops/raw_pci_ext_ops are left unfiltered and only pci_root_ops is wrapped. On x86 everything funnels through raw_pci_read(), so filtering there would cover both and make "before hardware access" accurate.

7. Framing. An instance is a full kernel with its own CR3: it can map ECAM directly (bus-granular, so it covers sibling VFs including other instances'), use CF8/CFC, or map any physical page. The config filter prevents accidents, not attacks. The IOMMU domain is the one real boundary, and it constrains the device rather than the instance. Worth saying so explicitly.

[congwang-mk]

Separate from the item list above, a thought about the shape rather than the bugs.

SR-IOV VFs are built assuming something mediates. VF config space is impoverished by design because a hypervisor is expected to synthesize the missing parts from the PF's SR-IOV capability. Multikernel's premise is that there is no mediator, so the series meets a mediator-shaped hole and fills it two ways: the manifest for BARs and identity, which works because those are static, and a config-op filter for everything else, which does not, because the filter lives inside the kernel it is meant to constrain. The interrupt gap is the same mismatch surfacing where it cannot be papered over.

But multikernel already has the right mechanism: a host that owns global state and a message ring to reach it. That suggests splitting along the axis that actually exists:

• Static facts (BAR resources, identity, ECAM geometry): manifest, as you already do. This is the strongest part of the series.
• Shared control planes (config space, MSI vector allocation and IRTE programming): host-mediated over the message ring. Config access is not a fast path, and mediating it makes the filter enforceable rather than advisory. For MSI, the host allocates vectors targeting instance CPUs, programs the IRTEs, and returns address/data; the instance's MSI domain becomes a thin client. That is what a VMM does for a guest, expressed as a host-kernel service instead of emulation.

Keep the synthetic roots and the IOMMU domain either way. The identity-mapped domain in particular fits well: with no second-level translation it range-restricts a device rather than virtualizing an address space, which is simpler than what VFIO needs.

The obvious cost is latency on mediated config access, and I have not measured what that does to something like igbvf at probe time. Worth a quick experiment before committing.

[nickolaev]

Separate from the item list above, a thought about the shape rather than the bugs.

SR-IOV VFs are built assuming something mediates. VF config space is impoverished by design because a hypervisor is expected to synthesize the missing parts from the PF's SR-IOV capability. Multikernel's premise is that there is no mediator, so the series meets a mediator-shaped hole and fills it two ways: the manifest for BARs and identity, which works because those are static, and a config-op filter for everything else, which does not, because the filter lives inside the kernel it is meant to constrain. The interrupt gap is the same mismatch surfacing where it cannot be papered over.

But multikernel already has the right mechanism: a host that owns global state and a message ring to reach it. That suggests splitting along the axis that actually exists:

• Static facts (BAR resources, identity, ECAM geometry): manifest, as you already do. This is the strongest part of the series.
• Shared control planes (config space, MSI vector allocation and IRTE programming): host-mediated over the message ring. Config access is not a fast path, and mediating it makes the filter enforceable rather than advisory. For MSI, the host allocates vectors targeting instance CPUs, programs the IRTEs, and returns address/data; the instance's MSI domain becomes a thin client. That is what a VMM does for a guest, expressed as a host-kernel service instead of emulation.

Keep the synthetic roots and the IOMMU domain either way. The identity-mapped domain in particular fits well: with no second-level translation it range-restricts a device rather than virtualizing an address space, which is simpler than what VFIO needs.

The obvious cost is latency on mediated config access, and I have not measured what that does to something like igbvf at probe time. Worth a quick experiment before committing.

Thanks for the review @congwang-mk
I have updated the branch to handle all the point except the interrupt ownership and the proposed host-mediated PCI configuration. Let me run sme more tests and see if these can fit on top of this branch or we want a follow-up.

[nickolaev]

OK, so the host-mediated PCI config is now in place and IRQ ownership is resolved.

[congwang-mk]

Code review of the SR-IOV VF assignment series. 8 findings inline: 3 that look like real breakage (link failure, atomic-context deadlock, use-after-free), 4 medium, 1 cleanup.

[congwang-mk]

Link failure when CONFIG_PCI_MMCONFIG=n. The five mk_pci_msi_* functions (230, 238, 271, 285, 306) are inside the #ifdef CONFIG_PCI_MMCONFIG block; the #else arm only re-provides mk_arch_snapshot_pci_host_bridges and the x86_init.pci hooks. multikernel.h declares them unconditionally under MULTIKERNEL && X86 && PCI, and drivers/pci/msi/msi.c:244 + irqdomain.c:15,19,31,34,42 call them unconditionally.

PCI_MMCONFIG depends on ACPI || JAILHOUSE_GUEST, so a spawn kernel with CONFIG_ACPI=n (what this series targets) fails to link vmlinux with five undefined references.

[congwang-mk]

Synchronous RPC re-dispatches the whole IPI ring from atomic context. mk_pci_remote_config() and mk_pci_send_irq_request() (:214) busy-wait up to 1s in mk_poll_ipi_messages(), which drains the ring and runs every handler, including mk_pci_irq_forward_handler() -> generic_handle_irq_safe() -> the assigned device's ISR.

Both callers run IRQs-off under a raw spinlock: config accesses come via pci_bus_read/write_config_* holding pci_lock; mk_pci_msi_bind() is reached from __pci_write_msi_msg() under desc->lock with IRQs off during irq_startup().

Deadlock: driver probe does pci_read_config_word(), the poll dispatches a forwarded MSI, that ISR issues a config read -> self-deadlock on the non-recursive pci_lock, same CPU. Even without re-entry, spinning 1s IRQs-off trips the hard-lockup watchdog.

[congwang-mk]

Instance freed after failed resource release -> use-after-free. This only WARN_ON_ONCE()s the error, then kfree(instance) two lines down.

On failure mk_pci_release_assignments() (kernel/multikernel/pci.c:1697-1717) breaks out of its loop, leaving the mk_pci_assignment linked on both instance->pci_assignments (now freed) and the global active list, with assignment->instance dangling. A VF whose FLR times out during teardown gets there; the next BUS_NOTIFY_UNBOUND_DRIVER runs mk_pci_assignment_failure_work(), which dereferences assignment->instance->state and ->name.

mk_instance_destroy() and mk_create_instance_from_dtb() do honor this error; the kref path is the outlier.

[congwang-mk]

Ring consumption is no longer FIFO. The new drain scans all MK_IPI_RING_SIZE slots from tail and consumes anything READY, skipping WRITING. Slot order no longer equals delivery order: A claims slot 5, B claims 6, B publishes first -> 6 is delivered before 5.

mk_vsock_ipi_handler() turns each payload into an skb on a byte-stream socket, so two concurrent senders now corrupt the stream rather than just delaying it. The goal here (an interrupted producer must not block others) is reachable without breaking ordering in the common case.

[congwang-mk]

MSI bind failure is swallowed, leaving a silently dead vector. mk_pci_msi_write_msg() (arch/x86/multikernel/pci.c:271) logs the mk_pci_msi_bind() error with pr_err_ratelimited() and then returns true unconditionally, so __pci_write_msi_msg() stores entry->msg and returns as if programmed.

On RPC timeout or -ESTALE/-ENODEV, pci_alloc_irq_vectors() and request_irq() both succeed while the host never enabled the corresponding vector. The VF driver then waits forever for an interrupt that never arrives, with no error visible to it.

[congwang-mk]

CPU-ownership mutex held across sleeping MSI work. This holds mk_cpu_ownership_lock() across mk_pci_irq_access(), which can msleep(MK_PCI_FLR_SETTLE_MS + 1), pci_alloc_irq_vectors(), request_irq() and free_irq() (synchronize_irq). mk_pci_cfg_work_fn() (:505) does the same.

Every mk_instance_transfer_cpus() / mk_send_cpu_add() / mk_send_cpu_remove() on any instance stalls behind an unrelated instance's MSI setup, and the requester's own 1s deadline can expire while the host is still in the FLR sleep -> spurious -ETIMEDOUT. Only the mk_pci_request_route_stale check actually needs the ownership lock.

[congwang-mk]

Forwarded MSI silently dropped on allocation/backpressure failure. This hardirq handler forwards via mk_send_message_to_instance() -> __mk_send_message(), which does a kzalloc(GFP_ATOMIC) per interrupt and returns -ENOSPC from mk_ipi_ring_claim_slot() when the 64-slot ring is full. The handler only pr_warn_ratelimited()s and returns IRQ_HANDLED.

Edge-triggered MSI never re-asserts, so under memory pressure or ring backpressure the spawn kernel permanently loses that completion (e.g. an igbvf TX/RX cleanup interrupt) and the queue hangs. No retry, coalescing, or pending-interrupt fallback.

[congwang-mk]

Dead stores; the "saved backend" does not exist. raw_pci_ops/raw_pci_ext_ops are set to &pci_mmcfg and immediately overwritten by the filtered ops, and nothing captures pci_mmcfg anywhere.

The commit message says accepted accesses are "forwarded to the saved backend", but mk_pci_raw_read/write always go out as a host RPC. The ECAM windows registered by pci_mmconfig_add() and mapped by pci_mmcfg_arch_init() just above are therefore mapped and never used. Either wire the local backend up or drop the ECAM mapping and these two stores, since as written it reads like there is a local fast path.

[congwang-mk]

I am wondering why you have to handle clock in this PR? If anything is wrong with clock, please separate it out

[congwang-mk]

I read through the full series. The design is sound: the host keeps device ownership, the IOMMU domain is the real isolation boundary rather than config filtering, and the docs say that plainly instead of overclaiming. Generation and epoch tagging is applied consistently, and encoding owner CPU plus slot in the producer gate (so a force-halt NMI cannot strand it permanently) is a genuinely good idea.

Below are correctness findings, then a request about how the series is packaged.

Findings

1. Refcount-0 resurrection in mk_instance_release() (kernel/multikernel/core.c)

The new early return in the kref release callback:

ret = mk_instance_release_resources(instance);
if (WARN_ON_ONCE(ret)) {
        pr_crit("Retaining multikernel instance %d ...");
        return;                       /* not freed, refcount is already 0 */
}

kref_put() has already driven the count to zero, but the instance is still in mk_instance_idr and on mk_instance_list, because mk_instance_destroy() only unlinks after a successful release. The next mk_instance_find() does idr_find() followed by kref_get() on a zero refcount, which trips refcount_warn_saturate("addition on 0"), and the following put frees an object other callers still hold.

This is reachable whenever mk_pci_release_assignments() returns -ETIMEDOUT or -EBUSY, which the series deliberately makes possible (mailbox quiesce timeout, VF still driver-bound). Suggested fix: re-init the reference before returning, or unlink from the idr and list first so the object is unreachable.

2. mk_ipi_ring_publish() spins on a cross-kernel gate with interrupts disabled (kernel/multikernel/ipi.c)

preempt_disable(); local_irq_save(flags);
for (retry = 0; retry < MK_IPI_PRODUCER_RETRIES; retry++) {   /* 10000 */
        old = atomic64_cmpxchg_acquire(&ring->producer_gate, 0, token);
        ...
        cpu_relax();
}

producer_gate lives in shared memory, so the owner may be a CPU belonging to a different kernel. A host CPU can therefore spin 10000 cpu_relax() iterations with local interrupts off waiting on a spawn that is stalled, descheduled, or NMI-parked. Console output reaches this path through vsock, as the old comment in this function noted.

This converts a lock-free MPSC ring that admitted up to 63 concurrent producers into a single global gate, and the failure mode is a dropped message (-EAGAIN) rather than backpressure. The recoverability argument for the gate design is convincing; holding it with IRQs off and spinning on a remote kernel is the part I would push back on.

3. mk_reply_cancel() can permanently strand reply slots (kernel/multikernel/ipi.c)

In the cancel loop, any state that is neither WRITING nor EXECUTING (notably COMMITTED, which an earlier timeout installs) falls out of the loop into the cancelled: label, bumps cancelled_slots, and returns 0, so the caller reports -ETIMEDOUT while the slot is never returned to FREE.

A COMMITTED slot is only reclaimed by mk_reply_publish(). If the host worker never reaches publish (instance force-halted between mk_reply_begin_execute() and publish), that slot is gone for the rest of the epoch. There are only MK_REPLY_SLOTS == 16 and every PCI config access consumes one, so a small number of these wedges config access with -ENOSPC until respawn. occupied_failures will grow with no recovery path.

4. mk_do_cpu_add() can leave a CPU online but untracked (kernel/multikernel/hotplug.c)

After add_cpu() succeeds:

mk_cpu_ownership_lock();
if (root_instance->cpus) {
        ret = mk_cpu_set_add(root_instance->cpus, cpu_id);
        if (ret)
                pr_warn(...);
}
mk_cpu_ownership_unlock();
if (ret)
        goto unlock_transaction;

Two issues. On mk_cpu_set_add() failure the function returns an error with the CPU already online and the rollback mk_hotplug_op never recorded, so nothing will undo it. And when root_instance->cpus is NULL, the if (ret) tests the leftover value from the CPU bring-up call rather than a fresh result. The previous code warned and continued, which was at least self-consistent.

The pre-reserve added just above is the right idea; it just is not carried through to the failure path.

5. XLF_MULTIKERNEL_IPI_V2 through V7 claim six bits of xloadflags

arch/x86/include/uapi/asm/bootparam.h now takes 0x0100 through 0x2000 in the documented x86 boot protocol UAPI word for a private host/spawn handshake, with five of the six values unused. struct mk_shared_data already carries abi_magic and abi_version, and the ELF note now carries ipi_abi_version, so the boot-protocol bit is only needed as a coarse "this bzImage speaks the protocol at all" gate. One bit would do, and I would expect the x86 maintainers to say the same when this eventually goes upstream.

Smaller notes

• mk_pci_msi_prepare() accepts MK_PCI_MSI_FAILED as a valid starting state alongside IDLE, so it re-prepares after a teardown that is known to have failed. The host-side SETUP does retry mk_pci_release_irqs() first, but "failed teardown" is exactly the case where that retry is least likely to succeed.
• x86_multikernel_pci_init() calls panic() when pci_scan_single_device() returns NULL. Fail-closed is right for assignment generally, but a transient enumeration miss now kills the spawn rather than failing the instance.
• mk_instance_confirm_parked() gained an early if (!instance->cpus_on_slot) return 0; ahead of the new mk_ipi_ring_recover_halted() call, so that path reports success without recovering a gate a dead producer may still own.
• DECLARE_PCI_FIXUP_EARLY(PCI_ANY_ID, PCI_ANY_ID, mk_pci_restore_resources) runs on every device on every CONFIG_PCI kernel. It is inert on the host, but a global ANY_ID fixup for a subarch-specific feature would be better gated on X86_SUBARCH_MULTIKERNEL.
• Two counters exist but are unreachable: the spawn's config-access latency (mk_pci_cfg_count / total_ns / max_ns) is only printed once via pr_notice during enumeration, and mk_pci_control_pool_exhausted is incremented and never read. Both belong in the new stats file, especially the latency one given that config access is the hot path.
• kernel/multikernel/pci.c at ~2600 lines covers leases, IOMMU domains, IRQ forwarding, the control-plane message handler, config access, and the transaction machinery. Splitting it along those lines would make it much easier to review. The host/spawn split against arch/x86/multikernel/pci.c is already clean.

Request: split the IPI transport rework out

The commit split within the PR is good, but three commits are standalone infrastructure that happens to ride along with the feature:

• multikernel: make resource reservation atomic
• multikernel: serialize CPU ownership transfers
• multikernel: make IPI publication ordered and recoverable

I would like at least the IPI transport work to land as its own series. It rewrites the shared ring, adds the reply table, bumps the shared ABI to v7, burns a boot-protocol flag bit, and breaks every existing DTB through MK_FDT_COMPATIBLE. That is a larger and riskier change than the SR-IOV feature it is supporting, it affects every multikernel user rather than only those assigning VFs, and findings 2 and 3 are both in it. Reviewed on its own it would get the attention it deserves, and SR-IOV could then be reviewed as a feature rather than as a feature plus a transport rewrite.

The CPU ownership serialization is a similar case on a smaller scale. The mk_cpu_set locking rework in particular (adding the raw_spinlock_t, out-of-lining the three inline accessors, rewriting reserve/add/copy as retry loops) changes an API used throughout the subsystem and stands on its own merits. Worth at least its own commit, separate from the route-pinning changes it enables.

To be clear on why the CPU work is here at all: mk_pci_request_route_stale() makes CPU ownership a security-relevant fact, which does require the mk_cpu_set locking, the route migration in mk_send_cpu_remove(), and the CPU 0 exclusion. That chain is sound. But mk_do_cpu_add() and mk_do_cpu_remove() operate on root_instance->cpus, which the route validation never reads, so the transaction and ownership locking there is consistency work rather than a prerequisite. That is also where finding 4 crept in.