Internal USB driver binding, hot-plug and unplug by valentinbreiz · Pull Request #3262 · CosmosOS/Cosmos

valentinbreiz · 2026-09-26T16:27:04Z

Stacked on #3260 (base feature/driver-kit-engine), which sits on #3259, #3258 and #3257. Retarget it to gen3 once those merge.

This is the fifth step towards letting a user kernel register its own PCI and USB drivers. It adds USB binding to the driver engine from #3260: a registered USB class driver is offered every interface the built-in drivers left, both at boot and when a device is plugged in later. When the device is pulled out, its binding is torn down.

Everything stays internal. Each type already has its final name, namespace and signature, so the PR that opens the seam only needs to make them public and mark them experimental (COSMOS0003). There is no public API change and no PublicAPI.*.txt change.

Problem

No USB driver can be added. UsbManager offers each interface to a hard-coded list (hub, keyboard, mass storage). A kernel has no registration point, and nothing hands a driver a USB interface, its endpoints or its control pipe.
Two internal names clash with the seam. The internal UsbDriver, UsbEndpointType and UsbTransferStatus have the names the seam needs, so both sets could not coexist in HAL without CS0104 errors.
A USB binding has no end. When a device is pulled out, a built-in lets go through Disconnect. Nothing withdraws a mouse or a network link from System's managers: MouseManager and NetworkManager only ever register.
A network handle outlives its device. NetworkAdapter carried the registration index. Once a device leaves and another takes its slot, an old handle names the new device.

Fix

Renames (no behaviour change):

The internal UsbDriver becomes UsbClassDriver.
UsbEndpointType and UsbTransferStatus move unchanged into Cosmos.Kernel.HAL.Drivers.Usb.

Seam (Cosmos.Kernel.HAL.Drivers.Usb, internal):

The driver side: UsbDriver (Probe, and Remove for unplug), UsbDriverRegistration and UsbMatch.
UsbDeviceContext:
- ControlIn and ControlOut, in thread context;
- during Probe only, OpenInterruptIn and TryOpenBulk;
- UsbTransferResult.Length reports the bytes the transfer actually moved.
Supporting types: UsbDeviceInfo, UsbInterfaceInfo, UsbEndpointInfo, UsbBulkPipe, UsbReportHandler, UsbDirection, UsbRequestKind and UsbRecipient.
Every type has its final shape.

Registration and binding (DriverCore, KitUsbDriver):

Registering. DriverCore.Register(UsbDriverRegistration) follows the PCI rules.
- PCI and USB drivers share one set of names, so a name identifies one driver in the device list.
- Built-in names are refused: PCI owners, plus "hub", "HID boot keyboard" and "mass storage".
The pass. After the PCI functions, the pass offers every interface that no built-in bound, one device and one interface at a time.
- Ranking: Device(v,p) > Interface(c,s,p) > (c,s) > (c); ties go to the earlier registration.
- Every decision is logged, e.g. [Drivers] usb/1-5:1.0 -> usb-boot-mouse (interface match).
- Paths follow Linux's scheme: usb/<bus>-<root port>[.<hub port>...]:<config>.<interface>.
Hot-plug. KitUsbDriver is the last class driver in UsbManager's list, after the hub, keyboard and mass-storage built-ins, so built-ins keep first pick of every device, at boot and on hot-plug.
- It stays out of the way until the pass has run. After that, a device the hot-plug thread enumerates gets the same ranked offering.
- UsbManager's log names the registration that bound an interface.
Reports. OpenInterruptIn wraps the handler in a trampoline that stays disarmed until Bound. A report that arrives before Bound, after a failed attempt, or after unplug is dropped, never buffered. The report path does not allocate or take locks.
No fall-through. xHCI cannot close a pipe, and an interrupt pipe starts transferring as soon as it opens. So an attempt that opened a pipe and then declined, failed or threw ends the offering of that interface, and a line is logged. An attempt that opened nothing falls through to the next candidate, as with PCI.
Teardown of a declined or failed attempt:
1. Disarm the report trampolines.
2. Drop the publications and work items, and cancel the events.
3. Invalidate the pipes and the context.

Unplug (DriverCore.RemoveUsbBinding). This runs on the hot-plug thread, before ReleaseDevice frees the pipes:

IsPresent turns false, and the report trampolines are disarmed.
The published mouse and link are marked withdrawn and leave their managers, through withdraw hooks System installs next to the existing publication hooks.
Work items are dropped and events cancelled. It waits up to 1 s for a work item of this binding that is running at that moment (DriverWorkQueue.IsRunningItemOf), and logs an overrun.
The driver's Remove runs inside try, with the re-entrancy guard set.
The context moves to Removed.

After unplug, transfers return Disconnected.

System:

MouseManager keeps its mice in a copy-on-write array. UnregisterMouse disables the mouse, clears its handler and releases the buttons it may have held.
NetworkManager.UnregisterDevice:
- it compacts the table with interrupts masked, then publishes the new count;
- it moves the primary to the first remaining device;
- it removes the device's address-map, MAC-map and IPConfig entries.
IPConfig now publishes an array it replaces whole, so a thread walking it while a device leaves no longer throws "Collection was modified".
PublishedNetworkDevice checks for withdrawal again with interrupts masked. A send or delivery that was preempted just before the unplug then never reaches the driver or the stack afterwards.
NetworkAdapter carries a generation stamp instead of the index. A handle names only the device it was taken for, and IsValid turns false once that device leaves. The public signatures are unchanged; Equals and GetHashCode now use the stamp.

Device list. DriverCore.Devices now lists every USB interface after the PCI functions: path, owner (a built-in's name, a registration's name, or none), vendor and product IDs, and class triple. The hot-plug thread rebuilds it after every enumeration and every disconnect.

Host controller. One generic change: xHCI now reports how many bytes a device-to-host control transfer moved (Interrupt-on-Short-Packet on the data stage, the way Linux does it). Built-in callers are unchanged.

Test runner:

Device ids. Each device on a profile's "usb" list gets the QEMU id usbdev<n>. The USB stick, drive and controller ids are unchanged.
New guest requests, with helpers in TR:
- usb-device-unplug n and usb-device-plug n;
- usb-pointer-move n dx dy [buttons], which moves one USB mouse.
How pointer input is routed. QEMU 10.2.2 aborts the VM when input-send-event names a USB device (Property 'qemu-fixed-text-console.device' not found). The runner therefore selects the mouse with mouse_set and sends the events without a device.
Profiles and QEMU arguments.
- A profile with any "usb" device now also gets a QMP monitor.
- usb-tablet joins the "usb" allow-list.
- The Drivers usb-mouse profile becomes usb-hid, with usb-mouse, usb-tablet and usb-kbd.

Verification

Build. make setup gives 0 errors, and no warning falls on a line this PR adds or changes.

Host tests.

Cosmos.Tests.Patcher 188 / 0 failed. This includes 50 new tests: the request parsing, the exact QMP JSON against a fake QEMU monitor, and the ids.
Cosmos.Tests.Tools 37 / 0.
Cosmos.Kernel.Tests.System 208 / 0.

QEMU arguments. I compared old and new argument lines for all 166 combinations of suite, arch, profile and modifier. The 158 that don't use a "usb" list are byte-identical, including every Storage USB-stick cell. The 8 Drivers usb-hid lines differ only by the added device ids and the QMP monitor.

Drivers suite. 80 tests per cell (47 before), final run:

Cells	x64	arm64
Total (tests / failed / skipped)	320 / 0 / 203 (4 cells)	1200 / 0 / 801 (15 cells)
usb-hid	41 passed	41 passed ×3 (bare, gicv2 with polled xHCI, gicv3 with MSI-X through the ITS)
edu, nvme, rtl8139	39, 19, 18 passed	same ×3
e1000e-arm64	n/a	16 passed ×3

The new usb-hid cells check:

Registration: a name another driver already uses, invalid registrations, Register after the pass or from a USB driver's callback.
Ranking: a device match that declines falls through to the boot-mouse driver; the class-only catch-all is never offered the interface.
The bind: SET_PROTOCOL; ControlIn reads the 18-byte device descriptor (0627:0001); members that are Probe-only throw after Probe; endpoints of another interface are refused; the published mouse reaches MouseManager.
Reports only after Bound: the test driver requests 4 ms idle reports during Probe, and none reaches its handler before Bound.
No fall-through: the tablet's first driver opens its endpoint and fails, and the lower-ranked match is never offered it. The log reads: [Drivers] usb/1-6:1.0 -> no driver: usb-tablet-fails opened an endpoint, which cannot be closed, so no other driver is offered the interface.
End-to-end input: usb-pointer-move 0 12 7 1 moves the pointer by exactly (12, 7) with the left button held. The driver's own running sums show the movement came through it, not through x64's PS/2 mouse.
Unplug:
- Remove runs once, after the withdrawal;
- the mouse leaves MouseManager with its button released;
- the old context answers Disconnected;
- work items and events are cancelled;
- the interface leaves the device list.
Replug:
- the factory runs again on the hot-plug thread, with a new driver instance;
- the new mouse is registered and listed;
- reports reach the new binding.
Network withdrawal:
- on the tablet's replug, a link-publishing driver binds after the first one declines;
- on its unplug, the link leaves NetworkManager and the primary is fixed (the link is primary on arm64 bare and gicv2);
- an old NetworkAdapter does not name the next link that takes its slot.
Races, repeated 100 times each: a send or delivery that was preempted just before the unplug never reaches the driver or the stack afterwards, and a route lookup survives the removal.
Built-ins first: the built-in keyboard driver keeps its interface at boot and on replug, although two user registrations match it.
Device list: it lists every USB interface with its owner.

Each rule was checked by breaking it. The implementers and the fixer removed or changed, one at a time:

at bind time: arming at open, the disarm in teardown, the no-fall-through return;
during unplug:
- the work-item wait;
- the withdrawal;
- the event cancel;
- IsPresent;
- the device-list rebuild;
- the hot-plug bind;
- Remove;
the placement and removal paths:
- the generation stamp;
- the kit's place after the built-ins;
- UnregisterMouse;
- the network stack's cleanup;
- the link's withdrawal;
- the primary fix;
the review fixes: the re-check before a send and before a delivery, the IPConfig array, the button release.

Each time, the cells written for it failed, and no unrelated cell did.

Other suites, QEMU. Same totals as #3260, test by test:

x64: HelloWorld, Memory, GarbageCollector, Threading, Timer, Interrupts, Pci, Virtio, Storage (6 cells, all USB hot-plug cells included), Network (2), Graphic (3).
arm64: HelloWorld, Memory, Threading, Interrupts (3), Pci, Virtio (2), Storage (18), Network (2).
The one exception is GarbageCollector on arm64, which fails GC_InteriorPointerRoot as it does on Internal driver engine and PCI driver binding #3260. Fix the GC freeing objects rooted only by an interior pointer #3261 fixed it on gen3, and this stack picks the fix up when it is rebased.

USB off. A HelloWorld kernel with Keyboard and Storage off, so USB is off by the SDK's cascade, still passes 3/3 on x64 and arm64. Its image holds no symbol of KitUsbDriver, the USB seam, the trampoline, UsbManager or the engine's USB paths; with USB on, the same kernel holds about 60.

Real hardware. The runs above are QEMU only. A USB mouse on a real xHCI still needs checking.

Known gaps.

The MAC-map cleanup on unplug has no cell: that map is only reachable through an internal HAL interface the test kernel cannot name. The address-map and IPConfig cleanup are checked.
Only a deliberate break of the code in testing exercises the 1 s overrun branch of the work-item wait.
Disarming the trampolines at unplug has no cell of its own: once the device is gone, no report can arrive to observe. The disarm on a failed attempt is covered.
The network stack still takes no lock. A lookup in thread context that is preempted between finding a map entry and reading its value could see the device leave in between, a window of a few instructions. Closing it needs a stack-wide change.
NetworkManager still has 8 fixed slots, and a 9th device is dropped silently. This predates this PR; hot-plug reuses the freed slots.
On x64, a device plugged back in takes about 1.5 to 2.2 s to bind. The Storage stick takes as long, so the delay is in port reset and enumeration, not in the kit.
The Drivers suite now holds temporary InternalsVisibleTo grants from HAL, Core and System. The PR that opens the seam removes them.
QemuLauncher.UsbDeviceId now returns the "usb" list id (usbdev<n>). The stick id moved to UsbStickId, and every in-tree caller is updated.

Found here but not changed:

UsbBulkOnlyTransport.Open leaves bulk IN open when opening bulk OUT fails, and the interface then goes to the next driver. It is harmless today only because xHCI accepts reopening an endpoint.
ProfileCatalogTests.DriversSuiteCoversTheGicVersionsItsCellsNeed was missing Internal driver engine and PCI driver binding #3260's nvme cell, which is why Internal driver engine and PCI driver binding #3260's host tests are red. This PR's version of that line includes it.

A device published by a USB driver leaves when it is pulled out, so the managers gain the paths that let it go: - MouseManager keeps its mice in a copy-on-write array; UnregisterMouse disables the mouse, clears its handler and releases held buttons. - NetworkManager.UnregisterDevice compacts the table with interrupts masked, moves the primary to the first remaining device, and purges the device's address map, MAC map and IPConfig entries. - IPConfig publishes an array it replaces whole, so a thread walking it while a device leaves never sees the collection change under it. - NetworkAdapter carries a generation stamp instead of the index, so a handle never names a device that later takes its slot.

…em on unplug The internal UsbDriver becomes UsbClassDriver, and UsbEndpointType and UsbTransferStatus move unchanged into Cosmos.Kernel.HAL.Drivers.Usb, next to the seam: UsbDriver, UsbDriverRegistration, UsbMatch and UsbDeviceContext (control transfers, interrupt IN and bulk pipes). DriverCore registers USB drivers in the same name space as PCI ones. The pass offers every interface no built-in bound, most specific match first; KitUsbDriver, last in UsbManager's list, does the same for devices plugged in later, so built-ins keep the first pick. - Reports reach a driver only after Bound, through an armed trampoline. - An attempt that opened a pipe ends the offering of its interface: xHCI cannot close a pipe. - On unplug, before the controller frees the device: the handlers are disarmed, the published mouse and link withdrawn through sinks System installs, work items and events cancelled, then Remove runs. The device list gains one record per USB interface, rebuilt after every enumeration and disconnect. xHCI reports how many bytes a control transfer moved. Every type stays internal in its final shape until the seam opens.

The test runner can now unplug and replug one device of a profile's "usb" list (usb-device-unplug/usb-device-plug n) and move a USB mouse (usb-pointer-move n dx dy [buttons]). Each such device gets the QEMU id usbdev<n>; mouse_set routes the input, because input-send-event with a USB device id aborts QEMU 10.2.2. The usb-mouse profile becomes usb-hid (mouse, tablet, keyboard). Per cell, the suite checks: - USB registration rules, ranking and fall-through after a decline; - the boot-mouse bind, ControlIn, reports only after Bound, and a real pointer move through the driver; - no fall-through once a failed attempt opened the tablet's endpoint; - unplug: Remove after withdrawal, the mouse and a published link leaving their managers, Disconnected transfers, cancelled work; - replug: a new driver instance binds on the hot-plug thread; - the built-in keyboard keeping its interface, and the device list. System grants the suite temporary internals access, as HAL does.

github-actions · 2026-09-26T16:37:26Z

🧪 Drivers Tests

Cell	x64	arm64
all	⚠️ 117/320, 203 skip	⚠️ 399/1200, 801 skip
e1000e-arm64	n/a	⚠️ 16/80, 64 skip
e1000e-arm64+gicv2	n/a	⚠️ 16/80, 64 skip
e1000e-arm64+gicv3	n/a	⚠️ 16/80, 64 skip
edu	⚠️ 39/80, 41 skip	⚠️ 39/80, 41 skip
edu+gicv2	n/a	⚠️ 39/80, 41 skip
edu+gicv3	n/a	⚠️ 39/80, 41 skip
nvme	⚠️ 19/80, 61 skip	⚠️ 19/80, 61 skip
nvme+gicv2	n/a	⚠️ 19/80, 61 skip
nvme+gicv3	n/a	⚠️ 19/80, 61 skip
rtl8139	⚠️ 18/80, 62 skip	⚠️ 18/80, 62 skip
rtl8139+gicv2	n/a	⚠️ 18/80, 62 skip
rtl8139+gicv3	n/a	⚠️ 18/80, 62 skip
usb-hid	✅ 41/80, 39 skip	✅ 41/80, 39 skip
usb-hid+gicv2	n/a	✅ 41/80, 39 skip
usb-hid+gicv3	n/a	✅ 41/80, 39 skip

Skipped

x64: 203 skipped: this cell attaches no USB device (96), this cell attaches no edu device (72), this cell attaches no NVMe controller (12), this cell attaches no RTL8139 (9), this cell attaches no unclaimed PCI function (5), this cell attaches no unclaimed 82574L (4), this cell attaches no unclaimed Ethernet function (3), this cell attaches neither edu nor an NVMe controller (2)
arm64: 801 skipped: this cell attaches no USB device (384), this cell attaches no edu device (288), this cell attaches no NVMe controller (48), this cell attaches no RTL8139 (36), this cell attaches no unclaimed PCI function (15), this cell attaches no unclaimed 82574L (12), this cell attaches no unclaimed Ethernet function (9), this cell attaches neither edu nor an NVMe controller (9)

📎 Artifacts

Architecture	Test Results	UART Log	Kernel ISO
x64	XML	Log	ISO
arm64	XML	Log	ISO

Conversation

valentinbreiz commented Sep 26, 2026

Problem

Fix

Verification

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Drivers Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Fat Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 GarbageCollector Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Graphic Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Math Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Memory Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026

🧪 Network Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026

🧪 Pci Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Power Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Runtime Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Storage Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Threading Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026

🧪 Timer Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 Virtio Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 27, 2026

🧪 File Tests

📎 Artifacts

Uh oh!

github-actions Bot commented Sep 27, 2026

🧪 HelloWorld Tests

📎 Artifacts

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading

github-actions Bot commented Sep 26, 2026 •

edited

Loading