Add USB mass storage support over xHCI bulk transfers - #3255
Merged
valentinbreiz merged 17 commits intoSep 24, 2026
Merged
Conversation
Class drivers could only use control transfers and interrupt IN pipes, which is all a keyboard needs; mass storage runs on bulk endpoints. UsbDevice gains OpenBulkEndpoint, BulkIn, BulkOut, ResetEndpoint and ClearHalt. On xHCI a bulk pipe is synchronous, one Normal TRB at a time through a DMA bounce buffer placed so no TRB crosses a 64 KiB boundary; longer transfers are split and a short packet ends them. A failed or timed-out transfer leaves the endpoint Stopped with its ring cleared (Stop Endpoint or Reset Endpoint, then Set TR Dequeue Pointer), and ResetEndpoint also restarts the data toggle, by dropping and re-adding a non-halted endpoint. SuperSpeed endpoints take bMaxBurst from their companion descriptor. Commands and synchronous control transfers are now serialized by a mutex each: endpoint recovery issues them from whichever thread hit the error, no longer only from the boot-time enumeration.
USB sticks, card readers and USB disks carry SCSI commands over the Bulk-Only Transport. The driver binds those interfaces, asks for the LUN count, and turns every unit with a medium into a block device named usb0, usb1, ...: INQUIRY, TEST UNIT READY until ready (an empty card slot is skipped), READ CAPACITY (10, then 16 past 2^32 blocks). Reads and writes go out as READ/WRITE(10), or (16) past 2^32, 64 KiB at most per command. A failed command's sense data is fetched at once; a UNIT ATTENTION or a transport error, after which the device is reset, is retried. Flush sends SYNCHRONIZE CACHE and stops once a device rejects it, as most flash drives do. StorageManager registers the units after the AHCI and NVMe devices, so an internal disk stays the primary one, which puts them in diskinfo, lspart and mount like any other disk. USB now also comes up when only storage is enabled.
A usb disk kind attaches the image as a usb-storage stick on a shared qemu-xhci controller, in test profiles and in `cosmos run --disk image,usb`. The Storage suite runs in a new usb profile under the same block-device assertions as ahci and nvme, which makes 6 cells on x64 and 18 on arm64; the arm64 step and job budgets grow to match. Device_LargeTransfer now moves 257 blocks instead of 32, past every driver's per-command limit plus one, so splitting a request into several commands and a short last one are covered.
List USB sticks among the devices StorageManager registers, note that they are only found at boot, and show `cosmos run --disk image,usb`.
🧪 Interrupts Tests
Skipped
📎 Artifacts
|
🧪 Storage Tests
Skipped
📎 Artifacts
|
A hot-plug thread, started once the scheduler's timer ticks, waits for port changes: the xHCI Port Status Change Event on a root port, the status change endpoint on a hub port, or a 250 ms poll when the controller has no MSI-X. It clears the change bits, drops whatever sat on a port that disconnected, and after a 100 ms debounce enumerates what was plugged in. A device that leaves is marked disconnected from the interrupt handler already, so transfers waiting on it return the new Disconnected status instead of timing out and skip recovery. Its subtree is then released children first: each class driver's Disconnect, the device leaving the manager's list, and the controller disabling its slot once no control or bulk transfer still holds its memory. Hub port handling moves into UsbHub, which also follows the hub's status change endpoint. The mass storage and keyboard drivers keep copy-on-write lists and report units that come and go through attach/detach callbacks. I/O on a removed stick throws IOException at once, and names are reused from the lowest free usbN. Without the scheduler's timer (x64 with ACPI off) hot-plug stays off and the devices are the ones found at boot.
Kernel.Start starts the USB hot-plug thread once interrupts are on. A stick that shows up is registered and its partitions scanned like one found at boot; one that goes away is unregistered with its partitions, the primary device moves on, and every filesystem mounted from one of its partitions is detached without a flush, since the disk is gone. USB keyboards join and leave KeyboardManager the same way. The device, partition, mount and keyboard tables are replaced whole on every change instead of changed in place, so a reader on another thread always sees a consistent list. Partition scans run outside the lock and are published only if their device is still registered. TryUnmount drops the mount outside the lock, and TryFormat refuses a partition that is mounted.
`umount <mountpoint>` unmounts and moves the shell out of it. An I/O error, such as a stick pulled out under a running command, is reported and the shell keeps going. The boot stick is auto-mounted through its partition, so it is detached if pulled out, and an empty disk list now says a USB disk can be plugged in.
USB sticks and keyboards can be plugged in and pulled out while the kernel runs. Explain what happens to mounts (partition mounts are detached, source-string mounts stay), to unmount before pulling a stick, how to try it from the QEMU monitor, and where Kernel.Start starts the hot-plug thread.
TR.RequestHost sends a new HostRequest frame (109), and the engine acts on it as soon as it shows up on the UART: `usb-unplug [n]` removes stick n from the xHCI controller with device_del, and `usb-plug [n]` puts it back on the same image with drive_add and device_add. It gets there through QMP: a profile with a USB disk launches QEMU with its monitor connected to a port the engine listens on (QemuLaunchOptions.MonitorPort), and USB sticks get stable QEMU ids. Nothing replies; the test waits for the change, within the 10 s the engine gives it before taking the guest for hung.
Five tests in the usb cells pull the stick out and plug it back in: the stick leaves StorageManager and the driver, I/O on it throws IOException, it comes back under its old name with the sector written before the unplug, and a filesystem mounted from its partition is detached with it, then mounts again with its file intact. They skip on other cells and where the hot-plug thread cannot run (x64 with ACPI off).
Bring the files this branch adds or changes in line with docs/articles/dev/coding-guidelines.md. No behavior changes. - Move constructors after fields and properties. - Split UsbEndpointType into its own file. - Replace pass-through fields with get-only auto-properties. - Use explicit types instead of var, is null / is not null instead of == null / != null, interpolation instead of concatenation, and collection expressions. - Add license headers and a <summary> to QemuHotPlug.DisposeAsync. - Rename HostRequestScanner.Magic to s_magic. - Remove comments that only restate the code. - Explain why the USB drivers' lazily filled statics stay null.
Track .claude/agents/ so every contributor gets the code-cleaner agent, which reads docs/articles/dev/coding-guidelines.md and cleans source files to it without changing behavior. The rest of .claude, such as settings.local.json, stays ignored.
The parser's range check stopped at TestDestructiveReached (108), so HostRequest (109) frames were skipped as noise and their case never ran.
DisposeAsync waited for the monitor connection before it stopped the listener. If QEMU exited before connecting, for example because it rejected an argument, the accept never completed: its token belongs to a source the host disposes without cancelling. The engine then hung at await using. Stop the listener first, which ends the pending accept.
IndexOf and LastIndexOf both return -1 when qemu-xhci is missing, so the one-controller check passed with no controller at all. Assert that the controller is there too.
Move USB support from the future releases to the Gen3 features, with what it covers now and what it doesn't yet (USB mouse, EHCI). Drop the first-release percentage badge.
Add USB hot-plug for sticks, hubs and keyboards
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #3254 (xHCI USB stack): the base is
feature/xhci-usb-keyboard, to be retargeted togen3once that merges.Problem
The USB stack from #3254 can only run control transfers and interrupt IN pipes, which is all a keyboard needs. USB sticks, card readers and USB disks are mass storage devices: they carry SCSI commands over bulk endpoints (the Bulk-Only Transport), so a plugged-in stick was enumerated and then left with
no driver, never reachedStorageManager, and did not show in DevKernel'sdiskinfo,lspartormount.Fix
xHCI bulk transfers.
UsbDevicegainsOpenBulkEndpoint,BulkIn,BulkOut,ResetEndpointandClearHalt, so class drivers stay host-controller agnostic.XhciBulkPipe) is synchronous: one Normal TRB per chunk, through a DMA bounce buffer. The 16 pages are page aligned but not 64 KiB aligned, so the pipe uses the longer side of the 64 KiB boundary they may straddle (at least 32 KiB, a multiple of every packet size). Longer transfers are split into several chunks, and a short packet ends the transfer.ResetEndpointmust also restart the host's data toggle. Reset Endpoint does that, but only on a Halted endpoint. Any other endpoint is dropped and added back through Configure Endpoint (Linuxxhci_endpoint_reset).bMaxBurstfrom their SuperSpeed Endpoint Companion descriptor, whichParseConfigurationnow reads.Mass storage class driver.
UsbMassStorageDriverbinds interfaces with class 0x08, subclass 0x06 (SCSI transparent) and protocol 0x50 (Bulk-Only).UsbBulkOnlyTransportsends the CBW, runs the data stage and reads the CSW. It clears a STALLed data stage and retries a STALLed CSW once. On a bad CSW, a phase error or a failed stage, it runs Reset Recovery (Bulk-Only Mass Storage Reset, then clear both halts).UsbMassStorageis aBlockDeviceper logical unit, namedusb0,usb1, …:Flushsends SYNCHRONIZE CACHE, and stops sending it once a device rejects it with ILLEGAL REQUEST, as most flash drives do.StorageManager.RegisterHalDevicesregisters the units after the AHCI ports and NVMe namespaces, so an internal disk stays the primary device.KeyboardEnabled || StorageEnabled).Tests. A
usbdisk kind attaches an image as ausb-storagestick on a sharedqemu-xhcicontroller, both in test profiles and incosmos run --disk image,usb.usbprofile under the same assertions asahciandnvme. That makes 6 cells on x64 and 18 on arm64, so the arm64 step and job budgets go from 45/60 to 60/75 minutes.Device_LargeTransfernow moves 257 blocks instead of 32, past every driver's per-command limit plus one. This covers splitting one request into several commands, with a short last one.Verification
Real hardware: the author confirmed it works on the Lenovo Legion (AMD Phoenix) that #3254 was brought up on.
DevKernel, QEMU x64 (q35, blank AHCI and NVMe disks plus a 64 MiB stick holding an MBR and a FAT32 partition):
After shutdown, the host reads the file back from the image (
mcopy -i usb.img@@1M ::/COSMOS.TXT -printsWritten by Cosmos over USB).fsck.vfat -nfinds no damage, only the FSInfo free-cluster hint, which the FAT driver never updates on any disk.A second boot put a FAT disk on AHCI (so the auto-mount takes it) and added more devices:
usb0;mount 2 0 /usb, file from the previous boot still thereusb-hubusb1, "Unpartitioned filesystem volume detected";mount 3 0 /flop(FAT16), read and write, write checked from the host[USB storage] usb1 (LUN 0): no medium, skippedDevKernel, QEMU arm64 (virt, GICv3):
[xHCI] Running, events via MSI-X,usb0bound, FAT auto-mounted.catandwritework, and the host readsHello from ARM64back from the image.Storage suite (
make test KERNEL=Storage):usb,usb+gicv2(polled, no ITS),usb+gicv3: 69 passed each, same as ahci; the acpi-off cells skip like ahci/nvme (no PCIe discovery without ACPI)Device_LargeTransferat 257 blocks passes in all 6 x64 cells.dotnet test tests/Cosmos.Tests.Patcher(RunCommand and QemuLauncher tests, including the new usb cases): 56 passed.dotnet format --verify-no-changes --severity erroris clean on the changed files.Known gaps: