KVM/VMware: warm/change-tracking migration and direct VDDK import into Ceph/RBD and Linstor, plus importVm adoption of existing RBD root volumes#13656
Open
andrijapanicsb wants to merge 15 commits into
Conversation
Operator-driven warm VMware-to-KVM migration built on VDDK and VMware Changed Block Tracking (CBT). Keeps a source-equivalent KVM-side replica current through delta cycles, then finalizes with virt-v2v in-place at cutover. Supports both filesystem (qcow2) and native Ceph/RBD (raw image) primary storage targets, with preflight validation, async lifecycle operations, persisted migration state, and Import/Export UI integration. Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
Extends the existing VDDK + virt-v2v VMware-to-KVM import path so the converted disk can be written straight into an RBD pool as a raw image, with in-place virt-v2v finalization. Adds host-capability detection for qemu-img RBD support, RBD qemu copy, virt-v2v in-place, and direct RBD import, advertised via ReadyCommand and reconciled by the agent manager. Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
Extends the KVM unmanaged import / manage-volume flow so an existing raw RBD image can be adopted directly as a VM root volume, including the RBD-format check in the volume orchestrator and the libvirt volume inspection path. Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
Unifies the host-capability keys that the CBT warm-migration and VDDK direct-RBD-import features introduced independently under different names, using feature-neutral names that describe what is probed: host.vmware.cbt.support -> host.vddk.blockcopy.support host.vmware.cbt.in.place.finalization.support -> host.vddk.blockcopy.inplace.finalization.support host.vmware.cbt.rbd.support -> host.vddk.blockcopy.rbd.support host.qemu.img.rbd.support -> host.qemu.rbd.support host.virt.v2v.inplace.support / host.virtv2v.in.place.version -> host.virtv2v.inplace.* Drops the redundant host.rbd.qemu.copy.support alias (identical to host.qemu.rbd.support) and its hostSupportsRbdQemuCopy() delegate. Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #13656 +/- ##
============================================
+ Coverage 19.64% 19.75% +0.10%
- Complexity 19789 20154 +365
============================================
Files 6368 6420 +52
Lines 575109 581895 +6786
Branches 70371 71170 +799
============================================
+ Hits 112994 114945 +1951
- Misses 449829 454473 +4644
- Partials 12286 12477 +191
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
andrijapanicsb
requested review from
GabrielBrascher,
JoaoJandre,
bernardodemarco,
rafaelweingartner and
wido
and removed request for
JoaoJandre
July 20, 2026 22:54
Contributor
Author
|
@blueorangutan package kvm |
|
@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18626 |
andrijapanicsb
changed the title
…isk import The "Manage Instances" source dropdown offered "Import QCOW2 image from Local/Shared Storage", but importVm now adopts an existing ROOT disk from Filesystem, NetworkFilesystem, SharedMountPoint and RBD (raw) pools, so the QCOW2-only wording is misleading. Rename both entries and their wizard title/description to "Import Instance using existing ROOT disk from Local/Shared Storage", across the en, pt_BR and te locales. Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
docs/vmware-cbt/architecture.md and docs/vmware-cbt/README.md were added without the Apache license header, which failed the RAT license check and the pre-commit insert-license hook. Add the standard ASF header (identical to the one already present in docs/vmware-cbt-migration.md). Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
…ey names State Apache CloudStack 4.24.0.0 as the target release (schema delivered via the 4.24.0.0 upgrade path, with a note that the DDL currently rides in schema-42210to42300.sql pending the main version bump), and replace the pre-consolidation host-capability key names with the current host.vddk.blockcopy.* / host.virtv2v.inplace.version names. Signed-off-by: Andrija Panic <andrija.panic@gmail.com>
Staged VMware-to-KVM imports (virt-v2v conversion to an NFS temporary location followed by a copy to the destination pool) can now target Linstor primary storage: - Linstor added to the staged-conversion destination pool types, so destination pool discovery considers cluster- and zone-wide Linstor pools alongside NFS and RBD. - The import host is validated to have access to each Linstor destination pool (it must be a LINSTOR satellite connected to the pool); the RBD qemu-copy capability check is preserved and the validation renamed to validateStagedImportHostSupport. - The KVM-side converted-disk move already dispatches by pool type to LinstorStorageAdaptor.copyPhysicalDisk, which spawns the resource from the pool's resource group and qemu-img converts the qcow2 into the DRBD block device; converted-disk metadata for Linstor pools now reports the pool UUID and volume name instead of parsing NFS mounts (Linstor pools have no local mount path). The imported volume keeps the copy-time UUID as its path; the Linstor driver addresses existing volumes by path, so later volume lifecycle operations (attach, resize, snapshot, delete) work unchanged.
Extends the direct (non-staged) VDDK import path, previously available only for Ceph/RBD, to Linstor primary storage. With usevddk=true and forceconverttopool=true on a Linstor conversion pool, the KVM host copies each VMware disk over nbdkit/VDDK straight into a pre-created DRBD block device and finalizes it with virt-v2v in place - no NFS staging and no intermediate qcow2 copy. Server side: - Linstor added to the VDDK direct-conversion pool allow-list; the RBD/Linstor "requires usevddk" validation is shared. - Conversion host selection for Linstor requires VDDK support, in-place virt-v2v support, and access to the Linstor pool (the host must be a LINSTOR satellite); explicit host selections are validated for pool access as well. - CheckConvertInstanceCommand carries a new in-place finalization check so unsupported hosts fail fast with a clear message. - The powered-off, non-cloned source VM requirement now applies to any direct block-storage import. KVM agent side: - The direct conversion routine handles both targets: for Linstor it pre-creates each resource at the source capacity through the storage adaptor (qemu-img cannot create DRBD devices, unlike RBD images), copies with qemu-img convert -n to the local DRBD device, and feeds virt-v2v-in-place a <disk type='block'> domain XML - no qemu URI or RBD-style network disks needed. - Storage access is probed with a 4 MiB test volume and a qemu-io write/read round-trip before any data transfer. - Linstor disk names use a shorter '-dNN' suffix: LINSTOR resource names are capped at 48 characters, which the RBD '-disk-NNN' scheme would exceed. - The forced-conversion import short-circuit accepts Linstor temporary locations, backed by a new LinstorStorageAdaptor.listPhysicalDisks implementation (resource definitions filtered by the pool's resource group), replacing the previous UnsupportedOperationException.
Extends the VMware CBT warm-migration framework, previously limited to qcow2 file and Ceph/RBD targets, with a generic raw-block-device target type backed by Linstor primary storage. The initial full sync, the incremental CBT delta cycles and the powered-off cutover all write straight into the DRBD block device of a pre-created Linstor volume. Target classification and orchestration: - New VmwareCbtTargetStorageType.RAW_BLOCK_DEVICE; Linstor pools classify as supported block-device targets that require in-place virt-v2v finalization (the qcow2 fallback cannot write to devices). - Linstor added to the CBT-compatible pool types for explicit and implicit destination pool selection. - Conversion host selection requires VMware CBT migration support, in-place finalization support and access to the Linstor pool (the host must be a LINSTOR satellite); both auto-selection filtering and explicit host validation enforce it. - The destination storage probe now covers block-device pools: the agent creates a 4 MiB volume through the storage adaptor, round-trips a qemu-io write/read on the local device and deletes it. Naming: block-device target volumes are named cbt-<mig8>-<diskId>, where mig8 is the first eight characters of the migration UUID. LINSTOR resource names are limited to 48 characters (the volume name gets a cs- prefix as a LINSTOR resource), so the RBD-style names carrying the full migration UUID do not fit. The short marker keeps cleanup guarded: both the management server and the agent only delete volumes whose names start with the marker derived from the migration UUID. Agent-side data path: - Initial sync pre-creates each Linstor volume at source capacity via the storage adaptor (qemu-img cannot create DRBD devices) and runs the nbdkit/VDDK captive copy with qemu-img convert -n to the local device path. - Delta cycles resolve the device path and patch changed extents with the existing qemu-io write mechanism; raw device targets are accepted by disk validation. - Cutover feeds virt-v2v-in-place a <disk type='block'> domain XML with the local device path - the qemu-nbd localhost bridges required for RBD are unnecessary for block devices. - Cleanup deletes only marker-guarded volumes through the adaptor. The finalized volumes are imported through the existing shared path; imported Linstor volumes keep the target name as their volume path, which the Linstor driver uses to address existing volumes.
Covers the release note for the three import modes (staged, direct VDDK, CBT warm migration), the operator guide's destination-specific host requirements and operational notes (satellite/node naming, single Primary writer, diskful replica preference, DRBD size rounding, heartbeat fencing interaction, retained volume names), and the as-built design guide's storage-target section (RAW_BLOCK_DEVICE mapping, pre-created devices, convert -n, block-disk XML finalization, 48-character resource-name constraint and the short cleanup marker).
Found during live validation on an Ubuntu 24.04 LINSTOR lab node: virt-v2v-in-place 2.4.0 does not know the -O (write updated output XML) option, which only exists from virt-v2v 2.5 on (EL9 ships 2.5.x, where the RBD flows were originally validated). Both the VMware CBT cutover finalization and the direct VDDK import finalization passed -O unconditionally, so otherwise fully capable hosts failed the in-place step at the very end of a migration. Nothing ever consumed the output XML - both wrappers created a temporary file for it and deleted it unread - so the option and the temporary files are removed rather than gated on a version probe. Verified live: virt-v2v-in-place 2.4.0 with the same libvirt XML converts a guest on a DRBD block device successfully without -O.
The CBT delta sync copied each changed extent in two hops: qemu-img convert the nbdkit source window into a temporary raw file on local disk, then qemu-io write that file into the target at the same offset. For a raw block-device target (Linstor/DRBD) that doubles the local I/O and needs scratch space proportional to the changed set. For RAW_BLOCK_DEVICE targets the changed extent is now streamed from the nbdkit source window straight into the same target device window in a single qemu-img convert (raw source image-opts -> raw host_device target image-opts at matching offset/size) - no temp file, no scratch directory, no second process per chunk. -S 0 disables zero/sparse skipping so blocks the source cleared to zero are actually overwritten in the target, preserving delta correctness (a full copy can skip zeros because the target starts zeroed; a delta into an already-populated device cannot). -n keeps the pre-created device. The qcow2-file and RBD paths are unchanged - qcow2 cannot be addressed by a raw offset window, and the RBD URI target keeps the proven qemu-io path. Validated on a 3-node LINSTOR lab against a real VMware source: after a full baseline sync, 144 CBT changed extents applied through the new direct-copy script produced a device byte-identical to a full read of the delta snapshot (qemu-img compare: images are identical).
… available The cold direct-VDDK import and the VMware CBT initial full sync copy an entire disk from an nbdkit/VDDK source into a target. For a local raw block-device target (Linstor/DRBD) this is now done with nbdcopy (from libnbd) when the host has it, falling back to qemu-img convert otherwise. nbdcopy keeps many requests in flight and is what virt-v2v itself uses for this step, so it is typically faster than a single-connection qemu-img convert. Scope and safety: - Only the two full-disk copy paths, and only for RAW_BLOCK_DEVICE targets whose target is a local device path nbdcopy can write directly. The RBD URI target and qcow2 file targets keep qemu-img convert (nbdcopy cannot address an rbd: URI without a bridge, and is raw-only). - The CBT delta cycles are unchanged: they copy a caller-supplied list of scattered changed extents at fixed offsets, which is range-addressed work nbdcopy does not model - qemu-img/qemu-io remain correct there. - Availability is probed locally on the agent (nbdcopy --version); it is a pure optimization with an automatic fallback, so no server-side gating or new host capability is needed. - Correct on a freshly spawned thin device: nbdcopy skips source holes, and the device reads back as zeros, matching qemu-img convert's sparse behavior. Validated on a 3-node LINSTOR lab against a real VMware source: a 16 GiB disk copied via the new nbdcopy path is byte-identical to a full VDDK read of the source (qemu-img compare: images are identical), and ran about 20% faster than qemu-img convert on the same disk over network NBD transport.
andrijapanicsb
changed the title
Contributor
Author
|
@blueorangutan package kvm |
|
@andrijapanicsb a [SL] Jenkins job has been kicked to build packages. It will be bundled with kvm SystemVM template(s). I'll keep you posted as I make progress. |
A standalone design/validation document covering the Linstor-destination portion of this series, written so LINSTOR/DRBD maintainers can review the storage-side behavior: the RAW_BLOCK_DEVICE target model, pre-created resources with qemu-img convert -n / nbdcopy, in-place finalization on the local DRBD device, resource-naming constraints, delta-copy semantics, host prerequisites, the support matrix, and the lab/end-to-end validation performed.
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ el10 ✔️ debian ✔️ suse15. SL-JID 18633 |
Damans227
reviewed
Jul 21, 2026
| import static com.cloud.host.Host.HOST_VIRTV2V_INPLACE_SUPPORT; | ||
| import static com.cloud.host.Host.HOST_VIRTV2V_INPLACE_VERSION; | ||
| import static com.cloud.host.Host.HOST_VIRTV2V_VERSION; | ||
| import static com.cloud.host.Host.HOST_VDDK_BLOCKCOPY_INPLACE_FINALIZATION_SUPPORT; |
Collaborator
There was a problem hiding this comment.
Nit: these new static imports (HOST_VDDK_BLOCKCOPY_*) break the existing alphabetical import order (they land after HOST_VDDK_VERSION instead of before HOST_VDDK_LIB_DIR).
| import java.util.List; | ||
| import java.util.Map; | ||
|
|
||
| import org.apache.commons.collections.CollectionUtils; |
Collaborator
There was a problem hiding this comment.
Nit: this imports the legacy org.apache.commons.collections.CollectionUtils (also in LibvirtVmwareCbtCleanupCommandWrapper.java, LibvirtVmwareCbtCutoverCommandWrapper.java, LibvirtVmwareCbtPrepareCommandWrapper.java), while other new files in this PR use commons.collections4. Worth picking one.
| "message.desc.import.ext.kvm.wizard": "Import libvirt domain from External KVM Host not managed by CloudStack", | ||
| "message.desc.import.local.kvm.wizard": "Import QCOW2 image from Local Storage of selected KVM Host", | ||
| "message.desc.import.shared.kvm.wizard": "Import QCOW2 image from selected Primary Storage Pool", | ||
| "message.desc.import.local.kvm.wizard": "Import an Instance using an existing ROOT disk from the Local Storage of the selected KVM host", |
Collaborator
There was a problem hiding this comment.
Nit: this changes "selected KVM Host" to "selected KVM host" (lowercase) and the sibling string below drops "Storage Pool" to "Storage pool", a casing regression from the previous strings.
Contributor
Author
There was a problem hiding this comment.
Labeling is SO incosistent (plsm check that file 5000 times "host" and another 5000 times Host - I'm lazy - but feel free to change it if you want
I'm more interested in actual testing the value of the PR than semantically
Contributor
Author
|
@rp- — this PR now also adds Linstor as a destination for the To make that easy there's a standalone write-up committed in the PR: The Linstor-side design points I'd most like sanity-checked:
Validated on a 3-node LINSTOR/DRBD 9 lab: byte-identical disk copies |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR makes Ceph/RBD and Linstor first-class targets for KVM VM onboarding and adds an operator-driven warm VMware→KVM migration path. It bundles the following (Ceph/RBD plus Linstor as block-storage destinations):
1. VMware CBT → native RBD/qcow2 warm migration to KVM
An operator-driven, warm VMware-to-KVM migration built on VDDK and VMware Changed Block Tracking (CBT). It maintains a source-equivalent KVM-side replica kept current through repeated delta/sync cycles, then finalizes the guest with virt-v2v in-place at cutover — bounding downtime to the cutover window instead of a full cold copy.
qcow2) and native Ceph/RBD (raw image) primary storage.startVmwareCbtMigration,syncVmwareCbtMigration,cutoverVmwareCbtMigration,cancelVmwareCbtMigration,deleteVmwareCbtMigration,listVmwareCbtMigrations,checkVmwareCbtMigrationPrerequisites.VmwareCbtMigrationManagerImpl(+VmwareCbtMigrationServiceImpl, cutover policy), backed by new VOs/DAOs and a DB schema migration.docs/vmware-cbt/(architecture + README) anddocs/vmware-cbt-migration.md.2. Direct VDDK VMware import into Ceph/RBD
Extends the existing VDDK + virt-v2v VMware→KVM import path so the converted disk is written straight into an RBD pool as a raw image, with in-place virt-v2v finalization (no intermediate qcow2-on-NFS staging copy). Adds host-capability detection for qemu-img RBD support, RBD qemu-copy, virt-v2v in-place, and direct-RBD import — advertised via
ReadyCommandand reconciled by the agent manager.3.
importVmadoption of an existing RBD (Ceph) root volumeExtends the KVM unmanaged-import / manage-volume flow so an existing raw RBD image can be adopted directly as a VM ROOT volume. Adds RBD-aware format handling in
VolumeOrchestrator(RBD ⇒RAWinstead of the hardcodedQCOW2) and the RBD branch in the libvirt volume-inspection path (LibvirtCheckVolumeCommandWrapper), soCheckVolumeCommandsucceeds on RBD instead of failing with "Disk not found or is invalid".4. Consolidate KVM host-capability probe names across the RBD features
Because features (1) and (2) introduced host-capability keys independently, this commit unifies them under feature-neutral names that describe what is probed, and drops a redundant alias (
host.rbd.qemu.copy.support, identical tohost.qemu.rbd.support) and its delegate.5. Linstor (DRBD) primary storage as a migration destination
Extends all three VMware→KVM paths above — staged import, direct VDDK import, and warm CBT migration — so the converted disks can also land on Linstor primary storage, not just Ceph/RBD. A generic
RAW_BLOCK_DEVICEtarget type covers "write RAW into a host-local block device provided by the storage adaptor", so the same code paths serve Linstor (and lay groundwork for other block backends).qemu-imgcannot create a DRBD device), copies withqemu-img convert -n— ornbdcopywhen the host has it — into the local/dev/drbd/…device, and finalizes withvirt-v2v-in-placefed a<disk type='block'>domain XML (no qemu-nbd bridge needed for a local device).qemu-img convert -n -S 0), avoiding a temp-file round-trip.-Oflag from thevirt-v2v-in-placeinvocation (fixes in-place finalization on virt-v2v < 2.5, e.g. Ubuntu 24.04) — which benefits the RBD path too.Types of changes
Feature/Enhancement Scale
How Has This Been Tested?
Unit tests for all new and touched wrappers/managers/orchestration:
LibvirtImportConvertedInstanceCommandWrapperTest,LibvirtConvertInstanceCommandWrapperTest,LibvirtCheckConvertInstanceCommandWrapperTest,LibvirtCheckVolumeCommandWrapperTest,the
LibvirtVmwareCbt*suites,UnmanagedVMsManagerImplTest,VolumeImportUnmanageManagerImplTest, the server-sideVmwareCbt*suites,and
VolumeOrchestratorTest— all passing.Manual/lab validation of warm CBT migration and direct-RBD import against a Ceph/RBD KVM cluster.
Linstor destination validated byte-identical on a live 3-node LINSTOR/DRBD lab against a real VMware source (cold direct-VDDK and warm CBT;
qemu-img compare= images identical), and end-to-end through a liveimportVmthat produced a Stopped CloudStack VM with its root and data disks as Linstor volumes backed by DRBD resources UpToDate on all three nodes. New Linstor unit tests across the destination gate lists, host selection, the convert/import/CBT wrappers,listPhysicalDisks, and the block-device delta/nbdcopypaths — all passing.