Andrew Mercer
on this page

16.1 Live migration

  • Triggered from the UI (Migrate), by virtctl migrate, by VirtualMachineInstanceMigration, or automatically when a node enters maintenance mode (VMs with evictionStrategy: LiveMigrate).
  • Requirements: migratable (RWX block) volumes, a target node covered by the VM's networks, compatible CPU, enough resources.
  • Blockers: PCI/USB passthrough devices, vGPU (generally), LVM local volumes, container disks, hostDisk, CD-ROM with uploaded ISO in some versions.
  • Tune via KubeVirt migration config (bandwidth per migration, parallel migrations per cluster/node, completion timeout, post-copy). Large, write-heavy VMs may never converge without throttling or post-copy.

16.2 Maintenance mode

Host → Enable Maintenance Mode: cordons the node and live-migrates VMs away. VMs that can't migrate block maintenance unless you force-stop them. This is the key day-2 operation for firmware updates and hardware swaps.

16.3 Node failure behaviour

Setting vm-force-reset-policy: when a node goes down (not cordoned), Harvester waits a configurable period (default ~5 minutes), then force-deletes the VMIs on the dead node so they can restart elsewhere. Combined with Longhorn replicas, this gives you vSphere HA-like restart behaviour (not FT — VMs reboot).

16.4 PCI passthrough, SR-IOV, vGPU

Enable the pcidevices-controller add-on.

kubectl get pcidevices.devices.harvesterhci.io
  • PCI passthrough: claim a device (PCIDeviceClaim); the device is bound to vfio-pci and becomes allocatable to VMs on that node. VM gets node affinity automatically.
  • SR-IOV NICs: enable VFs on a physical function (SRIOVNetworkDevice), then pass VFs to VMs.
  • vGPU / SR-IOV GPU: for NVIDIA vGPU, enable the nvidia-driver-toolkit add-on with the (licensed) vGPU host driver, then enable SRIOVGPUDevice/VGPUDevice and pick profiles.
  • USB passthrough: USBDevice / USBDeviceClaim (e.g. license dongles, Zigbee sticks).

Prerequisites: IOMMU enabled in BIOS and kernel (Harvester enables intel_iommu=on/amd_iommu=on appropriately; verify IOMMU groups).

16.5 CPU pinning, NUMA, hugepages

  • CPU manager (static policy) enabled per node allows dedicated CPUs for VMs — essential for latency-sensitive workloads (NFV, real-time).
  • Isolated emulator thread keeps QEMU overhead off the guest's dedicated cores.
  • Hugepages reduce TLB pressure for large-memory VMs; configured on the host and requested in VM spec.

16.6 Hot-plug

  • Volume hot-plug: add/remove data volumes to running VMs.
  • NIC hot-plug and CPU/memory hot-plug: supported in recent KubeVirt/Harvester versions with caveats (guest support, maximums set at boot). Check your version.

16.7 Scheduling

  • Node selectors/affinity and VM anti-affinity (keep replicas of a clustered app apart):
affinity:
  podAntiAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchLabels:
            app: postgres
        topologyKey: kubernetes.io/hostname
  • Taints/tolerations for dedicated GPU or high-memory nodes.
  • The descheduler add-on rebalances VMs (via live migration) when nodes become uneven.

16.8 VirtioFS and service account mounts

v1.9 adds mounting a Kubernetes service account into a VM via virtiofs, letting VM workloads authenticate to the Kubernetes API like pods do — a nice bridge for hybrid apps.

16.9 KubeVirtBMC

An emerging pattern (and related project) exposes a virtual BMC (IPMI/Redfish) per VM, so bare-metal provisioning tools like Metal3, Tinkerbell or MAAS can treat KubeVirt VMs as if they were physical servers — great for testing bare-metal pipelines on Harvester.


← Virtual machines: images, templates, cloud-init · Index · Backup, snapshot, restore and DR →