Longhorn is the storage foundation for Harvester, so understanding it well pays off later.
6.1 Architecture¶
- Longhorn Manager (DaemonSet): controller, API, orchestrates volumes.
- Engine: one per volume, runs on the node where the volume is attached; synchronously replicates writes to replicas.
- Replicas: copies of the volume data on (ideally) different nodes/disks.
- Instance Manager: per-node pod hosting engine and replica processes.
- CSI plugin: Kubernetes integration.
- Backing images: base disk images shared by many volumes (Harvester VM images are backing images).
Each volume is effectively its own micro-controller — failure of one engine affects one volume, not the cluster.
6.2 Data engines¶
| V1 data engine | V2 data engine | |
|---|---|---|
| Datapath | iSCSI + tgt, sparse files on a filesystem | SPDK, userspace NVMe-oF, block devices directly |
| Maturity | Production default | Progressively maturing; Harvester exposes it for selected use |
| Performance | Good | Substantially lower latency/higher IOPS |
| Requirements | Any filesystem disk | Dedicated block devices, hugepages, more CPU (SPDK polls) |
6.3 Key features¶
- Snapshots (crash-consistent) and backups to S3 or NFS (incremental, block-level).
- Recurring jobs for snapshot/backup/cleanup.
- DR volumes — standby volumes in another cluster continuously restoring from backups.
- RWX via a share-manager (NFS) pod; also migratable RWX block volumes used by KubeVirt live migration.
- Replica auto-balance, data locality (
best-effort,strict-local), node/disk tags and selectors. - Volume encryption (LUKS via CSI secrets).
- Offline replica rebuilding, orphaned instance cleanup, recurring system backups (newer releases).
6.4 Sizing heuristics¶
- Replica count 3 for production; 2 can be acceptable with good backups; 1 only for scratch.
- Budget raw capacity ≈ provisioned × replicas × (1 / target utilization). With 3 replicas and 70% target utilization, 10 TB of VM disks needs ~43 TB raw.
- Longhorn over-provisioning (default 100–200%) relies on thin provisioning; monitor actual usage, not provisioned.
- Separate disks for OS vs. Longhorn data. NVMe strongly preferred.
- 10 GbE minimum for replication traffic; 25 GbE when using V2 or heavy write loads. A dedicated storage network helps a lot.
← Fleet: GitOps at scale · Index · Security and policy: NeuVector, Kubewarden, CIS →