Originally based on this article and others, then reworked:
for_eachover a hostname → IP map instead of parallel lists withcounttemplatefile()instead of the archivedtemplateprovider'stemplate_filedata source- Files shown as files (the original
cat <<EOFheredocs were unquoted, so the shell expanded every${...}before Terraform saw it) - Current Ubuntu LTS cloud image, current netplan and cloud-init syntax
Provider version. This page targets the
dmacvicar/libvirt0.8.x schema. Newer releases of the provider have reworked resource schemas; if you move past 0.8, check the provider's docs and changelog before reusing these blocks.
Prerequisites¶
# Fedora
sudo dnf -y install @virtualization genisoimage
sudo systemctl enable --now libvirtd
# Let your user manage system libvirt without sudo or a polkit prompt
sudo usermod -aG libvirt "$USER"
newgrp libvirt # or log out and back in
virsh -c qemu:///system list --all # should work without a password prompt
You need a Linux bridge (e.g. br0) attached to your LAN if VMs should get LAN addresses. Otherwise use libvirt's default NAT network and adjust the network config below.
Project layout¶
~/terraform/kvm/ubuntu/
├── versions.tf
├── providers.tf
├── variables.tf
├── main.tf
├── outputs.tf
├── terraform.tfvars
├── images/ # downloaded base image
└── templates/
├── cloud_init.yaml.tftpl
└── network_config.yaml.tftpl
mkdir -p ~/terraform/kvm/ubuntu/{images,templates}
cd ~/terraform/kvm/ubuntu
versions.tf¶
terraform {
required_version = ">= 1.6"
required_providers {
libvirt = {
source = "dmacvicar/libvirt"
version = "~> 0.8.3"
}
}
}
providers.tf¶
provider "libvirt" {
uri = "qemu:///system"
# Remote hypervisor over SSH:
# uri = "qemu+ssh://[email protected]/system"
}
variables.tf¶
variable "hosts" {
description = "VMs to create: hostname → static IPv4 address."
type = map(string)
}
variable "bridge" {
description = "Host Linux bridge the VMs attach to."
type = string
default = "br0"
}
variable "interface" {
description = "Guest NIC name as seen by netplan."
type = string
default = "ens3"
}
variable "prefix_length" {
type = number
default = 24
}
variable "gateway" {
type = string
}
variable "dns_servers" {
type = list(string)
default = ["1.1.1.1", "9.9.9.9"]
}
variable "domain" {
type = string
default = "local"
}
variable "memory_mb" {
type = number
default = 2048
}
variable "vcpu" {
type = number
default = 2
}
variable "disk_size_gb" {
type = number
default = 20
}
variable "base_image_path" {
type = string
default = "images/noble-server-cloudimg-amd64.img"
}
variable "ssh_public_key" {
type = string
}
terraform.tfvars¶
hosts = {
k8s-cp-01 = "192.168.0.60"
k8s-wk-01 = "192.168.0.61"
k8s-wk-02 = "192.168.0.62"
}
bridge = "br0"
gateway = "192.168.0.1"
dns_servers = ["192.168.0.1"]
ssh_public_key = "ssh-ed25519 AAAA... andrew@workstation"
main.tf¶
# One shared, read-only base image
resource "libvirt_volume" "base" {
name = "ubuntu-noble-base.qcow2"
pool = "default"
source = "${path.module}/${var.base_image_path}"
format = "qcow2"
}
# A copy-on-write disk per VM, grown to disk_size_gb
resource "libvirt_volume" "os" {
for_each = var.hosts
name = "${each.key}.qcow2"
pool = "default"
base_volume_id = libvirt_volume.base.id
format = "qcow2"
size = var.disk_size_gb * 1024 * 1024 * 1024
}
# cloud-init seed ISO per VM: hostname, user, SSH key, static network
resource "libvirt_cloudinit_disk" "seed" {
for_each = var.hosts
name = "${each.key}-seed.iso"
pool = "default"
user_data = templatefile("${path.module}/templates/cloud_init.yaml.tftpl", {
hostname = each.key
fqdn = "${each.key}.${var.domain}"
ssh_public_key = var.ssh_public_key
})
network_config = templatefile("${path.module}/templates/network_config.yaml.tftpl", {
interface = var.interface
ip_address = each.value
prefix_length = var.prefix_length
gateway = var.gateway
dns_servers = var.dns_servers
})
}
resource "libvirt_domain" "vm" {
for_each = var.hosts
name = each.key
memory = var.memory_mb
vcpu = var.vcpu
cloudinit = libvirt_cloudinit_disk.seed[each.key].id
cpu {
mode = "host-passthrough"
}
disk {
volume_id = libvirt_volume.os[each.key].id
}
network_interface {
bridge = var.bridge
addresses = [each.value] # informational for a bridge; the guest's netplan sets the IP
}
# Ubuntu cloud images can hang at boot without a serial console.
# If a VM sits at 100% CPU and never comes up, this is usually why.
console {
type = "pty"
target_port = "0"
target_type = "serial"
}
graphics {
type = "spice"
listen_type = "address"
autoport = true
}
}
outputs.tf¶
output "vms" {
description = "Hostname → configured IP."
value = { for name, d in libvirt_domain.vm : name => var.hosts[name] }
}
output "ssh_commands" {
value = { for name, ip in var.hosts : name => "ssh ubuntu@${ip}" }
}
The original output read addresses from network_interface[0].addresses, which libvirt only reports for networks it manages (DHCP leases on NAT networks). On a bridge with static IPs, output what you configured.
templates/cloud_init.yaml.tftpl¶
#cloud-config
hostname: ${hostname}
fqdn: ${fqdn}
manage_etc_hosts: true
users:
- name: ubuntu
groups: [sudo]
shell: /bin/bash
sudo: "ALL=(ALL) NOPASSWD:ALL"
lock_passwd: true
ssh_authorized_keys:
- ${ssh_public_key}
ssh_pwauth: false
disable_root: true
package_update: true
packages:
- qemu-guest-agent
runcmd:
- systemctl enable --now qemu-guest-agent
Key-only SSH, no root login, no passwords. If you genuinely need console password login for a lab, set a hashed password (passwd: with a mkpasswd -m sha-512 hash), never a plaintext one in Git.
templates/network_config.yaml.tftpl¶
version: 2
ethernets:
${interface}:
dhcp4: false
addresses:
- ${ip_address}/${prefix_length}
routes:
- to: default
via: ${gateway}
nameservers:
addresses:
%{ for dns in dns_servers ~}
- ${dns}
%{ endfor ~}
gateway4 is deprecated in netplan; routes with to: default replaces it.
Download the base image¶
curl -L --output-dir images -O \
https://cloud-images.ubuntu.com/noble/current/noble-server-cloudimg-amd64.img
Using a local file keeps apply fast and works offline. source can also be the URL directly, at the cost of a download per base volume creation.
Run it¶
terraform init
terraform fmt
terraform validate
terraform plan -out=tfplan
terraform apply tfplan
Add or remove a VM by editing the hosts map. Because of for_each, only that VM is created or destroyed; with the original count over a list, removing a host from the middle shifted every later VM's index and recreated them.
Tear down:
terraform destroy
Troubleshooting¶
failed to connect: authentication unavailable: no polkit agent available to authenticate action 'org.libvirt.unix.manage'¶
Your user isn't allowed to manage system libvirt without an interactive polkit prompt.
sudo usermod -aG libvirt "$USER"
newgrp libvirt
Could not open '/var/lib/libvirt/images/...': Permission denied¶
QEMU runs as the qemu user and SELinux confines it. Keep volumes in a libvirt storage pool (as above) rather than in your home directory. If you use a custom pool directory, label it:
sudo semanage fcontext -a -t virt_image_t '/srv/vms(/.*)?'
sudo restorecon -Rv /srv/vms
VM boots but has no network¶
- Check the guest NIC name:
virsh console <vm>, thenip link. Cloud images may useenp1s0rather thanens3; setinterfaceaccordingly. - Confirm the bridge exists and has the uplink:
nmcli connection show,bridge link. - cloud-init logs:
sudo cat /var/log/cloud-init-output.loginside the guest.
Changes to cloud-init don't apply¶
cloud-init runs once per instance. Changing the template replaces the seed ISO, but the guest won't re-run cloud-init. Recreate the VM:
terraform apply -replace='libvirt_domain.vm["k8s-wk-01"]' -replace='libvirt_volume.os["k8s-wk-01"]'