This document will take you through the process of initializing a Kubernetes from scratch on baremetal. I'll include steps to configure a hypervisor and maybe split into 3 separate documents in the future.

Install Kubernetes¶

Note: I have ansible tasks to address this, but thought I'd go through the process manually as a refresher.

Select a release¶

Latest Kubernetes releases page:

https://kubernetes.io/releases

I'm going to go with release v1.30

https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.30.md

Install Kubernetes tools¶

https://kubernetes.io/docs/tasks/tools

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.30/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.30/rpm/repodata/repomd.xml.key
EOF

sudo dnf update && sudo dnf -y install kubeadm kubectl kubelet && sudo systemctl enable kubelet

Lock this version:

sudo dnf -y install 'dnf-command(versionlock)' && \
sudo dnf versionlock kubelet kubeadm kubectl

Disable swap¶

Temporarily¶

sudo swapoff -a

Permanently¶

Edit /etc/fstab and comment out the swap entry

modprobe¶

Permanent¶

cat << EOF | sudo tee /etc/modprobe.d/kubernetes.conf
modprobe overlay
modprobe br_netfilter
EOF

Temporary¶

modprobe overlay
modprobe br_netfilter

sysctl¶

Permanent¶

cat << EOF | sudo tee /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
EOF

Temporary¶

sudo systctl --system

Install a container runtime¶

Containerd¶

https://kubernetes.io/docs/tasks/administer-cluster/migrating-from-dockershim/change-runtime-containerd/#install-containerd

Can be installed from binary, but easier - when manually doing this - to use Linux repos & dnf

https://docs.docker.com/engine/install/centos

sudo dnf remove docker \
docker-client \
docker-client-latest \
docker-common \
docker-latest \
docker-latest-logrotate \
docker-logrotate \
docker-engine

sudo dnf -y install dnf-plugins-core && \
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo && \
sudo dnf update && sudo dnf -y install containerd.io

containerd config default | sudo tee /etc/containerd/config.toml

Edit /etc/containerd/config.toml and make sure SystemdCgroup = true is set

sudo systemctl restart containerd && \
sudo systemctl enable containerd

Install nerdctl & crictl¶

nerdctl¶

https://github.com/containerd/nerdctl

version="2.0.3"
curl -L https://github.com/containerd/nerdctl/releases/download/v2.0.3/nerdctl-$version-linux-amd64.tar.gz --output nerdctl-$version-linux-amd64.tar.gz
sudo tar zxvf nerdctl-$version-linux-amd64.tar.gz -C /usr/local/bin
rm -f nerdctl-$version-linux-amd64.tar.gz

crictl¶

- https://github.com/kubernetes-sigs/cri-tools/blob/master/docs/crictl.md

version="v1.30.0"
curl -L https://github.com/kubernetes-sigs/cri-tools/releases/download/$version/crictl-${version}-linux-amd64.tar.gz --output crictl-${version}-linux-amd64.tar.gz
sudo tar zxvf crictl-$version-linux-amd64.tar.gz -C /usr/local/bin
rm -f crictl-$version-linux-amd64.tar.gz

Open firewall ports¶

firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly

sudo firewall-cmd --zone=public --add-port=6443/tcp --permanent && \
sudo firewall-cmd --zone=public --add-port=10250/tcp --permanent && \
sudo firewall-cmd --complete-reload

Edit kubelet systemd unit file¶

sudo echo 'KUBELET_RUNTIME_ARGS="--container-runtime=remote --container-runtime-endpoint=unix:///run/containerd/containerd.sock"' >> /usr/lib/systemd/system/kubelet.service.d/10-kubeadm.conf && \
systemctl daemon-reload && systemctl restart kubelet

Initialize a new cluster¶

sudo kubeadm init --pod-network-cidr 192.168.0.0/16

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install a CNI plugin¶

Kubernetes CNI Plugins

Calico¶

Install the Calico CNI plugin

Cilium¶

Install the Cilium CNI plugin

Add worker nodes¶

Print join command¶

sudo kubeadm token create --print-join-command

From each worker node in the cluster run the output of the above command.

kubectl get nodes -o wide

Troubleshooting¶

If things get really bad and kubectl commands are not working, crictl or nerdctl are what you'll want to use to check how the pods are doing.

sudo crictl ps -a

I actually haven't used nerdctl as of yet, but I'll update this later on when I have.

Install a cluster using Ansible¶

todo