Compiling a Custom Linux Kernel¶

A complete, from-source walkthrough for building and installing a custom Linux kernel on a mainstream distro (Debian/Ubuntu, Fedora/RHEL, and Arch notes included).

1. Overview and prerequisites¶

Compiling a custom kernel means fetching kernel source, tuning a .config file, building the image and modules, installing them, and pointing your bootloader at the result. Budget 20-60 minutes of CPU time and 15-20 GB of free disk space (source tree, build objects, and the installed kernel/modules all add up).

Packages you need (Debian/Ubuntu):

sudo apt update
sudo apt install build-essential libncurses-dev bison flex libssl-dev \
  libelf-dev dwarves bc rsync kmod cpio fakeroot git

Packages you need (Fedora/RHEL):

sudo dnf install gcc gcc-c++ make ncurses-devel bison flex openssl-devel \
  elfutils-libelf-devel bc rsync kmod cpio dwarves fakeroot git

Packages you need (Arch):

sudo pacman -S base-devel ncurses bison flex openssl elfutils bc \
  cpio pahole git

• libssl-dev/openssl-devel and libelf-dev are needed for module signing and BTF (BPF Type Format) generation.
• dwarves/pahole generates vmlinux BTF data used by modern BPF tooling; safe to skip if you disable CONFIG_DEBUG_INFO_BTF.
• Work as a normal user for fetching and configuring; only make modules_install and make install need root.
• Keep the source tree under your home directory or /usr/src — either works, but avoid building on a network filesystem (NFS breaks some build steps).

2. Obtaining kernel source¶

Option A - tarball from kernel.org (simplest, single version):

cd ~/src
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.11.tar.xz
wget https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-6.11.tar.sign
tar -xf linux-6.11.tar.xz
cd linux-6.11

Verify the tarball's PGP signature against the release manager's key before building anything from it:

xz -cd ../linux-6.11.tar.xz | gpg2 --verify ../linux-6.11.tar.sign -

Import the signing keys first if you don't have them (gpg2 --locate-keys [email protected] [email protected]).

Option B - git (best if you'll track a branch or apply patches):

git clone https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
cd linux
git checkout v6.11

Use git log --oneline v6.10..v6.11 -- drivers/net (swap the path) to see what changed in an area you care about, and git bisect if you're chasing a regression.

Which version to pick: the mainline tag on kernel.org is bleeding edge; a stable (X.Y.Z) release gets backported fixes on top of a mainline base; longterm releases (LTS, currently maintained for several years) are the safest choice for a machine you don't want to keep rebuilding. Match your distro's kernel config baseline where possible - it saves configuration work in the next step.

3. Configuring the kernel¶

Start from a known-good baseline instead of a blank config:

# Reuse the config of the kernel you're currently running
zcat /proc/config.gz > .config     # if your distro kernel exposes it
# or, if not exposed:
cp /boot/config-$(uname -r) .config

# Bring it forward to the new source tree's options,
# prompting only for genuinely new symbols
make olddefconfig      # accepts defaults for new options, non-interactive
# or
make oldconfig         # prompts you for every new option, interactive

Trim to just the modules your hardware actually loads (drastically cuts build time and image size):

lsmod > /tmp/lsmod.now
make LSMOD=/tmp/lsmod.now localmodconfig

Review or adjust the result:

make menuconfig     # ncurses UI
make nconfig        # newer ncurses UI, better search (press '/')
make xconfig        # Qt GUI, needs libqt5-dev or similar

Options worth checking deliberately:

• CONFIG_LOCALVERSION - set a suffix (e.g. -custom) so uname -r clearly identifies your build and it won't collide with distro kernel filenames.
• CONFIG_MODULE_SIG / CONFIG_MODULE_SIG_FORCE - module signing; needed if Secure Boot enforcement is on (see Section 6).
• CONFIG_DEBUG_INFO variants - leave off for a production/performance build; debug info roughly doubles build time and image size.
• CONFIG_PREEMPT vs CONFIG_PREEMPT_VOLUNTARY vs CONFIG_PREEMPT_NONE - low-latency desktop/RT workloads vs server throughput.
• Filesystem and storage driver options (CONFIG_EXT4_FS, CONFIG_BTRFS_FS, CONFIG_NVME_CORE, your disk controller's driver) - if these are missing or built as modules with no matching initramfs entry, the new kernel won't find its root filesystem at boot.
• Network driver for your actual NIC, and CONFIG_WIREGUARD / CONFIG_NETFILTER family if you rely on them.

Save early and often - make menuconfig writes .config on exit, but keep a backup (cp .config .config.bak) before big experiments so you can diff or restore.

4. Building the kernel and modules¶

make -j$(nproc)

• -j$(nproc) runs one compile job per CPU thread; on memory-constrained machines, cap it (e.g. -j4) to avoid the OOM killer during linking.
• This single target builds vmlinux, the compressed boot image (bzImage on x86, Image/Image.gz on arm64), and all modules marked m in .config.
• Typical wall time: a few minutes on a modern desktop with localmodconfig trimming, 30-60+ minutes for a full allmodconfig-style build.
• Watch for WARNING: lines about missing symbols or unmet dependencies - the build usually completes anyway, but they flag config problems worth fixing before you rely on the result.
• To rebuild just a subsystem after a config tweak: make M=drivers/net/ethernet/intel modules compiles only that directory's modules.
• make clean removes build objects but keeps .config; make mrproper wipes everything including .config - back up first.

Speeding up repeat builds:

sudo apt install ccache      # or dnf/pacman equivalent
export PATH=/usr/lib/ccache:$PATH

ccache caches object files by source+flags hash, so an incremental rebuild after a small source change or config toggle only recompiles what actually changed.

5. Installing modules and the kernel image¶

sudo make modules_install       # installs to /lib/modules/<version>/
sudo make install               # installs vmlinuz, System.map, config, and
                                 # (on most distros) triggers initramfs generation + bootloader hooks

make install on Debian/Ubuntu and Fedora/RHEL runs distro hook scripts (kernel-install, /etc/kernel/postinst.d/*) that copy the image into /boot, build the initramfs, and update the bootloader menu automatically. Arch's make install does not do this - you build and register the initramfs and boot entry manually there (Section 6).

If the initramfs wasn't generated automatically:

# Debian/Ubuntu
sudo update-initramfs -c -k 6.11.0-custom

# Fedora/RHEL
sudo dracut --force /boot/initramfs-6.11.0-custom.img 6.11.0-custom

The initramfs is what lets the kernel find and mount your real root filesystem before the bulk of drivers are loaded - if your root disk controller or filesystem is compiled as a module rather than built-in, a missing or stale initramfs is the single most common cause of a new custom kernel failing to boot.

6. Bootloader integration¶

GRUB (most distros):

sudo grub-mkconfig -o /boot/grub/grub.cfg      # Debian/Ubuntu, Arch
sudo grub2-mkconfig -o /boot/grub2/grub.cfg    # Fedora/RHEL

This scans /boot for kernel images and regenerates the boot menu, adding an entry for your new build alongside the existing distro kernel(s) - it does not remove the old entries, so you keep a fallback automatically.

Arch specifics: after make install copies vmlinuz-custom and you generate an initramfs (mkinitcpio -k 6.11.0-custom -g /boot/initramfs-custom.img), you must also run grub-mkconfig yourself, or add a manual boot entry if using systemd-boot (/boot/loader/entries/custom.conf).

Secure Boot: if Secure Boot is enabled in firmware, an unsigned custom kernel will be rejected at boot. Either sign it with your own Machine Owner Key (MOK) enrolled via mokutil, or disable Secure Boot in firmware for a build/test machine. Signing modules matters here too if CONFIG_MODULE_SIG_FORCE is set - unsigned modules will fail to load even if the kernel itself boots.

systemd-boot (used by some Arch and minimal setups instead of GRUB): entries live under /boot/loader/entries/*.conf; bootctl list shows what's currently registered, and you add a new stanza pointing at your custom vmlinuz/initramfs pair.

7. Booting, verifying, and rollback¶

Reboot and select the new entry from the GRUB menu (hold Shift, or Esc on some systems, if the menu is hidden by default). Once booted:

uname -r                         # confirms you're on the custom build
dmesg | grep -i error            # scan for driver/hardware errors
lsmod                            # confirm expected modules loaded
systemctl --failed               # any services that failed to start

Check the specific hardware you care about - networking (ip a), storage (lsblk, mount), and GPU/display if applicable - since a missing driver often shows up as "device present but not working" rather than a boot failure.

Rollback: your distro kernel entry in GRUB is untouched by default, so if the custom kernel fails to boot or misbehaves, reboot and select the old entry from the menu. Keep at least one known-good kernel installed at all times - don't remove your distro's stock kernel package until you've run the custom one successfully for a while. If GRUB itself won't show a menu, most distros let you interrupt boot by holding Shift (BIOS) or tapping Esc repeatedly right after POST (UEFI).

8. Troubleshooting and repeat-build tips¶

Packaging instead of raw make install (recommended if you rebuild often):

# Debian/Ubuntu - produces installable .deb packages
make -j$(nproc) bindeb-pkg
sudo dpkg -i ../linux-image-6.11.0-custom_*.deb

# Fedora/RHEL - produces .rpm via the kernel spec tooling
make -j$(nproc) binrpm-pkg
sudo rpm -ivh ~/rpmbuild/RPMS/x86_64/kernel-6.11.0-custom*.rpm

Packaging gives you clean dpkg -r/rpm -e removal, automatic bootloader hook integration, and an easy way to distribute the same build to multiple machines - generally nicer than bare make install once you're past the first experimental build.

Diffing configs between kernel versions or against a baseline:

scripts/diffconfig .config.old .config

Useful when a distro releases a new kernel and you want to see exactly which options changed before re-tuning your custom config.