Mainline Linux kernel — automated build & install for x86 servers

Builds the latest kernel.org mainline (currently 7.1-rcN), stable (7.0.x), or longterm (6.18.x / 6.12.x) into proper Debian packages and installs them through your normal apt flow. Tuned per-CPU.

Supports

CPU family	CPU_ARCH	Example boxes
Intel Sapphire Rapids (4th Gen Xeon Scalable, Xeon Max HBM)	spr	Xeon Max 9480
Intel Emerald Rapids (5th Gen)	emr	uses cpu-spr.config
Intel Granite Rapids (6th Gen)	gnr	uses cpu-spr.config
AMD Zen 2 — EPYC 7002 "Rome", Ryzen 3000	zen2	EPYC 7702, 7H12 …
AMD Zen 3 — EPYC 7003 "Milan", Ryzen 5000	zen3	EPYC 7763, 7773X …
AMD Zen 4 — EPYC 9004 "Genoa"/"Bergamo", Ryzen 7000	zen4	EPYC 9654, 9754 …
AMD Zen 5 — EPYC 9005 "Turin", Ryzen 9000, Strix	zen5	EPYC 9965, 9755 …

Tested on Ubuntu 22.04, 24.04, 26.04 — the scripts detect the OS version and adjust deps + toolchain accordingly. On 22.04 (gcc 11) building for Zen 4 / Zen 5 will trigger an automatic apt install gcc-13 / gcc-14.

Files

Path	What it is
kernel-build.conf	Central config: kernel channel, CPU_ARCH (auto or explicit), LOCALVERSION (auto or explicit), STRIP_DEBUG_INFO, ccache, Clang, paths.
kernel-build.local.conf	Optional, gitignored. Per-host overrides — sourced after kernel-build.conf. Lets each box pin its own settings without dirtying the shared config.
config/common.config	Kconfig fragment applied on every build: NUMA, KVM, BPF, BBR, FS, signing-disabled, schedulers, hardening.
config/cpu-spr.config	Sapphire Rapids overlay: MNATIVE_INTEL, INTEL_PSTATE, AMX/CET, INTEL_IDXD/IAA/QAT, KVM_INTEL_TDX, INTEL_IOMMU.
config/cpu-zen2.config	Zen 2 overlay (EPYC 7002): MNATIVE_AMD, AMD_PSTATE, AMD_IOMMU, KVM_AMD_SEV, AMD_HSMP.
config/cpu-zen3.config	Zen 3 overlay (EPYC 7003): + AMD branch-sampling, SEV-SNP.
config/cpu-zen4.config	Zen 4 overlay (EPYC 9004): + AVX-512, AMD_PSTATE_EPP, full SNP.
config/cpu-zen5.config	Zen 5 overlay (Turin / Strix): MZEN5 (with MNATIVE_AMD fallback), full 512-bit AVX-512.
lib/common.sh	Shared helpers: logging, dep installer, kernel.org JSON parsing, GPG keyring, CPU + Ubuntu detection, gcc auto-upgrade.
build-all.sh	Detect CPU + Ubuntu → install deps → ensure gcc version → fetch + verify kernel → compose .config → make bindeb-pkg.
install-all.sh	Publish .debs to local apt repo at apt-repo/, register source list, grant _apt traverse ACL, apt install, update-grub.
clean.sh	Three tiers — light / full / nuke. Always re-fetches kernel.org metadata and re-verifies the signature.
workdir/	All build state. Gitignored. Safe to delete (clean.sh nuke).
apt-repo/	Local APT repo served via file://. Gitignored.

Usage

cd /path/to/linux-kernel    # works from any clone location

./build-all.sh              # auto-detects CPU + OS + kernel
./install-all.sh            # publishes to local repo + apt-installs
sudo reboot

After the first install the local apt repo is registered as a permanent source, so subsequent updates are just:

./build-all.sh && ./install-all.sh
# — or —
./build-all.sh && sudo apt update && sudo apt upgrade

Per-host configuration

The shared kernel-build.conf sets sensible defaults (CPU_ARCH=auto, LOCALVERSION="" → derived as -<arch>-<channel>-rex). To override per-box without forking the file, drop a kernel-build.local.conf next to it:

# kernel-build.local.conf  (gitignored)
KERNEL_CHANNEL="stable"        # stick with stable on this prod box
CPU_ARCH="zen4"                # force, in case auto-detection ever misbehaves
JOBS=64                        # leave some cores for other workloads
STRIP_DEBUG_INFO="y"           # smaller image package on this disk-constrained box

Kernel naming examples

LOCALVERSION is auto-derived as -<CPU_ARCH>-<KERNEL_CHANNEL>-rex so each box's kernel is unambiguously identifiable:

Box	Channel	Resulting kernel name
Xeon Max 9480	mainline	7.1.0-rc1-spr-mainline-rex
EPYC 7763	stable	7.0.2-zen3-stable-rex
EPYC 9654	mainline	7.1.0-rc1-zen4-mainline-rex
EPYC 9755	longterm	6.18.25-zen5-longterm-rex

Verifying after reboot

uname -r                                                                # e.g. 7.1.0-rc1-spr-mainline-rex
zcat /proc/config.gz | grep -E '^CONFIG_(MNATIVE|MZEN|X86_AMD_PSTATE|INTEL_IDXD)='
grep -oE 'avx512_bf16|avx512f|amx_bf16|amx_int8|avx512_vp2intersect' /proc/cpuinfo | sort -u
dmesg | grep -iE 'amx|tdx|cxl|sapphire|amd-pstate|sev'

Cleaning

./clean.sh           # default: clear object files + staged debs
./clean.sh full      # also wipe extracted source + build dir
./clean.sh nuke      # also wipe tarball cache, git mirror, GPG keyring, apt-repo pool

Every level always re-fetches kernel.org metadata and re-verifies the signature so the cache is left trustworthy.

Toolchain auto-management

Target	Requires gcc	Action on Ubuntu 22.04 (gcc 11)
spr / emr / gnr / zen2 / zen3	gcc ≥ 11	nothing — system gcc is fine
zen4 / gnr	gcc ≥ 13	auto apt install gcc-13 + CC=gcc-13
zen5	gcc ≥ 14	auto apt install gcc-14 + CC=gcc-14

build-all.sh detects the running gcc version, compares against the target arch's requirement, and installs the newer compiler if needed. The system's default gcc is left alone — only the build picks up gcc-N.

GPU drivers (NVIDIA / AMD / out-of-tree DKMS)

install-all.sh auto-detects GPUs, installs/upgrades the right NVIDIA driver flavor, and rebuilds every registered DKMS module for the new kernel. Toggle off with MANAGE_DKMS="n" in kernel-build.conf if you manage drivers manually.

NVIDIA driver source — auto-selected by KERNEL_CHANNEL

KERNEL_CHANNEL	NVIDIA source path	Why
mainline (e.g. 7.1-rcN)	upstream git — clone NVIDIA/open-gpu-kernel-modules and build from source	Mainline-RC kernels often need post-release compat patches that haven't shipped in any released .run / .deb yet
stable (e.g. 7.0.x)	apt DKMS — nvidia-dkms-N${NVIDIA_FLAVOR}	Mature: apt-shipped DKMS source builds cleanly against stable kernels
longterm (e.g. 6.18.x / 6.12.x)	apt DKMS — nvidia-dkms-N${NVIDIA_FLAVOR}	Same — apt has driver versions known to compile against the LTS API

Override the auto-selection in kernel-build.conf:

NVIDIA_DRIVER_SOURCE="auto"      # default — pick by KERNEL_CHANNEL (recommended)
# NVIDIA_DRIVER_SOURCE="upstream"  # always build from upstream git
# NVIDIA_DRIVER_SOURCE="apt"       # always use apt DKMS package

Upstream-git path (mainline channel)

When the upstream path is selected, install-all.sh:

1. Determines the latest userspace version from apt (nvidia-utils-N${NVIDIA_FLAVOR} Version field).
2. Installs only userspace from apt: nvidia-headless-no-dkms-N${NVIDIA_FLAVOR}, libnvidia-*-N, nvidia-utils-N. The -headless-no-dkms-* variant is specifically designed not to pull nvidia-dkms-*, so apt's postinst won't trigger a DKMS build that would fail.
3. Clones https://github.com/NVIDIA/open-gpu-kernel-modules.git into workdir/sources/open-gpu-kernel-modules/.
4. Checks out a ref according to NVIDIA_GIT_REF in kernel-build.conf:
   • "auto" (default): tag matching apt userspace version (ABI guaranteed match), or main branch if no matching tag exists
   • "main": always main branch (latest fixes, possibly pre-release)
   • "595.58.03" etc.: pin to a specific tag
5. Runs make modules SYSSRC=/lib/modules/<new-kernel>/build to compile against the new kernel's headers.
6. Runs sudo make modules_install to drop the .kos into /lib/modules/<new-kernel>/.
7. Runs depmod -a <new-kernel>.

Build log goes to workdir/logs/nvidia-upstream-<kernel>-<time>.log for diagnostics.

apt-DKMS path (stable / longterm)

On first run with an NVIDIA GPU detected, the script:

1. Scans apt-cache for the latest nvidia-dkms-N${NVIDIA_FLAVOR} branch (default NVIDIA_FLAVOR="-server-open" = LTS server-certified + open kernel modules — the recommended combo for Turing+ GPUs in production).
2. Compares against your currently installed driver branch.
3. If they differ, prompts you (30-second timeout, default = upgrade):

   [?] Upgrade NVIDIA driver branch 595 -> 600 ?
       [Y/n/k(eep current)] (30s timeout, default=Y):
   

4. Saves your choice to workdir/.gpu-driver-state (just the branch number).
5. Installs nvidia-driver-N${NVIDIA_FLAVOR} + nvidia-dkms-N${NVIDIA_FLAVOR}
   • nvidia-utils-N — replacing Ubuntu's pre-built linux-modules-nvidia-*-generic that only works on stock kernels.
6. If the DKMS source is on disk but unregistered (dkms status empty), it apt install --reinstalls the dkms package to re-trigger dkms add.

Subsequent runs skip the prompt and just maintain the saved branch. Re-prompt by deleting workdir/.gpu-driver-state — or pin a different branch by writing the desired number into that file.

Override the flavor in kernel-build.conf:

NVIDIA_FLAVOR	What you get	Use when
-server-open (default)	LTS server branch + open kernel modules	Production servers with Turing+ GPUs
-server	LTS server branch + proprietary kernel modules	Older GPUs (Maxwell/Pascal) on servers
-open	Desktop branch + open kernel modules	Desktops with Turing+ GPUs
""	Desktop branch + proprietary kernel modules	Older desktop GPUs

AMD

amdgpu is in-tree. install-all.sh detects an AMD GPU (if present) and just prints the device — nothing to install, the kernel module is already part of the build. AMDGPU-PRO is rare and not auto-managed.

Generic DKMS rebuild

After the NVIDIA-specific step, the script runs dkms autoinstall -k <new-kernel> which rebuilds every registered DKMS module — NVIDIA, ZFS, VirtualBox, V4L2 loopback, anything else — for the new kernel. Then depmod -a and update-initramfs -u -k <new-kernel> fold them into the boot image.

Manual recovery if DKMS got into a weird state

Symptom: nvidia-smi says "NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver" after a fresh boot.

# Re-run the dkms package's postinst (re-registers source under /var/lib/dkms/)
sudo apt install --reinstall nvidia-dkms-<flavor>      # e.g. nvidia-dkms-595-server-open
dkms status                                            # should now list nvidia/X.Y.Z

# Force-build for the running kernel
sudo dkms autoinstall -k "$(uname -r)"
sudo depmod -a "$(uname -r)"
sudo modprobe nvidia nvidia_uvm nvidia_modeset nvidia_drm
nvidia-smi

If dkms autoinstall fails (typical on cutting-edge mainline RCs where the NVIDIA driver hasn't caught up to a renamed kernel symbol), check the build log at /var/lib/dkms/nvidia/X.Y.Z/build/make.log. Two common workarounds:

• Pin KERNEL_CHANNEL="stable" in kernel-build.conf to use the latest kernel.org stable instead of mainline-RC.
• Pull a newer nvidia-open source from upstream and dkms add it manually (Ubuntu's packaged version may lag behind for new kernels).

What's NOT done by these scripts

• Secure Boot signing. Assumes Secure Boot is off on every target box. Verify with mokutil --sb-state before running. If you need signed kernels, you'll have to add MOK enrollment + sbsign of vmlinuz + module signing, none of which is scripted here.
• AMDGPU-PRO (the proprietary AMD add-on). The free amdgpu is in-tree and Just Works — nothing to do.
• Removing old kernels. Kept around as fallbacks. Run sudo apt autoremove --purge once you're confident the new one boots.

Troubleshooting

• PGP verification failed. Either the kernel.org keys haven't been fetched (network issue on first run), or the tarball is corrupt. Re-run ./clean.sh full to force a re-download + re-verify.
• Wrong CPU architecture detected. Override with CPU_ARCH="zen4" (or whatever) in kernel-build.local.conf. Auto-detect uses CPU family + model
  • /proc/cpuinfo flags; if you've got an unusual core count or an unreleased variant, force it explicitly.
• bindeb-pkg complains about signing key. Shouldn't happen — common.config sets CONFIG_SYSTEM_TRUSTED_KEYS="" and CONFIG_MODULE_SIG=n. If it does, grep MODULE_SIG workdir/build/.config to confirm those are n.
• Build runs out of disk. ~30 GB needed for a debug-info build, ~6 GB without. df -h <WORK_DIR> to check.
• _apt permission warnings during apt update. Normally fixed by the ACL step in install-all.sh. If you moved the repo to a path whose entire chain isn't _apt-traversable, re-run install-all.sh to re-apply ACLs.
• gcc-13 / gcc-14 not in apt. On stock Ubuntu 22.04 minimal you may need sudo apt install software-properties-common && sudo add-apt-repository ppa:ubuntu-toolchain-r/test && sudo apt update first.