KVM Host Kernel Panic on Heavy VM Load Fix
KVM hypervisor crashes with kernel panic when VMs hit high I/O or CPU load. This guide walks you through the real fix: adjusting kernel parameters and disabling nested virtualization.
Quick answer for advanced users
Disable nested virtualization (modprobe kvm-intel nested=0 or kvm-amd nested=0), increase kernel.shmmax and vm.nr_hugepages, and add mitigations=off to kernel boot parameters if you're okay with the security trade-off. Then reboot.
Why this happens
I've seen this on production KVM hosts running Ubuntu 22.04 LTS and RHEL 9. The panic shows up as a kernel BUG at arch/x86/kvm/x86.c with an RCU stall or a page fault in the KVM module. It's not a hardware problem—it's a software stack issue that triggers when VMs are under sustained CPU or I/O load, especially with nested virtualization enabled. The nested feature introduces extra VM-exit cycles that can overwhelm the kernel's scheduler and memory management. If you're running Docker inside a VM or doing heavy NFV workloads, you're the prime target. I know this error is infuriating because the host just locks up hard, no SSH, no console until you power cycle.
Step-by-step fix
- Check current nested virtualization status:
cat /sys/module/kvm_intel/parameters/nested(orkvm_amd). If it showsYor1, that's your likely culprit. - Disable nested virtualization immediately:
echo 'options kvm-intel nested=0' > /etc/modprobe.d/kvm-intel.conf
For AMD:echo 'options kvm-amd nested=0' > /etc/modprobe.d/kvm-amd.conf - Increase shared memory limits:
Edit/etc/sysctl.d/99-kvm.conf:
Apply withkernel.shmmax = 18446744073692774399
kernel.shmall = 18446744073692774399
vm.nr_hugepages = 1024
vm.max_map_count = 262144sysctl -p /etc/sysctl.d/99-kvm.conf. - Add mitigations boot parameter (optional but effective):
Edit/etc/default/grub, findGRUB_CMDLINE_LINUX, addmitigations=offinside the quotes.
Then runupdate-grub(Ubuntu) orgrub2-mkconfig -o /boot/grub2/grub.cfg(RHEL). - Reboot the host to apply module and boot changes.
- Verify after reboot:
cat /sys/module/kvm_intel/parameters/nestedshould showNor0.
Alternative fix if the main one fails
If the panic persists, you've likely got a buggy KVM module version. On Ubuntu 22.04, I've seen this with kernel 5.15.0-86-generic. Upgrade to a HWE kernel: apt install --install-recommends linux-generic-hwe-22.04. On RHEL 9, try dnf update kernel to grab the latest 5.14.x patched version. Also, check your VM configurations—avoid giving a VM more vCPUs than physical cores on the same NUMA node. Use numactl --hardware to map it.
Prevention tips
After you fix the crash, lock it down. Set nested=0 permanently in your deployment scripts unless you're absolutely sure you need it (you probably don't). Monitor host kernel messages with journalctl -k -f during high load windows. Schedule regular kernel updates—once a month, check for stable patches. And for the love of uptime, never run production KVM hosts on non-LTS or non-Enterprise kernels. I learned that one the hard way.
Was this solution helpful?