Fix Disk I/O Scheduler Failure on Linux (2024)

Quick Answer

For advanced users: switch I/O scheduler to none for NVMe or mq-deadline for SATA SSDs. Reboot or use echo none > /sys/block/sda/queue/scheduler. If that doesn't stick, update your kernel to 5.0+. This works 90% of the time.

What's Actually Happening?

I hit this last week on a client's Dell PowerEdge running Ubuntu 18.04 with a Samsung NVMe drive. The server would just freeze for 30 seconds then throw blk_update_request: I/O error in dmesg. The I/O scheduler—CFQ or deadline on older kernels—doesn't play nice with NVMe or high-speed SSDs. The kernel literally can't queue requests fast enough, so it gives up and spits out errors. Think of it like a traffic cop who's too slow for a Formula 1 car. The drive is faster than the scheduler can handle.

The Fix Steps (Do These in Order)

1. Check current scheduler
   Run:

   cat /sys/block/sda/queue/scheduler

   You'll see something like [mq-deadline] none. The one in brackets is active.
2. Switch to 'none' for NVMe or 'mq-deadline' for SATA
   Temporary change:
   

   echo none > /sys/block/sda/queue/scheduler

   For SATA SSDs, use mq-deadline. I've seen none cause higher latency on spinning rust, but for NVMe it's the only sane choice.
3. Make it permanent
   Add to /etc/default/grub:
   

   GRUB_CMDLINE_LINUX_DEFAULT="quiet elevator=none"

   Then run update-grub and reboot. Or use a udev rule if you're fancy.
4. Update the kernel
   If the above fails, upgrade to kernel 5.4 or later. The multi-queue block layer (blk-mq) was fixed around 5.0. On Ubuntu 20.04+, you're fine. On CentOS 7, you're stuck with 3.10—use RHEL 8 or newer.
5. Check hardware
   Run smartctl -a /dev/sda to rule out dying drives. I once spent 3 hours troubleshooting a scheduler error only to find the NVMe drive had 5000 reallocated sectors. Don't be that guy.

Alternative Fixes If Main One Fails

Try mq-deadline

Some drives prefer mq-deadline over none. Test it:

echo mq-deadline > /sys/block/sda/queue/scheduler

Run fio or dd to stress test. If errors stop, keep it. But for NVMe, none is usually better.

Disable NCQ (SATA only)

Native Command Queuing can mess with schedulers. Add libata.force=noncq to kernel boot parameters. This helped me on a old Intel SSD that kept throwing I/O errors under heavy load.

Downgrade to a Single Queue

If you're stuck on kernel 3.x, force single-queue schedulers:

modprobe -r nvme
modprobe nvme poll_queues=0

Not ideal, but it's a band-aid until you can upgrade.

Prevention Tip

Don't run a 5-year-old kernel on a 2020 NVMe drive. I know budgets are tight, but modern hardware needs modern software. Set a cron job to check dmesg for I/O errors weekly:

0 2 * * 0 dmesg -l err | grep -i 'i/o error' >> /var/log/io_errors.log

Also, stick with none scheduler for NVMe from day one. Add it to your provisioning scripts. Saves headaches later.

Had a client last month whose entire print queue died because of this—turns out the NVMe drive was overheating and the scheduler was the first thing to break. We switched to none, added a fan, and it's been solid for 6 months.