Linux Multipath Failover Delay Fix: Stop the 30-Second Wait
Aggressive failover delay on multipath devices? Here's the real fix. Stop waiting 30+ seconds after a path drops. Edit multipath.conf and reload.
Quick answer for advanced users
Set poll_interval to 5 seconds and fast_io_fail_tmo to 10 in /etc/multipath.conf, then run systemctl reload multipathd.
Why your multipath failover feels like forever
I know this error is infuriating. You lose a path on your SAN, and then your apps hang for 30 seconds or more. I've seen this on CentOS 7 machines with Dell MD arrays and on Ubuntu 20.04 with NetApp storage. The default multipathd settings are conservative — they prioritize stability over speed. But in production, those 30 seconds feel like an hour.
The main culprit? poll_interval defaults to 10 seconds. That means multipathd checks paths every 10 seconds. When a path dies, it must miss one check and then wait another 10 seconds to retest. Throw in SCSI error handling timeouts, and you're looking at 20–40 seconds before the failover kicks in.
This tripped me up the first time too. A SAN switch went down, and our database cluster went into a split-brain because the failover took so long. I spent a weekend tuning these parameters. Here's what actually works.
Step-by-step fix
- Check your current multipath config: Run
multipath -lland look forpoll_intervalandpath_checker. If they're missing, you're using defaults. - Edit multipath.conf: Open
/etc/multipath.confwith your editor. Add or update thedefaultssection like this:
defaults { poll_interval 5 path_checker tur fast_io_fail_tmo 10 dev_loss_tmo 20 user_friendly_names no } - Reload multipathd: Run
systemctl reload multipathd. No reboot needed. Check the logs withjournalctl -u multipathd -n 20for errors. - Test the failover: Simulate a path failure by pulling a cable or using
echo offline > /sys/block/sdX/device/state. Watchmultipath -ll— it should show the failover within 5–10 seconds now.
When the main fix doesn't work
If you still see delays after changing poll_interval, check these:
- SCSI timeout is too long: Default
sd_modtimeout is 30 seconds. Lower it to 10 by addingecho 10 > /sys/block/sdX/device/timeoutfor each disk, or set it permanently in/etc/udev/rules.d/99-scsi-timeout.rules. - Path checker is wrong: The
tur(Test Unit Ready) checker is fast. The defaultdirectiois slower because it does a real I/O. Switch toturif you haven't. - Multipathd is stuck: Sometimes the daemon hangs. Run
systemctl restart multipathd— this forces a full restart, not just a reload.
Prevention tip — set it and forget it
Once you tune these values, they stick across reboots. But here's the catch: if you update your kernel or multipath-tools package, the config may get overwritten. Backup your /etc/multipath.conf file. I keep a copy in /root/multipath.conf.backup and run a cron job to check the file hash nightly. Overkill? Maybe. But after one botched update took out our failover for 45 minutes, I don't take chances.
Also, verify your SAN vendor's recommendations. Some arrays like EMC or HP 3PAR have specific path_grouping_policy settings. For most setups, multibus is fine, but if you use ALUA, stick with group_by_prio. Mixing these up can actually make failover slower.
That's it. Your failover should be under 10 seconds now. If it's not, check the SCSI timeouts again — that's usually the hidden problem.
Was this solution helpful?