Disk latency in virtual environments: 3 fixes that work

1. Queue depth saturation: The #1 cause

I've seen this more times than I can count. A client calls me, says their file server VM is crawling. RDP is slow, SQL queries time out. I check disk latency and it's consistently over 100ms. The fix? Queue depth saturation.

Windows VMs, especially older ones (Server 2008 R2, 2012 R2), default to a queue depth of 32 per storage adapter. That's way too high for a single spinning disk or even a RAID array. The hypervisor sees a bunch of requests queued up, and the disk can't keep up. Latency spikes. Reboots don't fix it.

How to check it

On the host (Hyper-V or VMware), look at the datastore or volume IOPS. If you see high average queue length (over 2 per disk) but low IOPS, you've got a queue depth problem. On the VM, check Resource Monitor under Disk. Look at Average Disk Queue Length. If it's over 2 per disk, you've found it.

The fix

Lower the queue depth in the hypervisor. For Hyper-V, edit the VM settings. Under SCSI Controller, you can set the queue depth per virtual disk. Set it to 64 or 128 for SSDs, 32 for spinning disks. For VMware, you can set it in the VMX file or via PowerCLI:

# For VMware - set queue depth per virtual disk
Get-VM -Name "YourVM" | Get-HardDisk | Set-HardDisk -DiskQueueDepth 64

Had a client last month whose entire print queue died because of this. Their file server VM had queue depth set to 128 on a 10K SAS array. Set it to 32, and latency dropped from 150ms to 15ms in minutes. No hardware change.

2. Storage contention from noisy neighbors

Second most common cause: other VMs on the same storage hogging all the IOPS. I've seen a dev VM running a heavy database backup that crushed the shared storage for everyone else. The fix is about isolation.

How to spot it

On the host, check storage latency per datastore or volume. If you see one VM with high latency on its own disks, but the latency on the shared volume is normal, it's the VM's internal issue (see #1 or #3). But if latency on the shared volume is high for all VMs, you've got noisy neighbors.

Look at the host's storage metrics. In vSphere, go to Monitor > Performance > Storage. Sort by IOPS. The VM with the highest IOPS is likely the culprit. Could be a backup job, indexing, a developer running a query. Shut it down temporarily to confirm.

The fix

First, reschedule the noisy job. Move backups to off-hours. For VMs that need constant high IOPS (like databases), use storage I/O control. In VMware, enable SIOC (Storage I/O Control) on the datastore. It automatically throttles VMs that use too many IOPS. In Hyper-V, use Storage QoS policies. Set a maximum IOPS per VM so one can't kill the others.

Also consider using separate datastores for high-IOP workloads. Put your SQL VM on its own SSD LUN. Your backup server can stay on the slower array. That's a simple fix that works.

3. Disk fragmentation inside the VM (yes, it's still a thing)

I know, fragmentation is old-school. But I still see it, especially on VMs that have been running for years without a restart. Windows defrag runs on a schedule, but if the VM has thin provisioned disks or snapshots, defrag can actually make things worse. But if the VM uses thick eager zeroed or fixed VHDX, defrag helps.

How to check it

Inside the VM, run defrag analysis:

# From an admin command prompt
fsutil fragmentation query c:

If fragmentation is over 10-15%, especially on system files, it's contributing to latency. I had a client with a 2012 R2 VM that hadn't been defragged in three years. Fragmentation was 35%. Latency was 80ms at idle. After defrag, latency dropped to 20ms.

The fix

Run a full defrag. But before that, check if the VM has snapshots. If it does, consolidate them first. Snapshots break the VHDX chain and make defrag less effective. Also make sure the virtual disk type is fixed (thick) not thin. Thin disks defrag slower and sometimes don't show full improvement.

Schedule regular defrag with Task Scheduler. Or use PowerShell to run it weekly:

Optimize-Volume -DriveLetter C -Defrag -Verbose

If you're on SSDs, don't defrag. You'll just waste writes. Instead, make sure TRIM is running. On Windows Server 2012 and newer, TRIM runs by default via the Optimize-Volume cmdlet with the -Retrim parameter. Check with:

fsutil behavior query DisableDeleteNotify

If it returns 0, TRIM is on. If 1, turn it on:

fsutil behavior set DisableDeleteNotify 0

Quick reference summary

Cause	Symptom	Fix
Queue depth saturation	High latency on one VM, low IOPS on host	Lower queue depth in VM settings (32 for HDD, 64 for SSD)
Storage contention	High latency on all VMs on same datastore	Use SIOC or Storage QoS, separate high-IOPS VMs
Disk fragmentation	High latency inside VM, low outside	Defrag (only on HDD VMs without snapshots)

If you try all three and still have high latency, check your physical hardware. Could be a failing disk, a bad SAS cable, or a host that's just too old. But in my experience, 80% of the time it's one of these three. Start with #1. That's the one that fixes most issues without buying anything.