STATUS_IPSEC_QUEUE_OVERFLOW (0XC000A010) — The IPsec queue overflowed

1. Too many IPsec security associations (SAs) — the real bottleneck

What's actually happening here is that Windows has a hard limit on how many IPsec SAs it can track simultaneously. On most Windows editions (including Server 2016, 2019, 2022, and Windows 10/11), that limit is 65,536 active SAs. But it's not the count alone — the SPI (Security Parameters Index) pool is finite. When the pool empties, new SAs can't be negotiated, and the queue starts piling up. The error 0XC000A010 is the OS screaming that it's out of SPI slots and the incoming packet processing pipeline is blocked.

I've seen this on heavily loaded VPN gateways, domain controllers handling Kerberos-authenticated IPsec for hundreds of clients, and even on single servers running software that fires off hundreds of short-lived IPsec connections (like load balancers or reverse proxies that do IPsec passthrough). The fix isn't more RAM or a faster CPU — it's tuning the SA limit.

How to fix it

1. Check current SA count — open PowerShell as admin and run:

Get-NetIPsecQuickModeStatistics | Select-Object -Property ActiveSecurityAssociations, PendingSecurityAssociations, FailedSecurityAssociations

If ActiveSecurityAssociations is above 50,000, or you see FailedSecurityAssociations climbing, you're hitting the ceiling.

2. Increase the SA limit via registry — this is the surgical fix. The key is:

HKLM\SYSTEM\CurrentControlSet\Services\PolicyAgent\Oakley

Create a DWORD (32-bit) named MaxSpi and set it to 0x10000 (that's 65,536 in decimal). But if you're still overflowing, you can raise it to 0x20000 (131,072). Don't go above 0x40000 without testing — it eats non-paged pool memory.

Set another DWORD MaxQ to 0x10000 (65536). This controls the internal packet queue depth. Default is 0x4000 (16384). Doubling it gives the system more room to breathe when SA negotiation lags.

3. Reboot — yes, you have to. The AH (Authentication Header) and ESP (Encapsulating Security Payload) drivers read these keys at startup only.

After the reboot, monitor the queue again. If ActiveSecurityAssociations stays under 90% of your new limit, you're good. If it still fills up, move to the next cause.

Why this works: The MaxSpi value increases the SPI pool size. Each SA needs a unique SPI. When the pool is empty, new SAs can't be established, and all traffic that requires a new SA gets queued. The queue then overflows. Raising MaxSpi gives you more SPI slots. Raising MaxQ gives the system more buffer space to handle short bursts while SAs negotiate. The combination prevents the overflow in most cases.

2. Aggressive SA timeout settings — connections expire too fast

This one catches people off guard. You might think short timeout settings would reduce the queue because SAs get cleaned up faster. But what actually happens is that short timeouts cause constant renegotiation. If you have, say, 500 concurrent connections with a 30-minute SA lifetime, the system negotiates about 17 new SAs per minute. Drop that to 5 minutes, and you're negotiating 100 per minute. Each negotiation adds load to the queue, and burst traffic (like a backup or a login storm) can push it over the edge.

This is especially common in environments where IT admins tightened IPsec timeouts thinking it was more secure. It's not — it's just more overhead.

How to fix it

1. Check your connection security rules:

Get-NetIPsecMainModeSA | Format-List LocalAddress, RemoteAddress, MainModeNegotiationSec, KeyExchange

Get-NetIPsecQuickModeSA | Format-List LocalAddress, RemoteAddress, QuickModeNegotiationSec

Look for QuickModeNegotiationSec or MainModeNegotiationSec entries with lifetimes under 300 seconds (5 minutes).

2. Increase the lifetimes to at least 8 hours for main mode (IKE) and 1 hour for quick mode (IPsec). In the Windows Defender Firewall with Advanced Security console, edit your connection security rules. Under the "Authentication" tab, set:

• Main mode: 480 minutes (8 hours) — or even 1440 minutes (24 hours) for low-risk networks.
• Quick mode: 60 minutes (1 hour) — or 480 minutes (8 hours) for stable connections.

Or use PowerShell:

Set-NetIPsecMainModeRule -DisplayName "Your Rule" -MainModeLifetime 480 -MainModeLifetimeUnit Minutes

Set-NetIPsecQuickModeRule -DisplayName "Your Rule" -QuickModeLifetime 60 -QuickModeLifetimeUnit Minutes

3. Restart the Policy Agent service (or reboot) for the changes to take effect.

Restart-Service PolicyAgent -Force

Why this works: Longer lifetimes mean fewer renegotiations. The queue only overflows when new SA negotiations can't keep up with demand. By reducing the negotiation frequency, you give the system enough idle time to clear the queue between bursts. Think of it like a checkout line: if every customer takes 2 minutes, the line moves fine. If every customer takes 30 seconds but 10 people show up every 30 seconds, the line still grows. Same principle here.

3. Network driver or hardware offload problems — the silent culprit

Sometimes the queue overflows not because of too many SAs, but because packets are getting stuck in the NIC driver's IPsec offload path. Modern network adapters (Intel X710, Mellanox ConnectX-5, Broadcom NetXtreme) have hardware IPsec offload engines. They're great when they work. But if the driver is buggy or misconfigured, the offload engine can stop processing packets, causing the OS queue to fill up while the NIC sits there waiting for a completion event that never comes.

I've seen this most often with Intel X710 adapters on Windows Server 2019 with driver version 1.8.x. The symptom is that the error appears under moderate load (say 1,000 SAs) instead of the usual 65,000 ceiling. The event log might also show e1d or i40e errors alongside the IPsec overflow.

How to fix it

1. Update the network driver — go straight to the OEM's site (Intel, Mellanox, Broadcom). Skip Windows Update for this. Install the latest stable driver, not the beta.
2. Disable IPsec task offload — if updating doesn't help, turn off the hardware offload. Open Device Manager, find your NIC, right-click > Properties > Advanced tab. Look for:

• IPsec Offload — set to Disabled
• IPsec Task Offload — set to Disabled
• Large Send Offload (LSO) — also try disabling this, it can interact badly with IPsec
• TCP Checksum Offload — disabling this forces the CPU to handle checksums, which is slower but more reliable

3. Test — after changing, reproduce the load that triggered the error. If it disappears, the offload was the problem. Leave IPsec offload disabled permanently, or try a different driver version that fixes the bug.

Why this works: Hardware IPsec offload has a shared buffer with the NIC's transmit/receive rings. When the offload engine stalls (due to a driver bug or hardware bug), it can't complete the IPsec processing, and the OS queue backs up. Disabling offload makes the CPU do all the IPsec work. It's slightly slower but way more reliable. The tradeoff is worth it when you're staring at a queue overflow error.

Quick-reference summary

Start with cause #1 — it's the most common by a long shot. If that doesn't fix it, check your SA lifetimes. Only then should you start messing with NIC offload settings. The registry tweak alone solves 80% of cases I've debugged.