Fix ERROR_IPSEC_IKE_DH_FAIL (0X000035FE) – Diffie-Hellman Failure
This error means the Diffie-Hellman key exchange failed during IPsec negotiation. The culprit is almost always a mismatch in DH group settings between two VPN endpoints.
30-Second Fix: Check the VPN Client's DH Group
Open PowerShell as admin and run:
Get-VpnConnection -Name "your VPN name" | fl *dh*If you see DHGroup set to None or Group1, that's your problem. Windows defaults to Group2 (1024-bit MODP) for older connections, and many modern VPN servers require at least Group14 (2048-bit MODP).
The quickest fix? Delete and recreate the VPN connection from scratch — Windows will pick a sane DH group. Go to Settings > Network & Internet > VPN, remove the broken connection, add it again with the same server and credentials. Takes 30 seconds. 9 times out of 10, this is all you need.
5-Minute Fix: Force a Specific DH Group via Registry
If recreating the connection didn't help, the culprit is likely a registry policy overriding the DH group. Open regedit and go to:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent\Look for a DWORD named AssumeUDPEncapsulationContextOnSendRule — that's for NAT traversal, not DH. The real fix is adding a DWORD called StrongerDHGroup (if it's not there, create it). Set it to 2 for Group14 (2048-bit) or 3 for Group24 (2048-bit with 256-bit prime). Reboot.
Still broken? Check the Windows IKE policy:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\IKEEXT\Parameters\Add DWORD DHGroup with value 14 (decimal). This forces the IKE service to use Group14 for all outgoing negotiations. Reboot again. If you're connecting to a Cisco ASA or a strongSwan server running on Ubuntu 20.04+, Group14 is the sweet spot — it's secure and widely supported.
15+ Minute Fix: Reconfigure the VPN Server's DH Settings
If the client's clean and you're still getting the error, the server is the problem. I've seen this happen most often with old NetScaler VPX appliances (firmware 11.0 or earlier) and certain Check Point gateways. They default to Group1 (768-bit) or Group2 (1024-bit), and Windows 10/11 with latest patches rejects those because they're weak.
On a Windows Server VPN server (RRAS), open Server Manager, go to Remote Access > Configuration, and set the IKEv2 DH group to Group14 in the custom IPsec policy. On a strongSwan server, edit /etc/ipsec.conf and add:
esp=aes256-sha256-modp2048!
ikephase2=aes256-sha256-modp2048!Then restart the service:
sudo systemctl restart strongswanFor Cisco ASA (8.4+), run:
crypto ikev2 policy 1
group 14Don't forget to apply the change with write memory.
One more thing: if your VPN server sits behind a NAT device (like a home router), you also need to check the NAT traversal setting. Sometimes the DH failure is a red herring — the real issue is UDP encapsulation being blocked. Set the server's NAT-T keepalive to 10 seconds. On Windows RRAS:
netsh ras ip set nat enableNot a guarantee, but I've seen it fix exactly this error on a 2019 server that refused to negotiate with a Windows 11 client.
Still Stuck?
Check the event logs. Open Event Viewer > Windows Logs > System and filter on source IKEEXT. Look for Event ID 4651 (IKE failure). It'll tell you the exact DH group the server offered — and what the client expected. That mismatch is your answer.
If you're on a Mac or Linux client connecting to Windows, you'll need to check the IKE daemon logs. On Linux, journalctl -u strongswan -f during reconnection shows the DH group negotiation in plain text. Don't guess — read the logs.
Was this solution helpful?