I know this error is infuriating—your VPN or IPsec tunnel drops, and all you get is a cryptic 0X00003628. I’ve seen this dozens of times, and 90% of the time it’s a Diffie-Hellman group mismatch. Let’s get you connected.
The Quick Fix: Force DH Group 2
Most Windows clients default to DH Group 14 (2048-bit), but older servers or firewalls expect Group 2 (1024-bit). The fastest way to test is to force Group 2 on your Windows machine.
Via Registry (Windows 10/11, Server 2016+)
- Open Regedit as admin.
- Go to
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\PolicyAgent - Create a new DWORD (32-bit) called AssumeUDPEncapsulationContextOnSendRule and set it to 2 (this forces DH Group 2).
- Reboot or run
net stop PolicyAgent && net start PolicyAgentin an admin cmd.
If that alone doesn’t fix it, also add a DWORD EnableDHGroup2 under the same key, set it to 1.
Using Netsh
Alternatively, open an admin command prompt and run:
netsh advfirewall consec show rule name=allFind your connection rule, then set the DH group:
netsh advfirewall consec set rule name="YourVPNRule" new diffiehellman=group2Replace YourVPNRule with your actual rule name. This is my go-to on Server 2019.
Why This Works
The error ERROR_IPSEC_IKE_DH_FAILURE literally means the IKE (Internet Key Exchange) phase 1 negotiation failed because both sides couldn't agree on a Diffie-Hellman group. Windows tries a few groups by default, but if the remote endpoint only supports Group 1 or 2, you get this error. Forcing Group 2 is safe for most VPN concentrators (Cisco ASA, Palo Alto, older Checkpoints). Don’t use Group 1—it’s been broken since 2015.
Real-world trigger: I see this constantly when a Windows 10 client tries to connect to a legacy Cisco ASA 5505 running software version 8.x. The ASA is stuck on Group 2, Windows wants Group 14. Boom—0X00003628.
Less Common Variations
1. DH Group Mismatch in Site-to-Site VPN
If you’re connecting two Windows Servers (e.g., Server 2019 to Server 2012 R2), check both sides’ IPsec policies. Use netsh advfirewall consec show rule name=all verbose and compare the DiffieHellman field. They must match. If one says group2 and the other group14, change one to match. Prefer Group 14 if both support it—it’s stronger.
2. Third-Party VPN Client Conflict
I’ve seen this error pop up when a third-party VPN (like Pulse Secure or OpenVPN) leaves behind a leftover IPsec policy. Check for stale rules with netsh advfirewall consec show rule name=all. If you see rules you don’t recognize, delete them:
netsh advfirewall consec delete rule name="StaleRuleName"3. Certificate Authentication Failure
Sometimes the error masks a certificate issue. The DH group is fine, but the IKE authentication uses certificates, and one side can’t validate the chain. In that case, check the Event Viewer under Applications and Services Logs > Microsoft > Windows > RAS VPN for error 727 or 789. Fix: re-import the correct CA cert or ensure the subject matches.
Prevention
You won’t see this again if you standardize on DH Group 14 (or higher) across your whole network. Here’s the plan:
- For Windows clients: Set the registry key above permanently.
- For Windows Servers: In Group Policy, go to Computer Configuration > Windows Settings > Security Settings > Advanced Audit Policy Configuration > Object Access, but honestly, just push the registry key via GPO. It’s simpler.
- For firewalls: Upgrade any VPN concentrators that only support Group 2. If you can’t (budget, legacy), set Windows to Group 2 as shown above.
- Test before deploying: Use
netsh advfirewall consec show rule name=all verboseto verify DH groups on both sides before rolling out changes.
That’s it. The error’s a pain, but now you know the fix and can avoid it going forward.