Fix ERROR_IPSEC_IKE_INVALID_HASH (0X0000362E) Fast

Quick answer: Re-enter the pre-shared key on both sides of the tunnel. If that doesn't work, delete and recreate the IPsec policy in the Windows Firewall with Advanced Security snap-in.

This error — ERROR_IPSEC_IKE_INVALID_HASH (0X0000362E) — means the hash check on the IKE authentication payload failed. In plain English: the two devices trying to establish an IPsec tunnel don't agree on the shared secret or the hashing algorithm. I've seen this more times than I can count. The culprit here is almost always a typo in the pre-shared key (PSK) or a mismatch between the algorithms configured (like SHA1 vs SHA256). Don't bother chasing drivers or checking network cables — this is strictly a configuration problem.

This usually pops up when you're setting up a site-to-site VPN between Windows Server and a third-party firewall, or between two Windows machines using IPsec policies. It can also happen after a policy update where someone fat-fingered the key.

Step 1: Verify the Pre-Shared Key

Go to the device that's throwing the error. Open the Windows Firewall with Advanced Security MMC snap-in (wf.msc).

1. Right-click Connection Security Rules and choose Properties.
2. Go to the General tab — look for the rule that applies to your tunnel.
3. Double-click that rule. Under the Authentication tab, check the Pre-shared key field.
4. Write it down exactly as shown — case-sensitive, no trailing spaces.
5. Now go to the other end of the tunnel (the VPN peer, firewall, or other Windows box). Check its PSK setting.
6. They must match character-for-character. If they don't, update one side to match the other.

Pro tip: I've seen people copy a PSK from an email and accidentally include a stray space or a zero that looks like an O. Re-type it manually rather than copy-paste if you're unsure.

Step 2: Match the Hash Algorithm

If the PSK is correct, the next suspect is the hash algorithm. Windows defaults to SHA1 for older systems, but modern peers might expect SHA256 or SHA384.

Check the rule's Properties again. Go to the Advanced tab and click Customize under IKE. Look at the Integrity check algorithm. Common mismatches:

• Windows Server 2012/2016 defaults to SHA1 — your peer might want SHA256.
• If your peer is a Cisco ASA or pfSense box, they often use SHA256 by default.
• Set both sides to the same algorithm. If in doubt, use SHA256 — it's widely supported and more secure.

Algorithm example: SHA256

Step 3: Delete and Recreate the Rule

Sometimes the rule itself gets corrupted — especially after a Windows update or a policy refresh. Don't waste time editing it. Delete the rule and build it fresh.

1. In the Windows Firewall MMC, select Connection Security Rules.
2. Right-click the rule and choose Delete.
3. Right-click Connection Security Rules and select New Rule...
4. Walk through the wizard: choose Isolation or Server-to-server depending on your setup. Enter the remote IP, choose authentication method (Pre-shared key), and set the same key on both ends.
5. Finish the wizard. Then click OK.
6. Test the connection again. The error should clear.

Alternative Fixes If Steps 1-3 Fail

I've had cases where none of the above worked. Here's what to try next:

• Check the IPsec policy on the group policy level. If you're in a domain, a GPO might be overriding your local settings. Run rsop.msc and look under Windows Settings -> Security Settings -> IPsec Policies. If a GPO is active, talk to your AD admin before touching it.
• Reboot both sides. I know it sounds basic, but after a policy change, the IPsec service sometimes holds old state. A reboot flushes it clean. I've seen this fix the error after a config change that looked correct on paper.
• Check the system event log. Open Event Viewer, go to Windows Logs -> Security, filter by Event ID 4654 (IPsec IKE failure). It often gives a specific reason like "hash algorithm mismatch" or "authentication method mismatch".
• Try disabling the rule temporarily. Sometimes a conflicting rule is the real issue. Disable all connection security rules except the one you need. Re-enable one at a time.

Prevention Tip

Document your IPsec policies — PSK, algorithms, and IP ranges — in a shared doc. When you update one side, update the doc and the other side immediately. I can't tell you how many times I've fixed this error by cross-referencing a stale config. Also, use a PSK that's at least 20 characters long with mixed case and numbers. Makes brute-forcing harder, but more importantly, it reduces the chance of accidental duplicates.

One last thing: if you're mixing Windows and non-Windows peers (like a Linux strongSwan box), make sure you're not defaulting to Kerberos or NTLM authentication — the peer won't have a domain trust. Always use pre-shared key or certificate-based auth in cross-platform setups.