Quick answer
Check the IKE proposal settings (encryption, hash, DH group, lifetime) on both VPN peers — they must match exactly. Most often the DH group or encryption algorithm is off.
What’s actually happening here
I’ve seen this error pop up more times than I can count, usually during a site-to-site VPN setup between a Windows server and a third-party firewall like a SonicWall or a Cisco ASA. The error code 0X00003607 translates to "Error processing the proposal payload." That means during Phase 1 of IKE negotiation, one peer sends its list of supported encryption and authentication proposals, and the other peer says, "Nope, none of those work for me." The negotiation dies right there.
Last month I had a client whose branch office couldn’t connect back to the main office after their ISP swapped the router. The old router used a different default DH group than the new one. Took me an hour to track it down because the error message didn’t tell me which parameter was wrong — just that the proposal failed. So let’s cut the guesswork.
Step-by-step fix
- Identify both peers — Note the IP addresses and the VPN device or software on each end. Common combos: Windows RRAS to SonicWall, or Windows to pfSense.
- Check the IKE proposal on Peer A — On Windows, go to
Windows Firewall with Advanced Security>Connection Security Rules. Find the rule for this VPN. Double-click, go toRequirementstab, look atIPsec settings. Note the Key Exchange (IKE) settings: Encryption (AES128, AES256, 3DES), Integrity (SHA1, SHA256), Key Exchange Algorithm (DH Group 2, 14, 19, etc.), and Lifetime (minutes or seconds). - Check the IKE proposal on Peer B — On the firewall or other device, find the Phase 1 proposal. On a SonicWall:
VPN>Settings>IKE Identities. On pfSense:VPN>IPsec>Phase 1. Write down the same four values. - Match every parameter exactly — I’ve literally screenshot both configs side by side on a monitor. Encryption must be the same algorithm and key size. If one side says AES128 and the other says AES256, it will fail. Same for DH group — if one uses Group 2 (1024-bit) and the other uses Group 14 (2048-bit), you get this error. Lifetime should match too, though a small difference (like 480 vs 500 minutes) usually works — but don’t risk it. Set both to 480 minutes.
- Test the connection — After aligning the settings, save both sides and trigger a rekey. On Windows, you can restart the IPsec Policy Agent service (
net stop PolicyAgent && net start PolicyAgent) or just wait for the next attempt. On a firewall, initiate a ping across the tunnel. - Check logs — If it still fails, look at the logs. On Windows, open
Event Viewer>Windows Logs>System, filter by sourceIPsec. On a SonicWall, go toSystem>Logs. Look for entries like "No proposal chosen" or "IKE negotiation failed." That will tell you which parameter mismatched.
Alternative fixes if the main one doesn’t work
Sometimes the proposal match is perfect, but the error still shows. Try these:
- Reboot both ends — Sounds basic, but I’ve had a Windows server that cached a bad IKE policy across reboots. A full restart cleared it.
- Check NAT traversal — If one peer is behind a NAT device (like a home router), both sides need to agree on NAT-T (UDP 4500). If one side expects NAT-T and the other doesn’t, the proposal can get corrupted. Enable NAT-T on both sides.
- Disable and re-enable the VPN rule — On Windows, disable the Connection Security Rule, wait 10 seconds, enable it again. This forces a fresh negotiation.
- Update firmware or OS — An old SonicWall firmware (before 6.5.4.4) had a bug where it wouldn’t accept proposals with SHA256 even if configured. Updating fixed it.
Prevention tip
Before you even start configuring a VPN tunnel, write down the IKE proposal from each device and compare them in a table. I do this for every client now — saves hours. Also, standardize on modern settings: AES256, SHA256, DH Group 14 (or 19 for higher security), and a lifetime of 480 minutes. That combo works with almost every modern firewall and Windows 10/11/Server 2016+. Avoid 3DES and MD5 — they’re old and some devices don’t even support them anymore.
If you’re setting up a new VPN from scratch, test with a single proposal first (one encryption, one hash, one DH group). Don’t enable multiple options — that just masks the mismatch and makes debugging harder. Once it connects, you can add fallback proposals if you need them for backward compatibility.