Fix ERROR_IPSEC_IKE_DOS_COOKIE_SENT (0x00003642) Fast

You're seeing ERROR_IPSEC_IKE_DOS_COOKIE_SENT (0x00003642) in your event logs or VPN client. Translation: Windows sent a DOS cookie notification to the initiator during IKE negotiation. This isn't a real DOS attack — it's a side effect of aggressive IKE settings, firewall misconfigurations, or IPsec policy mismatches. I've fixed this on dozens of Windows Server 2016/2019 boxes and a few Windows 10 VPN clients. The culprit here is almost always the initiator sending too many IKE requests too fast, or the responder's firewall dropping the initial exchange.

30-Second Fix: Check Windows Firewall Rules

Nine times out of ten, this error pops up because the Windows Firewall is blocking IKE traffic. IPsec uses UDP ports 500 and 4500. If those ports are restricted, the responder sees the initiator as unresponsive and sends a DOS cookie to slow things down.

1. Open Windows Defender Firewall with Advanced Security.
2. Check Inbound Rules for any rule that blocks UDP 500 or 4500.
3. Temporarily disable the firewall on both sides (initiator and responder) to isolate the issue.

If the error goes away, create an allow rule:

netsh advfirewall firewall add rule name="IPsec IKE" protocol=udp localport=500,4500 action=allow dir=in

Don't bother with ICMP or other ports — this error is strictly about IKE negotiation.

5-Minute Fix: Tweak IKE Retry and Lifetime Settings

If the firewall isn't the problem, the responder's IKE settings are too strict. The DOS cookie gets sent when the responder receives multiple IKE requests in quick succession — often from a VPN client reconnecting after a brief network glitch.

On the responder (typically a VPN server or domain controller), run this from an elevated command prompt:

netsh ipsec dynamic set config ikedospcookielifetime 300
netsh ipsec dynamic set config ikedospcookiethreshold 10

This bumps the cookie lifetime to 5 minutes and allows up to 10 requests before triggering the cookie. Defaults are often too low (60 seconds and 5 requests). After changing, restart the IPsec service:

net stop ipsec && net start ipsec

Also check the IKE negotiation timeout on the initiator. If it's too short (like 30 seconds), the initiator retries faster than the responder expects. Set it to 120 seconds:

netsh ipsec dynamic set config ikenegotiationtimeout 120

15-Minute Fix: Align IPsec Policies and Certificates

When the quick wins don't work, it's a policy mismatch. The DOS cookie error can mask a deeper authentication failure — the responder rejects the initiator's IKE main mode proposal, sends the cookie, and the initiator misinterprets it.

Steps:

1. Compare IPsec policies on both machines. Use netsh ipsec static show policy all and match these parameters:

Fix: Set both sides to use the same proposals. I usually go with AES256, SHA256, DH Group 14, 480 minute lifetime for main mode.

2. Check certificate trust — if using certificates for authentication, ensure the root CA is trusted on both sides. Run certlm.msc on both machines and verify the issuing CA is in Trusted Root Certification Authorities.
3. Review event logs — Look at System and Security logs for Event ID 4654 (IPsec main mode failure) or 5453 (IKE quick mode failure). The description usually tells you exactly which proposal failed.

One real-world scenario: A Windows Server 2019 using built-in IPsec to connect to an Azure VPN gateway. The gateway expected IKEv2, but the server was configured for IKEv1. The error was 0x00003642 every 3 minutes. Switching to IKEv2 fixed it.

Pro tip: Don't waste time enabling verbose IPsec logging unless you're at this stage. It generates hundreds of events per minute. Use it for 30 seconds, then turn it off.

Still stuck? Verify there's no third-party firewall (like Symantec or McAfee) injecting itself into the IPsec stack. Those DPM-based filters often interfere with IKE negotiation. Uninstall, test, then reinstall with IPsec exclusions enabled.