IPsec DoS Protection Blocking Multicast (0xC0368001)

Quick answer for advanced users

Run netsh ipsec dynamic set config doSprotection enabled=no in an elevated cmd, then reboot. If that's too broad, create a filter exception for multicast traffic (224.0.0.0/4) in your IPsec policy.

What's actually happening here

I had a client last month whose Windows Server 2019 firewall was dropping multicast heartbeats from their SQL cluster. The event log showed 0xC0368001 and they were convinced it was a network switch issue. Took me two hours to trace it back to IPsec DoS protection. This error fires when IPsec sees a multicast packet—like from routing protocols (OSPF, VRRP) or WS-Discovery—and thinks it's a DoS attack. Windows' built-in IPsec DoS protection is designed to block unicast floods, but it sometimes chokes on legitimate multicast traffic. If you're running Windows Server with IPsec policies or a VPN (especially SSTP or IKEv2), you'll hit this.

Fix steps

1. Identify the source. Check your Security event log for event ID 5453 or 5455 tied to 0xC0368001. Look at the source IP—it'll likely be a multicast address like 224.0.0.5 (OSPF) or 239.255.255.250 (WS-Discovery).
2. Disable IPsec DoS protection temporarily. Open cmd as admin and run:

   netsh ipsec dynamic set config doSprotection enabled=no

   Reboot or restart the IPsec Policy Agent service. If the error stops, you've confirmed the cause.
3. Create a filter exception (preferred). Instead of disabling DoS protection globally, add a rule to allow multicast traffic. In the Windows Firewall with Advanced Security, create a new inbound rule:
   • Protocol: Any
   • Remote IP: Any (or specific multicast range 224.0.0.0/4)
   • Action: Allow the connection
   • Profile: Domain, Private, Public (as needed)
   • Name: "Allow IPsec multicast exceptions"
4. Re-enable DoS protection if you disabled it. Run:

   netsh ipsec dynamic set config doSprotection enabled=yes

   Then apply the filter from step 3.

Alternative fixes if the main one fails

If you're using a third-party VPN client: Some clients (like Cisco AnyConnect or Pulse Secure) override Windows IPsec settings. Disable their built-in firewall or configure it to allow multicast. I've seen this happen with Check Point Endpoint Security—the fix was to add a policy exception for multicast traffic in the client's admin console.
If it's a Hyper-V or failover cluster: Windows Failover Clustering uses multicast for heartbeats. Go to Failover Cluster Manager, select your cluster, then right-click the network and choose Properties. Under "Allow cluster network communication on this network," make sure multicast is enabled. Also check your antivirus—I've seen McAfee's firewall block these same packets.
If you can't change IPsec settings: Sometimes group policy locks them down. Check with your domain admin or run gpresult /h gp.html to see which policies apply. You might need an exception in the GPO for multicast traffic.

Prevention tips

Don't disable IPsec DoS protection permanently unless you're on an isolated network. Instead, always use filter exceptions. For new server builds, add a firewall rule for multicast traffic before you deploy IPsec policies. If you're running DHCP servers, multicast for DHCPv6 can also trigger this—add an allow rule for that too. And keep your event log size large enough to catch these errors early; I set mine to 50MB minimum. That way, you see the pattern before it becomes a crisis.