0x80090326 Fix: Unexpected or Badly Formatted Auth Message

Where This Error Pops Up

You're trying to connect to a remote server—maybe SQL Server, a file share, or Outlook connecting to Exchange—and boom, you get SEC_E_ILLEGAL_MESSAGE (0x80090326). The exact wording: "The message received was unexpected or badly formatted." I've seen this most often in two places:

A client laptop connected via VPN (especially Cisco AnyConnect or OpenVPN) trying to access internal resources.
A web application using Integrated Windows Authentication behind a reverse proxy (like nginx or Apache) that isn't handling the Kerberos token right.

It's not a random glitch—it's a cryptographic handshake failure. The client sends an authentication token, but the server can't make heads or tails of it.

Root Cause: Broken Token or Time Mismatch

The error means the security layer (Schannel or SSPI) got a message that's malformed. Three main causes:

Time skew. Kerberos is picky—client and server clocks must be within 5 minutes. I had a client last month whose domain controller had drifted by 8 minutes. Suddenly, half the office couldn't authenticate to the file server. 0x80090326 was the error.
SPN mismatch. The Service Principal Name on the server doesn't match what the client's requesting. This happens when you access a service by its IP address instead of its FQDN, or when a proxy rewrites the Host header.
Token corruption. Some VPNs or proxies chop or modify the Kerberos ticket. I've seen OpenVPN with MTU issues truncate packets, and the server sees garbage.

In plain English: the server expected a ticket it could decrypt, but got a scrambled egg instead. Or the ticket's timestamp said it was from the future (because time's off).

Fix It in 4 Steps

Step 1: Check Time Sync Across All Machines

This is the first thing I check. On the client machine, run this in an admin command prompt:

w32tm /query /status

Check the 'Source' and 'Last Successful Sync Time'. If it's not syncing to your domain controller or a reliable NTP source, fix it:

w32tm /resync

Then check the server's time too. They should be within seconds. If they're not, configure NTP on both ends. I use time.windows.com for standalone servers, but domain-joined machines should sync from the domain controller.

Step 2: Verify the SPN

If the time's fine, move to SPNs. Open a command prompt on the server and run:

setspn -L <computername>

Look for the SPN your client is trying to use. For SQL Server, it's usually MSSQLSvc/hostname.domain.com:1433. For HTTP, it's HTTP/hostname.domain.com. If it's missing, add it:

setspn -A HTTP/hostname.domain.com domain\computername

Also make sure the client is using the FQDN, not the IP or a short name. If you're behind a proxy, the proxy must pass the original host header untouched.

Step 3: Tweak VPN/Proxy Settings

If the error only happens through a VPN or proxy, the problem's likely packet fragmentation or token rewrite. For OpenVPN, try setting mssfix 1400 in the client config. For IIS behind a reverse proxy, check that the proxy isn't stripping the Authorization header. I've also seen this fixed by enabling ExtendedProtection in IIS—it forces the client to send a channel binding token, which some proxies mangle.

Step 4: Reset the Kerberos Ticket Cache

Sometimes the client just has a stale ticket. Run this on the client:

klist purge

Then try connecting again. This flushes the Kerberos cache and forces a fresh authentication. I've had cases where that alone solved it.

If It Still Fails

If none of that works, dig into the event logs. On the server, open Event Viewer and look under Windows Logs > Security for Event ID 4768 or 4769. Those will tell you if the ticket request was denied. Also check System logs for Schannel errors—Event ID 36888 or 36874 with the same 0x80090326 code.

Another thing: if you're using NTLM instead of Kerberos (maybe due to a workgroup setup or legacy app), the fix is different. NTLM doesn't care about time or SPNs, but it can still fail if the proxy rewrites the NTLM challenge response. In that case, disable Negotiate and force NTLM on the client with a registry key:

HKLM\SYSTEM\CurrentControlSet\Control\Lsa\msv1_0\RestrictSendingNTLMTicket

Set it to 0. But only do that as a last resort—NTLM's weaker.

Bottom line: 95% of the time, it's time sync or a bad SPN. Don't waste hours rebuilding the server. Start with the clock, then the SPN, then look at your network gear. You'll be back online in 10 minutes.