Fix XACT_E_PROTOCOL (0x8004D105) MSDTC error

You're here because of error 0x8004D105

That error message—"The interface call made was incorrect for the current state of the protocol"—usually shows up when a distributed transaction fails. You see it in event logs, SQL Server Management Studio, or in custom apps that make COM+ calls. It's frustrating because the error text doesn't tell you what's wrong. Let's fix it.

The fix: reset MSDTC security configuration

Nine times out of ten, this error means MSDTC (Microsoft Distributed Transaction Coordinator) has a security setting that doesn't match what the calling application expects. You don't need to reinstall anything. You don't need to rebuild the cluster. You just need to reset the security configuration and restart the service.

Step 1: Open the Component Services console

1. Press Windows Key + R, type dcomcnfg, and hit Enter.
2. In the left pane, expand Component Services > Computers > My Computer.
3. Right-click My Computer and select Properties.

Step 2: Reset the MSDTC security settings

1. In the Properties window, go to the MSDTC tab.
2. Click the Security Configuration button.
3. In the Security Configuration window, look at the checkboxes under Transaction Manager Communication.
4. If this is a server that participates in distributed transactions (like a SQL Server or app server), check these boxes:
   • Network DTC Access
   • Allow Inbound
   • Allow Outbound
   • No Authentication Required (for local trust scenarios) OR Mutual Authentication Required (for domain-joined servers). I'll tell you which to pick in a second.
5. If you're on a domain and all machines are joined, choose Mutual Authentication Required. If you're in a workgroup or have cross-domain transactions, choose No Authentication Required—but only do that if you understand the security implications.
6. Click OK, then Apply, then OK to close everything.

Step 3: Restart the MSDTC service

1. Press Windows Key + R, type services.msc, and hit Enter.
2. Scroll down to Distributed Transaction Coordinator. Right-click it and select Stop.
3. Wait 10 seconds, then right-click again and select Start.
4. Close the Services window.

Step 4: Test the transaction

Try whatever operation was failing before—execute the SQL query, run the COM+ method, or trigger the distributed transaction. If the fix worked, the error should be gone. If it's still there, move to the registry cleanup below.

Why this works

The error code XACT_E_PROTOCOL (0x8004D105) means the caller made a request that doesn't match the current state of the MSDTC protocol. That's a fancy way of saying the security settings on the MSDTC service don't match what the application expects. When you reset the security config and restart the service, you force MSDTC to reinitialize its protocol state machine, dropping any stale sessions or cached security tokens. It's the same reason restarting a router fixes most network problems—you're clearing out bad state.

Less common variations of this problem

Cluster environments

If you're running a Windows Failover Cluster (like a SQL Server FCI), the MSDTC resource can get its security settings out of sync with the cluster registry. The fix is the same—open Component Services on the active node and reset security—but you also need to make sure the same settings apply to the cluster MSDTC resource. Use this PowerShell command to check the cluster resource security:

Get-ClusterResource "MSDTC" | Get-ClusterParameter

If the security parameters are missing or wrong, run the above GUI steps on the active node, then bring the resource offline and online again.

Firewall blocking RPC ports

Sometimes the error appears not because of a config mismatch, but because the firewall blocks the dynamic RPC ports that MSDTC uses (typically 49152–65535). Check your firewall logs. If you see dropped packets on those ports, add an exception. On the server hosting the transaction, run this command to see which ports MSDTC is listening on:

netstat -ano | findstr "3372"

Port 3372 is MSDTC's static endpoint. If that's not listening, the service isn't running correctly. But if you see other high ports in the 49xxx range being used, those are the dynamic ones.

Corrupted MSDTC registry key

I've seen cases where the MSDTC registry key under HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC gets corrupted—usually from an incomplete uninstall or a failed patch. If the GUI fix doesn't work, delete and recreate the key. Here's how:

1. Open Registry Editor as Administrator (regedit).
2. Go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSDTC.
3. Right-click the MSDTC key and select Export to back it up.
4. Right-click again and choose Delete. Confirm.
5. Close Registry Editor.
6. Open an Administrator Command Prompt (cmd).
7. Run msdtc -uninstall. Wait for it to finish.
8. Run msdtc -install.
9. Go back to Component Services, set the security config again, and restart the service.

This nukes the entire MSDTC configuration and rebuilds it from scratch. It sounds drastic, but it works when nothing else does.

Prevention

To keep this error from coming back, you need to standardize your MSDTC security settings across all machines that participate in distributed transactions. If you're using SQL Server linked servers, make sure every server has the same DTC security configuration. For domain environments, use Group Policy to enforce the settings:

• Computer Configuration > Administrative Templates > System > Distributed Transaction Coordinator > Enable Network DTC Access.
• Set it to Enabled with the correct inbound/outbound options.

Also, patch your servers regularly. A lot of these issues were more common on older builds of Windows Server 2016 and 2019. The 2022 version handles protocol state much better.

One more thing: if you're running third-party transaction monitors or custom COM+ components, make sure they're not manually changing the DTC security after your fix. I've seen apps that do that during installation—they overwrite the security config you just set. Check the application logs for any messages about "MSDTC security modified by..." and block those calls if possible.

Short version: reset MSDTC security, restart the service, test. If it fails, nuke the registry key and reinstall MSDTC. You almost never need to reinstall Windows.