Fix LAG failure after switch reboot: LACP timeout & hash

You have a server with two NICs bonded in a Link Aggregation Group (LAG) — say, an Intel X710-DA2 on Ubuntu 22.04 with an active-backup or LACP mode. The switch is a Cisco Catalyst 3850 stack. Everything works fine until one of the two stack members reboots for a firmware update. After it comes back, your LAG doesn't recover. All traffic stops. The server still shows the bond as up, but pings time out. You check the switch and see one port is down. The other port is up. But no data flows.

What's actually happening here is a mismatch between what the server expects and what the switch offers after the reboot. The server's bond driver (like bonding or teamd) still thinks both links are valid because the LACP negotiation hasn't completed on the rebooted port. But the switch sees only one active member, and its hash algorithm now distributes traffic differently — sometimes sending frames to the dead port. Those frames get dropped.

The root cause is the LACP timeout timer. The server's bond waits for a LACP PDU from the switch. If the switch reboots, it takes a few seconds to start sending PDUs again. During that gap, the bond enters a fallback mode where it sends traffic on the first available port — but the switch's forwarding table still points to the old port mapping. The hash algorithm also changes because the active port count changed. So you end up with traffic loops or black holes.

How to fix it

Skip the gentle approach. The real fix is to force the bond to renegotiate cleanly. Do it in this order:

Clear the bond on the server — Stop the bond interface completely. On Ubuntu:
```
sudo ifdown bond0
sudo rmmod bonding
```
This wipes the kernel's internal state. The reason step 3 works is that the module reload forces a fresh LACP negotiation.
Reload the bonding module —
```
sudo modprobe bonding mode=4 miimon=100 lacp_rate=slow
```
Use lacp_rate=slow here. I know fast sounds better, but slow gives the switch time to stabilize after a reboot. I've seen fast rate cause flapping on Catalyst 3850s.
Bring the bond back up —
```
sudo ifup bond0
```
Now watch the switch. Clear the port-channel on the switch too:
```
clear port-channel 1 counters
```
That resets the hash counters.
Verify both ports are in the channel — On the switch:
```
show etherchannel 1 summary
```
You want both ports shown as P (in port-channel). If one shows D (down), check the physical cable or SFP.

If that still fails, the next culprit is the hash algorithm. On the switch, the default is src-dst-ip. But some NIC drivers use layer2+3 hash. They don't match. Change the switch's channel-group load-balance to src-dst-mac — that works with most bond drivers in LACP mode:

port-channel load-balance src-dst-mac

What to check if it still fails

Three things I've seen people miss:

LACP port priority mismatch — If the server's bond uses active-backup and the switch expects LACP, you get one port active and the other standby. The standby port never passes traffic. Check your bond config — mode=4 (802.3ad) is what you want, not mode=1 (active-backup).
STP blocking the port — After a reboot, Spanning Tree can keep a port in blocking state for 30+ seconds. On Cisco switches, enable PortFast on the switchports:
```
interface GigabitEthernet1/0/1
 spanning-tree portfast
```
MTU mismatch — If one side has jumbo frames and the other doesn't, the bond stays up but large packets get dropped. Ping with ping -M do -s 1472 to test.

Bottom line: LAG failures after a switch reboot are almost always a timing issue between LACP renegotiation and the hash algorithm still pointing to dead ports. Forcing the bond to reload and matching the hash algorithm fixes 9 out of 10 cases. If yours is the 10th, check STP and MTU.