Kubernetes Node Stuck Not Scheduling Pods: Quick Fix

Your Node Won't Schedule Pods? Here's the Fix

Yeah, it's frustrating when your cluster goes quiet and pods just sit there in Pending. I've seen this more times than I'd like, especially with small business clusters running on bare metal or cloud VMs. Let's cut the crap and get it scheduled.

Step 1: Check If the Node Is Cordoned or Tainted

Run this first:

kubectl describe node <node-name>

Look for Taints and Conditions. If you see a taint like node.kubernetes.io/unschedulable:NoSchedule, it's cordoned. Uncordon it:

kubectl uncordon <node-name>

Had a client last month whose node stayed cordoned after a maintenance window. Took them two days to notice.

Step 2: Check Resource Pressure

If the node has MemoryPressure or DiskPressure in conditions, pods won't schedule. This happens when pods use too much memory or the disk is full. Check disk space:

kubectl exec -it <pod-name> -- df -h

Or SSH into the node and run df -h. If disk is above 90%, you need to clear logs or add storage. I usually start with journalctl --vacuum-size=200M on the node.

Step 3: Check Kubelet

Sometimes kubelet just stops reporting. Check its status:

systemctl status kubelet

If it's dead, restart it:

systemctl restart kubelet

Then wait a minute and check kubectl get nodes again. If it's still not ready, look at kubelet logs:

journalctl -u kubelet -n 50

You'll often see "failed to get node" or some permission error.

Why This Works

Taints are Kubernetes way of saying "this node is special, don't put random stuff here." Uncordon removes that. Resource pressure means the node is full — pods can't schedule because there's no room. Kubelet dying means the node is just dead to the cluster. Each fix addresses one of these common causes.

Less Common Variations

Variation 1: Node Has Too Many Pods

Kubernetees has a default limit of 110 pods per node. Check with:

kubectl get pods --all-namespaces --field-selector spec.nodeName=<node-name> | wc -l

If it's close to 110, you need to distribute pods or increase the limit (not recommended for beginners).

Variation 2: Node Labels Don't Match

If you use nodeSelector or nodeAffinity in your pod spec, make sure the node has the right labels. Check labels:

kubectl get node <node-name> -o yaml

Add a label if needed:

kubectl label nodes <node-name> your-label=value

Seen this with a client who renamed a node but forgot to update deployment labels.

Variation 3: Network Plugin Down

If the node can't reach the cluster network, pods won't schedule. Check that kube-proxy and calico or flannel pods are running on the node:

kubectl get pods -n kube-system -o wide | grep <node-name>

If they're missing, restart the network plugin.

Prevention

Set up regular checks. I use a cron job that runs kubectl get nodes every 5 minutes and alerts me if a node isn't Ready. For resource pressure, monitor disk and memory usage on each node. Use node-problem-detector if you want to get fancy, but honestly, a simple script that checks journalctl for kubelet errors works fine for small clusters.

Also, document taints you apply. Don't just taint a node and forget about it. I keep a text file with the reason and date for every taint or cordon action.

Pro tip: If you're running a home lab or small cluster, don't overthink it. Most of the time it's a full disk or a dead kubelet. Fix those first before diving into network plugins.

That's it. Your node should schedule pods now. If not, check the kubelet logs again — they usually tell you exactly what's wrong.