30-Second Fix: Check the Health Check Path
Most of the time, this is a dumb config mistake. If you're using an Application Load Balancer (ALB) or Network Load Balancer (NLB), the health check path matters. You set it to /health but your app only serves /index.html. That's a fail.
Go to your Target Group settings in the AWS Console. Look at Health check path. Make damn sure it matches a real endpoint your app returns a 200 on. A common one is / or /health. But I've seen people type /healthz and their app doesn't have that route. Change it to / for a quick test.
Also check the Health check interval and Unhealthy threshold. Defaults are 30 seconds and 2 failures. If your app takes longer than 5 seconds to respond, bump the interval to 60 seconds. Don't set thresholds above 5 — it just delays the inevitable.
One real-world scenario: Your Auto Scaling Group is attached to an ALB. The health check hits /api/status. But your app's database is down, so it returns a 503. Fix the database or change the health check to a static page like /index.html.
5-Minute Fix: Firewall Rules and Security Groups
If the health check path is right but instances still fail, it's almost always a network block. The load balancer needs to reach your instances on the health check port. For an ALB, that's usually port 80 or 443. For an NLB, it's whatever you set.
Check the Security Group attached to your instances. It must allow inbound traffic from the load balancer's security group. The easiest way: find the load balancer's security group ID (looks like sg-12345). Then edit your instance security group to add an inbound rule:
Type: HTTP (or HTTPS)
Port: 80 (or 443)
Source: Custom — paste the load balancer security group ID
Don't open port 80 to 0.0.0.0/0 unless you're okay with the whole internet hitting your app. Use the security group reference — it's cleaner and more secure.
Also check the Network ACL on your subnets. Network ACLs are stateless, so you need both inbound and outbound rules. Inbound: allow ephemeral ports (1024-65535) from the load balancer subnet. Outbound: allow the health check port (80 or 443) to the load balancer subnet. Miss that outbound rule and traffic gets dropped.
Real example: I had a client whose Auto Scaling kept killing instances. Turned out the Network ACL was blocking outbound traffic on port 80. The load balancer could send the request but the response never came back. Added the outbound rule, fixed it in 2 minutes.
15+ Minute Fix: Application and OS-Level Issues
If both the path and the network are fine, the problem is inside the instance. The app itself is not responding correctly.
First, SSH into a failing instance. Check if your web server (Nginx, Apache, IIS) is running. On Ubuntu:
sudo systemctl status nginx
If it's dead, restart it and check logs:
sudo journalctl -u nginx --since "5 minutes ago"
If it's running but responding slowly, test the endpoint manually from inside the instance:
curl -I http://localhost/health
If you get a timeout or a 5xx, your app is broken. Common causes: memory leak, database connection pool exhausted, or a misconfigured reverse proxy. Check /var/log/nginx/error.log or /var/log/apache2/error.log.
For Windows instances running IIS, check the Event Viewer under Windows Logs > Application. Look for errors from the WAS (Windows Process Activation Service) or W3SVC. IIS has a habit of crashing silently if the app pool runs out of memory.
Another possibility: your instance's OS firewall (iptables, ufw, Windows Firewall) is blocking the health check. On Linux, run:
sudo iptables -L -n | grep :80
If you see DROP rules, that's your problem. Flush them temporarily for testing:
sudo iptables -P INPUT ACCEPT
sudo iptables -F
If that fixes it, add a permanent rule to allow the load balancer subnet.
One last thing: check your Auto Scaling Group's health check grace period. It defaults to 300 seconds (5 minutes). If your app takes longer than that to start, the health check fails before it's ready. Increase the grace period to 600 seconds (10 minutes) in the ASG settings. This is a common issue with Java apps that have a long startup time.
If none of this works, enable detailed monitoring in the Auto Scaling Group. That gives you CloudWatch metrics every 1 minute instead of 5 minutes. Look at StatusCheckFailed_System and StatusCheckFailed_Instance. If you see system check failures, it's an AWS hardware issue — terminate the instance and let Auto Scaling launch a new one on different hardware.
Pro tip: Always test your health check endpoint manually from a separate EC2 instance in the same VPC before blaming Auto Scaling. If it works from there, the network is fine. If not, the app or OS is the issue.