Fix systemd service restart loop in 2 minutes

You're staring at systemctl status and the service keeps flipping between active (running) and failed every few seconds. I've been there—it's like the machine is mocking you. Let's break the cycle.

Immediate fix: reset the unit and check journal

First, stop the loop manually so you can breathe:

1. Run sudo systemctl stop your-service-name. Replace your-service-name with the actual unit (like nginx or myapp).
2. Now reset the failed state: sudo systemctl reset-failed your-service-name. After that, you should see systemctl status your-service-name show inactive (dead)—no more looping.
3. Check the logs to see what's crashing: sudo journalctl -u your-service-name --since "5 minutes ago". Scroll to the bottom. Look for the last few lines before Main process exited. That's your smoking gun.

If the logs show exit code 1 or a segfault, the app itself is dying. If you see start-limit-hit, systemd's throttling kicked in after repeated failures.

Why it loops: the StartLimitInterval trap

By default, systemd gives a service 5 attempts within 10 seconds. If it keeps failing, systemd hits a StartLimitIntervalSec limit and stops trying. But here's the kicker: if your unit file has Restart=always or Restart=on-failure (which is common for daemons), systemd restarts immediately after each crash. That creates the loop you see.

The fix isn't changing Restart—it's fixing whatever's killing the process. But I'll show you how to adjust the limits too, in case the service needs more time to start.

Step-by-step permanent fix

1. Open the service unit file: sudo systemctl edit --full your-service-name. This creates an override in /etc/systemd/system/.
2. Add these lines inside the [Service] section (or adjust existing ones):

   [Service]
   Restart=on-failure
   RestartSec=5
   StartLimitIntervalSec=0

   RestartSec=5 adds a 5-second delay between restarts—stops the frantic loop. StartLimitIntervalSec=0 disables the restart limit entirely (no throttling).

3. Save the file, then run sudo systemctl daemon-reload. After that, start the service: sudo systemctl start your-service-name.
4. Wait 30 seconds and check systemctl status your-service-name. It should stay active (running) or fail cleanly once—no more cycling.

But don't stop here. If the service still crashes repeatedly, you're masking the real problem. Go back to the journalctl output and fix the application error.

Less common variations

These edge cases bite people regularly:

• Timeout loop: Service fails to start within TimeoutStartSec (default 90 seconds). Systemd kills it, restart kicks in, repeat. Fix by adding TimeoutStartSec=300 in the [Service] section if the app is slow to initialize.
• Dependency loop: Service requires another unit that also keeps failing. Check journalctl -xe for dependency failed messages. Fix the dependency first, then restart both.
• PID file corruption: Service writes a stale PID file, systemd thinks it's already running, fails to start, restarts. Add PIDFile=/var/run/your-service.pid and ExecStartPre=/bin/rm -f /var/run/your-service.pid to clean it each time.
• Resource exhaustion: Service leaks memory or file handles, hits system limits (LimitNOFILE, MemoryMax), crashes. Use systemctl show your-service-name | grep -i limit to see current caps. Add LimitNOFILE=65536 or MemoryMax=512M in the [Service] section.

Prevention: don't let it happen again

Three things to set early:

1. Always test with Restart=on-failure— avoid always unless you really want it to restart on normal exit. It causes loops when you're debugging.
2. Set a sane RestartSec— I use 2-5 seconds for most services. Prevents rapid-fire restarts from flooding the logs.
3. Monitor with systemd-tmpfiles or a cron job— check systemctl list-units --state=failed daily. A loop that happened once might come back after a reboot.

That's it. The loop is broken, the logs are checked, and you've got a config that won't repeat the madness. Deploy the same pattern across your fleet and save yourself the headache later.