Fix Linux System Clock Drift (Ubuntu 22.04 & RHEL 9)

You've seen this: the system clock is drifting by minutes per day

It usually shows up in dmesg or syslog as "system clock drift detected". Or you notice it when SSH authentication fails because the time is off by 30 seconds. I've had it happen on a fresh Ubuntu 22.04 VM that was left running for 3 days — the clock was 5 minutes ahead. On RHEL 9, a database server started rejecting logins because the NTP offset hit 4 minutes. The trigger is almost always a misconfigured time sync service or a hardware clock that's dying.

Root cause: your time sync service isn't talking to a good NTP pool

The Linux kernel keeps time using the system clock (software) and the hardware clock (RTC). When they drift apart, the kernel starts complaining. The real fix is making sure you're using a modern time daemon like chrony (not old ntpd) and pointing it to a reliable NTP pool. On Ubuntu 22.04, systemd-timesyncd is the default, but it's too simple for production servers — it can't handle large drifts. On RHEL 9, chronyd is the default, but it often ships with generic pool settings that don't work well behind a firewall or in a VM.

Step-by-step fix: stop the drift with chrony

Step 1: Check current time status

timedatectl status

Look for "System clock synchronized: yes" and "NTP service: active". If it says "no" or "inactive", you've found your problem. Also run:

chronyc tracking

If you get "Cannot talk to daemon" or "Not synchronised", chrony is either not installed or not running.

Step 2: Install chrony (if missing)

On Ubuntu 22.04:

sudo apt update && sudo apt install chrony -y
sudo systemctl enable --now chrony

On RHEL 9:

sudo dnf install chrony -y
sudo systemctl enable --now chronyd

Step 3: Configure chrony for your region

Edit /etc/chrony/chrony.conf (Ubuntu) or /etc/chrony.conf (RHEL). Replace the default pool lines with region-specific pools. For example, use pool.ntp.org but with a country code to reduce latency:

pool us.pool.ntp.org iburst
pool 2.us.pool.ntp.org iburst
pool 3.us.pool.ntp.org iburst
# fallback if others fail
pool pool.ntp.org iburst

Add this line to handle large drifts (common on VMs):

makestep 1.0 3

This tells chrony to step the clock if the drift is more than 1 second, up to 3 times at startup. Without it, chrony will slowly slew the clock, which can take hours.

Step 4: Restart chrony and verify

sudo systemctl restart chrony
chronyc sources -v

You should see "^*" (synced) next to at least one NTP source. Then check:

chronyc tracking | grep -E "Stratum|Offset|Last offset"

A good offset is under 10 milliseconds. If it's in the seconds, give it a minute and check again.

Step 5: Disable systemd-timesyncd (Ubuntu only)

sudo systemctl stop systemd-timesyncd
sudo systemctl disable systemd-timesyncd

I've seen them conflict — both try to sync and the clock never settles. Chrony is more robust, so let it win.

If it's still drifting after the fix

Check the hardware clock: Run sudo hwclock --show. If it's way off from system time, your RTC battery might be dead. Replace the battery on physical servers. On VMs, the hypervisor's time sync can override Linux. Disable VMware's time sync with toolsTime.synchronize = FALSE in the VMX file if you're using chrony.
Firewall blocking NTP: NTP uses UDP port 123. Run sudo firewall-cmd --list-ports (RHEL) or sudo ufw status (Ubuntu). If blocked, add an allow rule.
VM snapshots: After restoring a snapshot, the clock can jump. Run sudo chronyc -a 'burst 4/4' to force a fast resync.
Kernel parameter: If drift is massive (minutes per hour), add tsc=reliable to your kernel boot parameters in /etc/default/grub and run sudo update-grub. This helps on older CPUs with unstable TSC.

Skip the fancy ntpdate one-shot commands — they don't fix the root cause. You need a daemon that constantly adjusts. Chrony is the way to go for Linux servers in 2024.