Fix 'Permission denied (publickey)' When SSHing Into Ubuntu 22.04

What's actually happening here is...

SSH is saying: "I tried public key auth, and I don't have a key I trust for you." That's it. The server either didn't find your key, didn't accept it, or wasn't allowed to try it. Nine times out of ten it's file permissions on your local ~/.ssh or the server's ~/.ssh/authorized_keys. Let's go in order. Stop when it works.

Step 1: The 30-second fix — check permissions on both sides

OpenSSH is paranoid. If ~/.ssh is group- or world-writable, or if your private key is readable by anyone else, ssh silently ignores the key. No warning, no error until the auth failure.

On your local machine:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519 ~/.ssh/id_rsa
chmod 644 ~/.ssh/id_ed25519.pub ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/config

On the server (if you can still get in some other way):

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

The reason step 1 works so often: ssh refuses to use a private key that's world-readable. That's a security feature, not a bug. You can actually see it complain if you bump verbosity, which brings us to the diagnostic that's worth memorizing:

ssh -vvv user@host

Look for lines like Permissions 0644 for '~/.ssh/id_ed25519' are too open or Offering public key: ... followed by Server accepts key: ... no. That tells you exactly who's rejecting what.

Step 2: The 5-minute fix — is the right key even being offered?

You might be offering a key that the server doesn't know about. Common scenario: you have five keys in ~/.ssh, your ssh-agent is holding three of them, and none of them is the one in authorized_keys on the server.

Check what's loaded:

ssh-add -l

If the key you expect isn't listed:

ssh-add ~/.ssh/id_ed25519

Now confirm the server actually has the matching public key. If you have any other way in (password auth, a console, a second key), check on the server:

cat ~/.ssh/authorized_keys

Compare the fingerprint. On your local side:

ssh-keygen -lf ~/.ssh/id_ed25519.pub

On the server, for each line in authorized_keys:

ssh-keygen -lf ~/.ssh/authorized_keys

If the fingerprints don't match, your key was never installed. Reinstall it — and this is the part people get wrong — make sure you copy the .pub file, not the private one:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@host

If ssh-copy-id isn't available (some minimal distros), do it the manual way, and mind the newline at the end of the file:

cat ~/.ssh/id_ed25519.pub | ssh user@host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

That trailing newline thing isn't superstition. If the last line of authorized_keys has no newline, sshd sometimes fails to parse it. Add one if you hand-edited the file.

Step 3: The 15+ minute fix — server-side sshd config and SELinux

If permissions and keys are correct, the server is refusing to accept public key auth at all. Now you're reading config.

Check sshd_config

On the server:

sudo grep -Ei 'pubkeyauthentication|authorizedkeysfile|permitrootlogin|allowusers|denyusers' /etc/ssh/sshd_config /etc/ssh/sshd_config.d/*.conf

You want to see PubkeyAuthentication yes (or nothing, since yes is the default). Red flags:

• PubkeyAuthentication no — self-explanatory. Change it, reload sshd.
• AuthorizedKeysFile pointing somewhere weird — the default is .ssh/authorized_keys. If someone changed it to an absolute path with a typo, auth fails silently.
• AllowUsers or DenyUsers excluding your user — common on hardened boxes.

Reload after any change:

sudo systemctl reload ssh

Before reloading, run sudo sshd -t to verify the config parses. A bad config plus a reload on a remote box equals a lockout and a trip to the console.

SELinux / AppArmor on non-standard paths

If your authorized_keys lives somewhere unusual (say /home/team-keys/), SELinux on RHEL-family systems will block sshd from reading it. On Ubuntu 22.04 SELinux is usually off, but AppArmor profiles exist. Check:

sudo aa-status | grep sshd

On RHEL/CentOS/Rocky, the fix is to relabel:

sudo restorecon -Rv ~/.ssh

That restores the correct ssh_home_t context on .ssh. You can confirm with ls -Z ~/.ssh/authorized_keys. If it says something like user_home_t instead of ssh_home_t, that's your problem.

Check auth logs

Still stuck? The server's auth log tells you which stage failed.

sudo tail -f /var/log/auth.log

(On RHEL use /var/log/secure.) Lines worth reading:

• Authentication refused: bad ownership or modes for directory /home/user/.ssh — permissions, step 1.
• Failed publickey for user from ... ssh2: RSA SHA256:... — server got the key but didn't match. Wrong key, step 2.
• Connection closed by authenticating user ... [preauth] without further detail — often a config mismatch or shell issue, step 3.

Quick reference

One last thing. If you're seeing this against a cloud VM you just spun up, check that you're using the exact user the AMI expects — ubuntu on Ubuntu AMIs, ec2-user on Amazon Linux, admin on Debian. Logging in as root when PermitRootLogin prohibit-password is set will hand you this exact error, and no amount of chmod will help.