Fix 'Permission denied (publickey)' SSH in Linux

Yeah, this one's annoying — you set up your key, copied it to the server, and still get Permission denied (publickey). It happens on Ubuntu 22.04, Debian 12, CentOS 9, doesn't matter. The good news: it's almost always a simple fix. Let's get you in.

The fast fix

Run these on your local machine, assuming your private key is ~/.ssh/id_rsa:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_rsa
ssh-add ~/.ssh/id_rsa
ssh user@server

That's it for most cases. If it still fails, check the server side:

sudo grep -E "PubkeyAuthentication|AuthorizedKeysFile" /etc/ssh/sshd_config

Make sure PubkeyAuthentication yes is uncommented, and the AuthorizedKeysFile path points to .ssh/authorized_keys. Then restart sshd: sudo systemctl restart sshd.

Why the chmod matters

OpenSSH is paranoid about file permissions — and for good reason. If your private key is readable by group or others, sshd assumes it's compromised and silently refuses to use it. The error you see is misleading; it's not that the key is wrong, it's that sshd won't even try it.

The ~/.ssh directory should be 700 because otherwise other users on your machine could drop a malicious authorized_keys file there, and the key file itself needs 600 so no one else can read your private key. What's actually happening is sshd checks these permissions before attempting authentication. If they're loose, it acts like the key doesn't exist.

The ssh-add step is for when you're using an SSH agent. If you have multiple keys or your key has a passphrase, the agent needs to hold the key in memory. Without it, ssh tries to read the key directly, but if the agent is running and doesn't have the key, it can cause weird failures. Adding it explicitly removes that variable.

Less common variations

1. You're using the wrong username

Sounds dumb, but I've spent 20 minutes on this. On cloud servers (AWS EC2, DigitalOcean), the default user is often ubuntu, ec2-user, or admin — not root and not your local username. Check your provider's docs.

2. The authorized_keys file has wrong permissions on the server

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Same logic as the client side. If authorized_keys is world-writable, sshd ignores it.

3. SELinux context (CentOS/RHEL/Fedora)

If you're on a Red Hat-based system and permissions are fine, SELinux might be blocking it. Check with:

restorecon -Rv ~/.ssh

That resets the SELinux context for your SSH files. Happens when you copy files from elsewhere — the context gets lost.

4. You appended the public key to the wrong file

It needs to be in ~/.ssh/authorized_keys on the server, one key per line. Not in id_rsa.pub, not in a different user's directory. Double-check the path.

5. The server's sshd_config has AllowUsers or DenyUsers

These directives override everything. If your user isn't listed in AllowUsers, you're locked out regardless of your key. Check /etc/ssh/sshd_config for those lines.

6. The key type isn't supported

Older servers might reject newer key types like Ed25519. If you generated your key with ssh-keygen -t ed25519 and the server runs old OpenSSH (earlier than 7.0), generate an RSA key instead: ssh-keygen -t rsa -b 4096.

Prevention

Stop this happening again:

• Set correct permissions immediately after generating a key: chmod 700 ~/.ssh && chmod 600 ~/.ssh/id_*.
• Use ssh-copy-id user@server to append your public key — it handles permissions on the remote side automatically.
• If you manage multiple servers, create a ~/.ssh/config file. It saves you from typos and lets you specify the right key per host:

Host myserver
    HostName 192.168.1.100
    User ubuntu
    IdentityFile ~/.ssh/mykey
    IdentitiesOnly yes

The IdentitiesOnly yes line is important — it tells ssh to use only the key you specified, not all keys in your agent. Otherwise, if the server rejects one key, ssh tries the next, and you get a barrage of permission denied messages before it finally connects or not.

One more thing: if you ever see Permission denied (publickey,password), that means password auth is also off. You're not getting in via SSH at all — you'll need console access via your provider's web terminal. That's a different problem, usually from a misconfigured sshd_config. But that's for another day.

For most people, the chmod fix is all you need. Test it, and if you're still stuck, run ssh -vvv user@server and look for lines that say Offering public key and Authentications that can continue. That verbose output tells you exactly what the server is accepting and rejecting.