Fix SSH Permission Denied for Private Key (Linux)

The 30-Second Fix: File Permissions on Your Private Key

I've seen this exact error more times than I can count. You copy your SSH key, try to connect, and get Permission denied (publickey). The culprit here is almost always file permissions that are too loose. SSH is paranoid about this — it won't touch a private key that's readable by anyone else.

Run this immediately:

chmod 600 ~/.ssh/id_rsa
# Or whatever your private key is called — id_ed25519, id_ecdsa, etc.

Then try connecting again:

ssh user@hostname

If that worked, you're done. 8 times out of 10, this is all you need. The 600 means owner can read and write, and nobody else gets anything. SSH wants exactly that.

Don't bother with 777 or 644 — those will fail every time. I've had junior devs argue with me about this. Don't be that person.

The 5-Minute Fix: Check .ssh Directory and authorized_keys

Still broken? The issue might be in the .ssh directory itself or the authorized_keys file on the server. Here's what to check:

On your local machine

The .ssh directory must be 700 (owner only). Public keys (.pub files) can be 644 — they're meant to be shared. But the private key stays at 600.

chmod 700 ~/.ssh
chmod 644 ~/.ssh/id_rsa.pub
chmod 600 ~/.ssh/id_rsa

On the remote server

SSH also checks the server-side permissions. If you can log in another way (password or another key), fix these:

# As the user you're trying to connect as
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 644 ~/.ssh/authorized_keys2  # Rare, but some setups use it

Also check the home directory permissions. If /home/username is writable by group or others, SSH will refuse keys. Run:

ls -ld ~/
# Should show drwx------ or drwxr-x--- at most
chmod 755 ~/   # Or 750 if you're strict

A common scenario I see: someone copies their .ssh folder from a backup with wrong permissions. This fix catches that.

The 15+ Minute Fix: sshd_config and Debug Logging

If permissions look right and it still fails, the problem is likely the SSH server config. You'll need root or sudo access on the remote machine.

Step 1: Check sshd_config

Open /etc/ssh/sshd_config and look for these lines:

PubkeyAuthentication yes
AuthorizedKeysFile .ssh/authorized_keys
PasswordAuthentication no  # If it's set to no, password login won't work
ChallengeResponseAuthentication no

If PubkeyAuthentication is set to no, that's your problem. Change it to yes. Also make sure AuthorizedKeysFile points to the right place. Default is .ssh/authorized_keys, which is relative to the user's home directory. Don't change it unless you have a reason.

If PasswordAuthentication is no, you won't get a password prompt — which might make you think the key is failing when really SSH is refusing all auth. Set it to yes temporarily for testing, then change it back.

Step 2: Restart SSH and test

sudo systemctl restart sshd
# On older systems: sudo service ssh restart

Step 3: Enable debug logging on the client

This is where you actually see what SSH is thinking. Run your connection with verbose output:

ssh -vvv user@hostname

Look for lines like these in the output:

debug1: Authentications that can continue: publickey — good, publickey auth is allowed
debug1: Offering public key: /home/user/.ssh/id_rsa — it's trying your key
debug2: we did not send a packet, disable method — key was rejected
debug1: Received disconnect from ... : 11: Bye Bye — server closed the door

The most helpful line is often something like:

debug1: Authentications that can continue: publickey,password

If you see publickey but it still fails, the debug output will tell you why — usually "bad permissions" or "key not found in authorized_keys".

Step 4: Check SELinux or AppArmor (if enabled)

On RHEL, CentOS, or Fedora, SELinux can block SSH even with correct permissions. Check with:

getenforce
# If it says Enforcing, try:
sudo setenforce 0
# Then test SSH. If it works, SELinux is the problem.

To fix permanently, adjust the context on .ssh:

restorecon -R -v /home/username/.ssh

On Ubuntu with AppArmor, it's less common but check /var/log/syslog for denials:

grep ssh /var/log/syslog | grep DENIED

Step 5: Check the key itself

Rare, but possible: the public key on the server doesn't match the private key on your client. Double-check:

# On the client:
ssh-keygen -lf ~/.ssh/id_rsa.pub

# On the server:
ssh-keygen -lf ~/.ssh/authorized_keys
# Or if you have multiple keys, the one you're using

The fingerprints should match. If they don't, you copied the wrong public key to the server.

One last thing: if you're on a Raspberry Pi or other ARM device, the SSH daemon might be compiled differently. I've seen cases where the default AuthorizedKeysFile is /etc/ssh/authorized_keys instead of the user's .ssh directory. Check sshd_config carefully on those.

That's the full flow. Start with chmod 600, then check directories, then dig into configs. 99% of the time you won't get past step one.