SSH key rotation tutorial (2026): Replace VPS admin keys safely without downtime or lockouts

Most SSH incidents on hosting servers don’t start with an exotic kernel exploit. They start with a leaked private key. Common causes include an offboarded employee’s laptop, keys copied into too many repos, or a “temporary” shortcut that became permanent.

In 2026, rotating SSH keys on a VPS should feel like routine maintenance. Plan it, verify it, and keep it reversible. It shouldn’t be a last-ditch scramble.

This SSH key rotation tutorial shows how to replace admin access keys on Ubuntu/Debian and RHEL-family systems (AlmaLinux/Rocky). It also covers multi-admin setups. You’ll keep a known-good way in at every stage. You’ll also finish with clean notes you can reuse next time.

What you’ll rotate (and what you won’t)

This isn’t an SSH redesign. You’re not switching ports, adding a bastion, or rebuilding your auth model.

The scope is simple. Add new public keys, prove they work, then retire the old ones.

• Rotate: public keys in ~/.ssh/authorized_keys (and optionally /etc/ssh/authorized_keys if you centralize).
• Verify: logins using the new key from at least two networks (office/home + mobile hotspot is a good pair).
• Preserve: your existing SSH daemon settings until you confirm access.
• Optional hardening after: disable password auth, tighten allowed users, and add rate limiting.

If you want a safe baseline before you touch access controls, run through our Ubuntu checklist first: server hardening tutorial for a new Ubuntu VPS.

Prerequisites and safety rails

Don’t rotate keys blind on a production box. Set guardrails first. That way, you can recover fast if something goes sideways:

1. Console access: confirm you can reach your provider’s web/VNC console or rescue mode. On a VPS, that’s your break-glass path. If you’re running client sites, consider managed VPS hosting so you’re not alone if access breaks.
2. At least one existing working SSH session: keep it open until you’re completely finished. Use tmux if you want extra safety.
3. Know which account(s) matter: root, a sudo admin, or both. Rotate keys for every account that can escalate privileges.
4. Back up authorized_keys: make rollback a single command, not a detective story.

Step 1: Inventory who can log in (real quick, but accurate)

Start by listing accounts that can SSH into the server. On most VPS setups, that’s root plus one or more sudo users.

# list human users (UID >= 1000) on Debian/Ubuntu
awk -F: '$3 >= 1000 && $3 < 65534 {print $1}' /etc/passwd

# list sudoers (may include groups)
getent group sudo 2>/dev/null || true
getent group wheel 2>/dev/null || true

Next, check which of those accounts actually have keys installed:

for u in root $(awk -F: '$3 >= 1000 && $3 < 65534 {print $1}' /etc/passwd); do
  f=$(getent passwd "$u" | cut -d: -f6)/.ssh/authorized_keys
  if [ -f "$f" ]; then
    echo "--- $u: $f"
    nl -ba "$f" | sed -n '1,120p'
  fi
done

Scan for keys you don’t recognize. Check the comment at the end of each key line. Watch for former employee names and duplicates in the wrong places.

Write down what needs to go. Don’t delete anything yet.

Step 2: Generate a new key pair (prefer Ed25519)

Generate the new key on your admin workstation. Ed25519 is fast and compact. It’s also supported almost everywhere you’d run a modern VPS.

Use a passphrase unless you’re storing the key in hardware.

ssh-keygen -t ed25519 -a 64 -f ~/.ssh/hmc-vps-prod-2026 -C "admin@yourcompany (prod rotation 2026)"

• -a 64 increases KDF rounds for better protection if the key file is stolen.
• Use a naming pattern you can recognize later (env + year + purpose).

On teams, avoid shared keys. Give each admin their own key. That makes removals straightforward when roles change.

Step 3: Add the new key without breaking the old one

One rule prevents most lockouts: add first, verify second, remove last.

If the old key still works, ssh-copy-id is the quickest safe option:

ssh-copy-id -i ~/.ssh/hmc-vps-prod-2026.pub user@your-server-ip

If you’re using a non-standard port:

ssh-copy-id -i ~/.ssh/hmc-vps-prod-2026.pub -p 2222 user@your-server-ip

Whenever possible, add the key to a non-root sudo user first. After that path works, decide whether root SSH should exist at all.

Step 4: Verify the new key works (with a “no surprises” command)

Open a second terminal. Log in using only the new key.

The -i flag plus IdentitiesOnly=yes prevents your agent from “helping” with the wrong key.

ssh -i ~/.ssh/hmc-vps-prod-2026 -o IdentitiesOnly=yes user@your-server-ip

After login, confirm you can escalate privileges (if that’s how you administer the box):

sudo -v
sudo -l | sed -n '1,40p'

If you allow root SSH (still common on small VPS setups), test it explicitly:

ssh -i ~/.ssh/hmc-vps-prod-2026 -o IdentitiesOnly=yes root@your-server-ip

Leave sshd_config alone for now. Keep the change set tight. You’re rotating keys, not refactoring SSH.

Step 5: Lock the new key to reduce damage if it leaks

You can restrict a key directly in authorized_keys. This is a strong fit for automation keys (deploy, backups). It also helps for break-glass access you rarely use.

Example: limit a key to a fixed source IP and disable port forwarding:

from="203.0.113.10",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA... comment

Two gotchas to keep in mind:

• IP restrictions can lock you out if your admin IP changes. Use them where the source IP is stable (office/VPN) or for non-human keys.
• If you use SFTP-only accounts, add restrictions carefully. This pairs well with our SFTP setup guide tutorial.

Step 6: Remove old keys cleanly (without guessing)

Once the new key is proven, remove old keys by line. Use your inventory. Don’t rely on memory.

Start with a dated backup. Then edit using line numbers.

# on the server
umask 077
cp -a ~/.ssh/authorized_keys ~/.ssh/authorized_keys.bak.$(date +%F)

Now edit:

nl -ba ~/.ssh/authorized_keys
sudoedit ~/.ssh/authorized_keys

Delete only the lines you flagged earlier. Save, then test again from a fresh terminal using the new key.

If you’re rotating multiple accounts (root plus one or more admins), repeat the backup/edit/test cycle for each account.

Step 7: Optional but smart: tighten sshd after rotation

After rotation is stable, enforce sane defaults. These changes usually cut brute-force noise. They also reduce risk without disrupting normal admin work.

Edit /etc/ssh/sshd_config (or /etc/ssh/sshd_config.d/*.conf on newer distros). Consider:

• Disable password auth once keys are verified: PasswordAuthentication no
• Disallow root login if you have a sudo admin: PermitRootLogin no
• Limit users on shared admin servers: AllowUsers admin1 admin2
• Reduce auth attempts: MaxAuthTries 3

Validate the config before you reload anything:

sudo sshd -t
sudo systemctl reload ssh || sudo systemctl reload sshd

Also consider firewall rules that only allow SSH from known IPs. If you haven’t done that yet, follow: VPS firewall setup guide tutorial.

Step 8: Rotate keys across multiple servers (without missing one)

On a fleet, the classic failure mode is simple: you miss a node. Treat rotation like a rollout, not a one-off change.

A practical approach for a small fleet:

1. Maintain a text inventory of hosts (hostname, IP, OS, role).
2. Rotate on one low-risk server first (staging, internal tools).
3. Rotate production servers in a maintenance window, keeping a console path available.

You can distribute the key with a basic loop from your workstation (no orchestration required):

for host in 203.0.113.11 203.0.113.12 203.0.113.13; do
  ssh-copy-id -i ~/.ssh/hmc-vps-prod-2026.pub admin@$host
done

Then verify each host with a non-interactive check:

for host in 203.0.113.11 203.0.113.12 203.0.113.13; do
  ssh -i ~/.ssh/hmc-vps-prod-2026 -o IdentitiesOnly=yes -o BatchMode=yes admin@$host 'echo OK $(hostname)'
done

Step 9: Audit and document (so the next rotation is boring)

After the change, aim for two outcomes. First, you should be able to prove what happened. Second, you should be able to repeat it later without relearning the process.

• Record the key fingerprint in your admin notes:

ssh-keygen -lf ~/.ssh/hmc-vps-prod-2026.pub

• Keep a “last rotated” date per server (a private ops doc is enough).
• Log the change in your ticketing system if you have one.

If the same VPS also handles email, confirm DNS and rDNS still match what you expect. Deliverability problems often start with small drift.

Use: email deliverability setup guide tutorial.

Common problems and quick fixes

Problem: “Permission denied (publickey)” after adding the key.

• Check file permissions: ~/.ssh should be 700, authorized_keys should be 600.

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chown -R $USER:$USER ~/.ssh

Problem: The key works for one user but not another.

• You probably updated the wrong account’s authorized_keys.
• Confirm the target account home directory: getent passwd username.

Problem: You changed sshd settings and locked yourself out.

• Use your provider console/rescue mode, revert /etc/ssh/sshd_config, then systemctl restart sshd.
• Next time, keep one open SSH session while you test a second session.

Problem: You suspect a key leak and need emergency rotation.

• Rotate immediately, then review auth logs for suspicious IPs.
• On Ubuntu/Debian: /var/log/auth.log. On RHEL-family: /var/log/secure.

sudo grep -E "Accepted publickey|Failed publickey" -n /var/log/auth.log | tail -n 50

For deeper log triage on a hosting VPS, see: VPS log analysis tutorial.

Production checklist: SSH key rotation on a hosting VPS

• Console access confirmed (provider panel/rescue).
• Existing SSH session stays open until rotation completes.
• New Ed25519 key generated with passphrase and clear comment.
• New key added to sudo user first, verified from a fresh terminal.
• Old keys removed only after verification; backup saved with date.
• sshd -t run before any reload.
• Optional: password auth disabled and user allowlist applied.
• Fingerprint recorded; rotation date documented.

Summary: rotate keys like you rotate backups—regularly

SSH keys are credentials. Treat them that way.

If you rotate on a schedule, a leak becomes a cleanup task. It doesn’t need to become an outage.

If you’d rather not build and maintain all the safety rails yourself, run production sites on a HostMyCode VPS or hand access management and patching to managed VPS hosting. You still decide who gets access. You just don’t have to do every step alone.

FAQ

How often should I rotate SSH keys on a VPS in 2026?

For small teams, every 90–180 days is a practical default. Rotate immediately when someone leaves, a laptop is lost, or a key was shared insecurely.

Should I disable root SSH after rotating keys?

If you have a working sudo user and console access, yes—disabling root SSH reduces the target surface. Do it after you confirm the new key works for the sudo account.

What’s the safest way to avoid lockouts during key rotation?

Keep one SSH session open, add the new key first, verify login from a second terminal using -i and IdentitiesOnly=yes, then remove old keys.

Can I rotate keys without changing my SSH port or setting up a bastion?

Yes. Key rotation stands on its own. If your risk model requires fewer exposed services, combine rotation with IP-restricted firewall rules and stricter sshd settings.