SSH Jump Host Setup Tutorial: Secure Admin Access to VPS & Dedicated Servers Without Exposing SSH (2026)

Leave SSH open to the internet and your auth logs turn into background noise. You’ll see constant scans, credential stuffing, and endless probes. The better fix is architectural. In this SSH jump host setup tutorial, you’ll put a small bastion VPS in front of your servers. Then you’ll route every admin login through it, without making daily work painful.

This pattern works well for agencies, resellers, and anyone who manages multiple VPS or dedicated boxes. It also fits managed hosting teams where you want one consistent access method for staff and clients.

What you’ll build (and what it replaces)

You’ll deploy a jump host (bastion) as the only machine with public SSH. Your web nodes, cPanel servers, mail servers, and databases move to private SSH. They’re reachable only from the jump host (or your office VPN, if you use one).

• Before: Every server has port 22 open to the world.
• After: Only the jump host exposes SSH; everything else only allows SSH from the jump host IP.

Hosting fit: run the bastion on a small, stable VPS. Use it to protect a larger fleet of HostMyCode VPS and dedicated servers.

If you’d rather not maintain the security baseline yourself, managed VPS hosting aligns well with a jump-host-first access model.

Prerequisites and assumptions

• A jump host VPS running Ubuntu 24.04 LTS (recommended) with a static public IPv4.
• At least one target server (Ubuntu/Debian/AlmaLinux/Rocky) you want to remove from public SSH.
• Root access initially (you’ll lock this down).
• OpenSSH 9.x (default on modern distros; good ProxyJump support).

If any server still allows password logins, fix that first. Keys plus a jump host are the core of this setup.

Step 1 — Provision the jump host with the right size and network posture

Your jump host doesn’t need big CPU. It needs uptime, predictable networking, and enough disk for logs.

• Size: 1 vCPU / 1 GB RAM is usually enough for small teams; bump to 2 GB if you’ll run MFA tooling, log shipping, or heavier auditing.
• Disk: 20–40 GB is fine, but don’t squeeze it too hard. Log retention adds up.
• Networking: Pick a region close to your admins to keep interactive SSH snappy.

Write down the jump host public IP. That IP becomes the only allowed SSH source for your target servers.

Step 2 — Create an admin user and disable direct root SSH

On the jump host:

adduser admin
usermod -aG sudo admin

Set up SSH keys (from your workstation):

ssh-keygen -t ed25519 -a 64 -C "yourname@jump"
ssh-copy-id -i ~/.ssh/id_ed25519.pub admin@JUMP_HOST_IP

Harden /etc/ssh/sshd_config (Ubuntu paths shown):

sudoedit /etc/ssh/sshd_config

Apply these minimums (adjust to your policy):

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AllowUsers admin
X11Forwarding no
AllowTcpForwarding yes
ClientAliveInterval 300
ClientAliveCountMax 2

Reload SSH safely:

sudo sshd -t && sudo systemctl reload ssh

If you prefer a safer rollout, schedule a change window. Treat it like any other production update.

HostMyCode’s operations-first approach to controlled changes mirrors the discipline in our cPanel maintenance mode tutorial: test, apply, verify, and keep a rollback path.

Step 3 — Lock down the jump host firewall to “SSH only” (and only from your IPs)

Don’t run a bastion with SSH open to everyone. Restrict inbound SSH to your office IP, VPN egress, or a small allowlist.

Using UFW on Ubuntu:

sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH only from your office/static IP
sudo ufw allow from YOUR_OFFICE_IP/32 to any port 22 proto tcp

# Optional: allow from a VPN egress IP too
sudo ufw allow from YOUR_VPN_EGRESS_IP/32 to any port 22 proto tcp

sudo ufw enable
sudo ufw status verbose

If you’re still deciding on an allowlist approach, the “don’t lock yourself out” checks in our UFW firewall setup tutorial apply here almost verbatim.

Step 4 — Configure ProxyJump on your workstation (clean, repeatable SSH)

This is where the setup starts paying off. Define the jump host once in ~/.ssh/config. Then connect to targets by name.

mkdir -p ~/.ssh
chmod 700 ~/.ssh
nano ~/.ssh/config

Example config:

Host jump
  HostName JUMP_HOST_IP
  User admin
  IdentityFile ~/.ssh/id_ed25519
  IdentitiesOnly yes
  ServerAliveInterval 60
  ServerAliveCountMax 3

Host web-01
  HostName 10.10.0.11
  User admin
  IdentityFile ~/.ssh/id_ed25519
  ProxyJump jump

Host cpanel-01
  HostName 10.10.0.21
  User root
  IdentityFile ~/.ssh/id_ed25519
  ProxyJump jump

Now you can do:

ssh web-01
ssh cpanel-01

Two practical notes:

• If your targets don’t have private IPs, you can still use ProxyJump with public IPs. You’ll get better auditability and a single chokepoint, but you won’t eliminate internet-facing SSH unless you also firewall targets to the jump host IP.
• For cPanel/WHM servers, consider a non-root admin with sudo. Keep root for true break-glass access.

Step 5 — Change each target server to accept SSH only from the jump host

This is the step that removes most of the risk. After this change, scanners can’t even reach SSH on your fleet.

On each target server, apply firewall rules that allow port 22 only from the jump host IP. Examples:

Option A: UFW (Ubuntu/Debian)

sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH only from the jump host
sudo ufw allow from JUMP_PUBLIC_IP/32 to any port 22 proto tcp

# Keep your service ports as needed (examples)
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

sudo ufw enable
sudo ufw status verbose

Option B: firewalld (AlmaLinux/Rocky)

sudo firewall-cmd --permanent --remove-service=ssh
sudo firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="JUMP_PUBLIC_IP/32" service name="ssh" accept'
sudo firewall-cmd --reload
sudo firewall-cmd --list-all

Safety rule: before you cut off public SSH, open a second session through the jump host. Confirm it stays connected.

Step 6 — Add MFA to the jump host (TOTP) without breaking automation

MFA belongs on the jump host because it’s your single ingress point. In 2026, TOTP via PAM is still one of the simplest options for SSH. It adds friction for attackers without adding a pile of moving parts.

On Ubuntu jump host:

sudo apt update
sudo apt install -y libpam-google-authenticator

As the admin user:

google-authenticator

Answer prompts to disallow reuse and allow time skew. Store the emergency scratch codes offline.

Edit /etc/pam.d/sshd and add:

auth required pam_google_authenticator.so nullok

Edit /etc/ssh/sshd_config and set:

KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

Reload SSH and test in a new terminal:

sudo sshd -t && sudo systemctl reload ssh
ssh jump

Operational caveat: MFA can break non-interactive sessions.

If you run automation from a CI runner, give it a separate user that’s key-only. Constrain it with ForceCommand and from= options in authorized_keys. Keep humans on MFA; keep automation tightly scoped.

Step 7 — Enforce per-user auditability (no shared keys, no shared accounts)

Jump hosts rarely fail because of OpenSSH. They fail when teams treat them as shared accounts with shared keys.

Create one user per admin and one key per device. On the jump host:

sudo adduser alice
sudo usermod -aG sudo alice

Then copy keys to that user and set AllowUsers accordingly.

On targets, aim for the same pattern. At minimum, make sure you can answer who accessed a system, from where, and when.

Step 8 — Add guardrails: SSH agent forwarding policy and safe defaults

Agent forwarding is handy, but it increases the blast radius if the jump host is compromised. A safer default is no agent forwarding and separate keys for the jump host and your targets.

In ~/.ssh/config you can explicitly disable forwarding:

Host jump
  ForwardAgent no

If you must use agent forwarding (short-term migrations, vendor access), keep it narrow. Limit it to specific hosts. Put an end date on it, and document the exception.

Step 9 — Make it easy: one-command access to panels and internal ports

Once you have a bastion, you can stop exposing admin panels publicly. Tunnel them instead.

Example: forward WHM locally (from your workstation):

ssh -J jump -L 2087:127.0.0.1:2087 root@10.10.0.21

Then open https://localhost:2087 in your browser. If you want more patterns (databases, webmail, internal apps), our SSH port forwarding tutorial goes deeper and fits neatly with a bastion-first setup.

Step 10 — Monitoring: detect brute force, new keys, and risky login patterns

Your jump host becomes your highest-signal security log source. Treat it like production security infrastructure, not “just another VPS.”

Quick diagnostics:

• Recent successful logins: last -a | head
• SSH auth failures (Ubuntu): sudo journalctl -u ssh --since "24 hours ago" | grep -i fail | tail -n 50
• Check currently connected sessions: w and ss -tnp | grep :22

Add Fail2Ban if you allow SSH from more than a tiny IP set. Then alert on unusual bursts of failures or new login sources.

Our log monitoring setup guide lays out a practical baseline without drowning you in noise.

Step 11 — Backup and recovery plan for the jump host

Jump hosts are small, which makes them easy to rebuild. That only works if you’ve captured the right configuration. Back up:

• /etc/ssh/sshd_config and any drop-in configs
• /etc/ufw/ (or firewalld rules)
• User list and group memberships
• Logs (at least a short retention window off-host)

Don’t back up private keys in a way that makes them easy to steal. Keys should live with users, ideally in hardware-backed storage.

If you’re standardizing retention and restore tests across a fleet, the structure in our server backup retention policy tutorial helps you avoid “we have backups” complacency.

Common pitfalls (and how to avoid them)

• Locking yourself out: keep an existing root session open while you test new SSH/firewall rules.
• Shared accounts: per-user access is non-negotiable if you manage client infrastructure.
• Agent forwarding everywhere: convenient, but risky. Use separate keys for jump vs targets.
• No IP allowlist on the jump host: MFA helps, but you’ll still invite noise and wasted time.
• Ignoring DNS/hostnames: stable naming in SSH config prevents mistakes during incidents.

Checklist: your “done” definition

• [ ] Only the jump host has public SSH.
• [ ] Target servers allow SSH only from the jump host IP.
• [ ] PasswordAuthentication is disabled everywhere.
• [ ] MFA is enabled for human jump-host users.
• [ ] Each admin has an individual account and unique keys.
• [ ] Panel access happens via tunnels, not public ports.
• [ ] Logs are monitored and retained long enough to investigate incidents.

Summary: a jump host is the simplest “big security win” you can deploy

If you manage more than one server, the bastion pattern pays off quickly. You get fewer exposed services, less internet noise, cleaner auditing, and safer access to admin panels.

The setup is straightforward. Once ~/.ssh/config is dialed in, operations usually get simpler.

If you want a stable place to run your bastion and keep the rest of your infrastructure private, start with a HostMyCode VPS. If you’d rather hand off patching, baseline hardening, and day-2 maintenance, managed VPS hosting is a practical option.

FAQ

Do I still need a firewall on each target server if I have a jump host?

Yes. The jump host reduces exposure, but the target firewall is what actually blocks direct SSH from the internet. Treat it as a requirement, not an extra.

Should the jump host and targets share the same SSH key?

Prefer separate keys. Use one key for the jump host and another for targets. That limits blast radius and makes rotation less painful.

Can I use this with cPanel/WHM servers?

Yes. Restrict SSH to the jump host, then tunnel WHM (2087) and cPanel (2083) as needed. It’s a clean way to avoid exposing panels publicly.

What if my ISP IP changes and I can’t reach the jump host?

Use a VPN with a stable egress IP, or maintain a small emergency allowlist process. Without one of those, you’ll eventually lock yourself out at the worst time.

Is MFA on SSH enough by itself?

It’s a strong control, but it doesn’t replace key-only auth, least-privilege users, and tight source IP rules. Stack the layers instead of betting on one.