One VPS can act as a solid “front door,” even when your app needs multiple backends. This Nginx load balancing tutorial shows how to put Nginx in front of two application servers and route traffic safely.
You’ll set up health-aware behavior, sticky sessions only when needed, correct client IP forwarding, and reloads that don’t cut off active connections.
The goal here is day-to-day operations, not theory. By the end, you’ll have a repeatable config you can run on a VPS or dedicated server.
You’ll also have a short checklist for the failures you’ll actually see.
If you’d rather not manage the edge box yourself, HostMyCode can handle the OS and web stack on managed VPS hosting, while you keep your app servers focused on application work.
Lab topology and prerequisites
We’ll use a simple three-node layout. Run it on three VPS instances, or one VPS plus two internal servers.
- Edge / LB: Ubuntu 24.04 LTS, public IP, Nginx 1.24+ (from Ubuntu repo), ports 80/443 open
- App1: private IP (example: 10.0.1.11), serves HTTP on :3000
- App2: private IP (example: 10.0.1.12), serves HTTP on :3000
You’ll need:
- SSH access to all servers
- A domain pointing to the edge server (A/AAAA record)
- TLS cert via Let’s Encrypt (we’ll use Certbot)
- Basic firewall rules (UFW or iptables) allowing inbound 80/443 to the edge, and allowing the edge to reach app servers
If you’re building the edge on a fresh instance, start with a clean, boring base install. A HostMyCode VPS fits this pattern well.
You can scale CPU/RAM on the edge as traffic grows. App servers can scale on their own schedule.
Step 1 — Prepare the two backend app servers (a quick, testable baseline)
Nginx can’t balance traffic to servers it can’t reach. Before you touch the load balancer, confirm each backend responds quickly and predictably.
1) Verify the app port listens on the right interface
On App1 and App2:
sudo ss -lntp | grep ':3000'You should see the service bound to the private IP (best) or 0.0.0.0. If it’s bound to 127.0.0.1, the edge won’t be able to connect.
2) Add a minimal health endpoint
Expose a fast endpoint that doesn’t touch the database. Common names are /healthz or /readyz.
Keep the contract simple:
- HTTP 200 when the app can accept traffic
- HTTP 500 (or 503) when it should be removed from rotation
If you don’t have one yet, fake it temporarily while you test networking and proxying. A tiny Nginx instance or minimal HTTP server is fine.
What matters is consistency. The edge should get the same reliable response every time.
3) Restrict backend access to the edge only
Don’t leave app ports exposed to the internet. On each app server (UFW example), allow :3000 only from the edge’s private IP (example: 10.0.1.10):
sudo ufw allow from 10.0.1.10 to any port 3000 proto tcp
sudo ufw deny 3000/tcp
sudo ufw statusIf you want a stronger baseline for Ubuntu servers, this matches the approach in our UFW firewall setup tutorial (2026).
Step 2 — Install Nginx on the edge server
On the edge/LB (Ubuntu 24.04 LTS):
sudo apt update
sudo apt install -y nginx
nginx -v
sudo systemctl enable --now nginxConfirm the default page loads on port 80. If you use UFW on the edge:
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw statusStep 3 — Build an upstream pool (round-robin + sane timeouts)
Create a dedicated Nginx site config. We’ll use /etc/nginx/sites-available/example.conf and an upstream called app_pool.
sudo nano /etc/nginx/sites-available/example.confPaste this baseline (adjust domain and IPs):
upstream app_pool {
# Default algorithm is round-robin
server 10.0.1.11:3000 max_fails=3 fail_timeout=10s;
server 10.0.1.12:3000 max_fails=3 fail_timeout=10s;
# Keepalive reduces TCP churn to backends under load
keepalive 64;
}
server {
listen 80;
server_name example.com www.example.com;
# Temporary: Let’s Encrypt HTTP challenge needs port 80
location /.well-known/acme-challenge/ {
root /var/www/html;
}
location / {
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 3s;
proxy_send_timeout 30s;
proxy_read_timeout 30s;
proxy_next_upstream error timeout http_502 http_503 http_504;
proxy_next_upstream_tries 2;
proxy_pass http://app_pool;
}
}Enable the site and test config:
sudo ln -s /etc/nginx/sites-available/example.conf /etc/nginx/sites-enabled/example.conf
sudo nginx -t
sudo systemctl reload nginxNow hit the domain a few times. If your backend can show a host-specific marker (a header or a small HTML clue), you should see requests alternate between App1 and App2.
Step 4 — Add active health checks (practical options in 2026)
Open-source Nginx ships with passive health checks. A backend gets marked down only after real requests fail.
For many VPS setups, this is enough. Keep timeouts tight, and return 503 quickly when the app isn’t ready.
If you want active probing (Nginx periodically calling /healthz), you generally have three workable choices in 2026:
- Passive checks only (simplest): use
max_fails,fail_timeout, andproxy_next_upstream. - Nginx Plus (commercial): built-in health checks and richer status.
- External checker: systemd timer/curl + dynamic upstream updates (works, but adds moving parts).
For most small teams, passive checks plus a proper readiness endpoint is the cleanest trade-off. You get failover behavior without adding a second control plane.
Improve failure behavior with a dedicated location
Add this block inside your server {} (before location /):
location = /healthz {
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 1s;
proxy_send_timeout 2s;
proxy_read_timeout 2s;
proxy_pass http://app_pool;
}That keeps health traffic cheap. It also prevents slow hangs from turning into long, confusing incidents.
Step 5 — Sticky sessions (only if your app really needs them)
If your app keeps session state in memory, round-robin will eventually bite you. Users get logged out, and carts “disappear.”
The durable fix is shared session storage (Redis, database) or stateless tokens. Still, for legacy apps or short migration windows, you may need stickiness immediately.
Open-source Nginx doesn’t ship with a full sticky-cookie module. The most common workaround is IP hash. It’s not perfect, but it’s simple and predictable:
upstream app_pool {
ip_hash;
server 10.0.1.11:3000 max_fails=3 fail_timeout=10s;
server 10.0.1.12:3000 max_fails=3 fail_timeout=10s;
keepalive 64;
}Know what you’re trading off:
- NAT / mobile networks can place many users behind one IP, skewing load.
- If a backend fails, clients still “jump” to the remaining server.
- This is a stopgap, not a replacement for shared session state.
If you can fix the root cause, do that. If you can’t this week, IP hash beats random session breakage.
Step 6 — Preserve the real client IP (logs and app behavior)
If you don’t forward client IPs correctly, every request looks like it came from the edge. That breaks rate limiting, geo rules, and most security analytics.
You already set forwarding headers in the proxy block. Next, make sure your application trusts the edge and reads the right header.
The exact setting depends on your framework, but the rules are consistent:
- Trust
X-Forwarded-Foronly from the edge server’s private IP. - Prefer a controlled proxy chain rather than “trust all proxies.”
On the edge, tune your access logs so you can see upstream behavior during an incident. Add this to http {} in /etc/nginx/nginx.conf (or a file in /etc/nginx/conf.d/):
log_format lb_main '$remote_addr - $host [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" '
'upstream=$upstream_addr upstream_status=$upstream_status '
'rt=$request_time urt=$upstream_response_time';Then in your server block:
access_log /var/log/nginx/example.access.log lb_main;With this in place, you can answer “which backend is timing out?” in a minute, not an hour.
Step 7 — Add TLS with Let’s Encrypt (Certbot) on the edge
In this design, terminate TLS on the edge. You manage certificates in one place, and your backends stay private.
Install Certbot for Nginx:
sudo apt install -y certbot python3-certbot-nginxIssue and install the certificate (adjust domain):
sudo certbot --nginx -d example.com -d www.example.comCertbot will ask about redirecting HTTP to HTTPS. Choose the redirect unless you have a specific reason to keep HTTP.
If you want a full TLS walkthrough, use our SSL certificate setup guide for Nginx on Ubuntu VPS (2026), then return here for the load-balancing pieces.
Step 8 — Zero-downtime config reloads and safe deploy workflow
Nginx can reload configuration without dropping established connections. Test first, then use reload (or nginx -s reload).
Use a repeatable reload routine
sudo nginx -t && sudo systemctl reload nginxIf you change configs often, keep /etc/nginx under Git. At minimum, document a rollback path.
The fastest outage fixes are often “revert and reload.”
Drain a backend during deploys (simple technique)
If you’re deploying App1 and you don’t want new requests landing there, mark it as down and reload:
upstream app_pool {
server 10.0.1.11:3000 down;
server 10.0.1.12:3000 max_fails=3 fail_timeout=10s;
keepalive 64;
}Reload Nginx, deploy App1, and test it directly from the edge. Then remove down and reload again.
This isn’t service discovery. It is easy to run, easy to explain, and hard to misapply.
Step 9 — Add a basic status page (visibility without a monitoring overhaul)
You’ll appreciate visibility the first time a backend goes “slow” instead of “down.” Open-source Nginx includes stub_status.
Lock it down to localhost, your office IP, or a VPN.
Create /etc/nginx/conf.d/stub_status.conf:
server {
listen 127.0.0.1:8080;
server_name _;
location /nginx_status {
stub_status;
allow 127.0.0.1;
deny all;
}
}Then:
sudo nginx -t
sudo systemctl reload nginx
curl -s http://127.0.0.1:8080/nginx_statusIf you want alerting around spikes and outages, pair this with our VPS monitoring tutorial (2026).
Step 10 — Troubleshooting checklist (the stuff that actually breaks)
Load balancing problems often show up as “random 502s” or “only some users can log in.” Use this checklist to narrow the cause quickly.
Quick diagnostics
- Check Nginx sees both backends:
grep -R "upstream app_pool" -n /etc/nginx - Tail access logs with upstream fields:
sudo tail -f /var/log/nginx/example.access.log - Tail error logs:
sudo tail -f /var/log/nginx/error.log - Test backends from the edge:
curl -sS -m 2 http://10.0.1.11:3000/healthzcurl -sS -m 2 http://10.0.1.12:3000/healthz
Common symptoms and fixes
- 502 Bad Gateway: backend port closed, app crashed, or firewall blocks the edge. Validate with
ss -lntpon backends and UFW rules. - 504 Gateway Timeout: backend is slow or hanging. Shorten upstream timeouts, add a cheap readiness endpoint, and profile the app.
- Users randomly logged out: sessions stored in memory. Fix sessions (best) or use
ip_hashas a temporary measure. - Wrong client IP in app logs: app isn’t trusting proxy headers correctly. Configure “trust proxy” only for the edge IP.
For host-level hygiene, daily reports help you spot patterns before they become outages. Logwatch on Ubuntu is a low-effort way to catch repeated upstream failures and noisy clients.
Hardening notes for an edge load balancer
The edge is internet-facing. Treat it as a security boundary, not “just a proxy.”
- SSH: keys only, restricted users, allowlists if practical. Follow our SSH lockdown tutorial (2026).
- Firewall: only 80/443 inbound. Outbound from edge to backends only on required ports.
- Updates: enable unattended security updates or patch weekly on a schedule.
- Rate limiting: if bots hammer login/API endpoints, add Nginx limits at the edge. (Keep it surgical—don’t throttle the whole site.)
Summary: the operational “minimum viable” load balancer in 2026
You now have an edge Nginx instance balancing two app servers, terminating TLS, failing over cleanly during upstream errors, and reloading config without cutting active connections.
For many production sites, that’s the sweet spot. It’s simple enough to maintain, and strong enough to hold up under real traffic.
If you’re building this for a client project or a growing business, keep the edge stable and boring. Invest your time in deploy discipline, quick rollback, and clear logs.
If you want that stability without handling OS patching and web stack tuning yourself, run this pattern on managed VPS hosting or move up to a larger dedicated server once traffic and TLS handshakes start dominating CPU.
If you want an edge load balancer that stays fast and predictable under real traffic, start with a VPS that delivers consistent CPU and network performance. HostMyCode offers HostMyCode VPS plans for hands-on control, and managed VPS hosting if you’d rather offload maintenance while keeping root-level capability.
FAQ
Do I need sticky sessions for load balancing?
Only if your app keeps session state on a single server. If you store sessions in Redis or use stateless tokens, you can stick with round-robin and avoid skewed load.
How many backends can one Nginx edge handle?
The backend count is rarely the real constraint. TLS handshakes, connection concurrency, and response size usually matter more. Start with two backends, watch CPU, and scale the edge vertically if needed.
Why am I seeing 502s only sometimes?
Intermittent 502s usually mean one backend is unhealthy, a firewall is flaky, or timeouts are too strict. Use upstream-aware logs ($upstream_addr and $upstream_status) to pinpoint which backend is failing.
Should I terminate TLS on the edge or on the backends?
Terminate on the edge to keep certificate management simple and reduce moving parts. Use private networking for the edge-to-backend hop, and restrict backend ports to the edge only.
Can I do blue/green deploys with this setup?
Yes. Mark one backend down, reload, deploy and test it, then re-enable it. With two backends, this gives you a straightforward rolling deploy flow with minimal user impact.