Secure vLLM API With Nginx, Keys, and Rate Limits

Your vLLM endpoint is real compute. If you leave it exposed, it can burn GPU hours, leak prompts, or become an open relay for abuse.

This tutorial shows a concrete, production-friendly way to secure vLLM API traffic with Nginx in front of an OpenAI-compatible vLLM server. You’ll add TLS, an API key check, optional IP allowlists, and basic rate limiting.

This is written for operators running a GPU VPS or a dedicated GPU box. You’ll get paste-ready configs and a few quick checks.

The goal is simple: don’t accidentally publish an unauthenticated inference endpoint.

Architecture you’ll deploy (and what it protects)

Run vLLM on a private bind (localhost). Put Nginx on the public interface.

Nginx will:

• Terminate TLS (HTTPS) using Let’s Encrypt.
• Enforce an API key check on every request (simple, reliable).
• Optionally restrict access by IP (allowlist) for admin/testing clients.
• Apply basic rate limits to reduce brute-force and cost blowups.
• Set sane proxy timeouts for long generations and streaming responses.

vLLM keeps speaking OpenAI-compatible HTTP. Your apps can keep using standard OpenAI client libraries. Just point them at your domain.

Prerequisites (VPS sizing, OS, DNS)

Infrastructure: vLLM expects a GPU machine. You can run it on CPU for experiments, but request times get long fast.

During generations, the API may even look “down.”

• OS: Ubuntu 22.04/24.04 LTS or Debian 12 on a VPS/dedicated server.
• GPU: NVIDIA GPU with drivers installed. VRAM needs depend on the model and quantization; budget conservatively.
• RAM: At least 16 GB for small models; 32–64 GB is more comfortable for production.
• Disk: Fast NVMe and enough space for model weights (often 10–100+ GB).
• Domain: A DNS A/AAAA record pointing to the server. Manage your domain at HostMyCode Domains if you want DNS and renewals under one roof.

If you still need a machine, a GPU-capable plan on a HostMyCode VPS (or a dedicated GPU server for sustained traffic) is the most straightforward path.

You get predictable performance and network control.

Step 1: Run vLLM bound to localhost only

The most important hardening choice comes first: do not bind vLLM to 0.0.0.0 while auth is still in flux.

Keep vLLM on localhost and expose only Nginx. This tutorial assumes vLLM listens on 127.0.0.1:8000.

Option A: vLLM already running

First, confirm it’s only reachable locally:

ss -lntp | grep 8000
curl -s http://127.0.0.1:8000/v1/models | head

You should see vLLM listening on 127.0.0.1:8000 (or localhost).

If it’s on 0.0.0.0:8000, fix the bind address in your vLLM run command or service file.

Option B: Quick run command (example)

If you want a baseline, start with this pattern. The exact flags vary by model and GPU.

The key detail is the host binding.

# Example pattern (adjust model and flags for your environment)
# Keep it bound to localhost:
vllm serve <your-model-id-or-path> --host 127.0.0.1 --port 8000

For a dedicated guide on getting vLLM running with an OpenAI-compatible API, see: Run vLLM on a GPU VPS With OpenAI-Compatible API.

Step 2: Install Nginx and basic tooling

On Ubuntu/Debian:

sudo apt update
sudo apt install -y nginx certbot python3-certbot-nginx apache2-utils

• nginx: reverse proxy and TLS terminator
• certbot: Let’s Encrypt certificates
• apache2-utils: provides htpasswd if you want Basic Auth for an admin-only endpoint (optional)

Step 3: Create a dedicated Nginx site for your API

Pick an API hostname, for example llm-api.example.com. Create a new Nginx server block:

sudo nano /etc/nginx/sites-available/vllm-api.conf

Start with a small HTTP (port 80) config. Certbot will modify it later.

server {
  listen 80;
  listen [::]:80;

  server_name llm-api.example.com;

  location / {
    return 200 "ok\n";
    add_header Content-Type text/plain;
  }
}

Enable it and reload:

sudo ln -s /etc/nginx/sites-available/vllm-api.conf /etc/nginx/sites-enabled/vllm-api.conf
sudo nginx -t
sudo systemctl reload nginx

Verify DNS and basic reachability:

curl -i http://llm-api.example.com/

Step 4: Add TLS (Let’s Encrypt) the safe way

Issue a certificate and let Certbot wire up TLS:

sudo certbot --nginx -d llm-api.example.com

When prompted, choose the redirect option so HTTP automatically upgrades to HTTPS.

If you prefer a reverse proxy that handles TLS with minimal config, Caddy is a solid option.

This tutorial sticks to Nginx because it makes rate limiting and header checks easy to control. For a Caddy walkthrough, see: Set Up Let’s Encrypt With Caddy for a VPS Site.

Step 5: Enforce an API key at Nginx (no app changes required)

Depending on how you run it, vLLM can accept an OpenAI-style API key header (Authorization: Bearer ...).

Even so, upstream auth is easy to mis-bind or bypass during changes.

Enforcing the check at Nginx keeps the gate in one place.

We’ll require a fixed bearer token. It’s not a full identity system.

It is a practical barrier that stops most scanning and opportunistic abuse.

Create a key

Generate a long random token and store it on the server. You can generate it locally, but here’s a server-side example:

sudo mkdir -p /etc/nginx/secret
sudo chmod 700 /etc/nginx/secret

KEY=$(openssl rand -hex 32)
echo "$KEY" | sudo tee /etc/nginx/secret/vllm_api_key.txt > /dev/null
sudo chmod 600 /etc/nginx/secret/vllm_api_key.txt

echo "Your API key is: $KEY"

Put that key in your password manager. Anyone who has it can use your inference endpoint.

Update the Nginx TLS server block

After Certbot runs, your Nginx config will include an HTTPS server section. Edit your site file:

sudo nano /etc/nginx/sites-available/vllm-api.conf

Inside the server { ... listen 443 ssl ... } block, replace the location / with the block below.

If you don’t see a 443 block, Certbot may have written a separate file.

Look under /etc/nginx/sites-enabled/ for your hostname.

# Put this inside the TLS server block for llm-api.example.com

# 1) Read the expected key from disk at startup
# Nginx can't read files directly into a variable without modules,
# so we hardcode the key OR use map with an env var.
# Practical approach: hardcode the key in config and protect the file.

# Replace YOUR_KEY_HERE with the generated token
set $expected_bearer "Bearer YOUR_KEY_HERE";

# 2) Only proxy if the key matches
if ($http_authorization != $expected_bearer) {
  return 401;
}

location / {
  proxy_pass http://127.0.0.1:8000;

  # Streaming responses (Server-Sent Events) behave better with these:
  proxy_http_version 1.1;
  proxy_set_header Connection "";
  proxy_buffering off;

  # Preserve client info
  proxy_set_header Host $host;
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto $scheme;

  # Long generations need longer timeouts
  proxy_read_timeout 300s;
  proxy_send_timeout 300s;
}

Why hardcode the key? It’s blunt, but it behaves predictably. It also avoids extra Nginx modules.

Treat the config as a secret (root-readable only). Rotate keys by updating the config and reloading Nginx.

Reload and test:

sudo nginx -t
sudo systemctl reload nginx

# Without key (should fail):
curl -i https://llm-api.example.com/v1/models

# With key (should succeed):
curl -i https://llm-api.example.com/v1/models \
  -H "Authorization: Bearer YOUR_KEY_HERE"

Step 6: Add IP allowlists (optional, but recommended for admin clients)

If only a few systems should ever call the API (app servers, an office IP, a WireGuard subnet), an allowlist cuts the attack surface quickly.

You can combine allowlist + API key. Or you can use an allowlist alone for internal-only endpoints.

Add this above the auth check in the TLS server block:

# Example allowlist: your office IP and one app server
allow 203.0.113.10;
allow 198.51.100.24;

# If you route traffic over WireGuard, allow the VPN subnet
# allow 10.10.0.0/24;

deny all;

Reload Nginx and test from an allowed IP.

If you lock yourself out, use your provider console to revert the config.

If you want a clean VPN path for management traffic, WireGuard plus a non-public API is usually the simplest approach. This guide may help: Set Up WireGuard VPN on Ubuntu VPS for Admin Access.

Step 7: Add basic rate limiting (requests per second + burst)

Rate limiting won’t stop a determined attacker. It will stop accidental loops, crude brute-force attempts, and “someone got the key and is hammering completions” from wrecking your GPU bill.

Nginx rate limiting has two parts:

• limit_req_zone (global, in http {})
• limit_req (per server/location)

Configure a rate limit zone

Edit the main Nginx config:

sudo nano /etc/nginx/nginx.conf

Inside the http { ... } block, add:

# Rate limit by client IP (binary form saves memory)
limit_req_zone $binary_remote_addr zone=vllm_per_ip:10m rate=2r/s;

Translation:

• 10m: shared memory zone size (enough for many clients)
• 2r/s: average 2 requests per second per IP

Now apply it to the vLLM location block:

# Put inside: location / { ... }
limit_req zone=vllm_per_ip burst=10 nodelay;

Practical tuning:

• For interactive chat: 1–3 r/s with burst 10–30 is common.
• For batch embeddings/completions: you may want higher bursts but lower sustained rates.
• If you have multiple users behind one NAT (office), IP-based limiting can punish everyone. In that case, rate limit on an API key identifier at the app layer, or split keys by user.

Reload and validate:

sudo nginx -t
sudo systemctl reload nginx

Generate a small burst to confirm you get 429 once you exceed the limit:

for i in $(seq 1 30); do
  curl -s -o /dev/null -w "%{http_code}\n" \
    https://llm-api.example.com/v1/models \
    -H "Authorization: Bearer YOUR_KEY_HERE"
done

Step 8: Harden request sizes and timeouts (avoid easy DoS)

LLM endpoints attract big payloads: long prompts, large tool schemas, and users pasting entire logs.

Set explicit caps so one request can’t eat memory. Also keep requests from tying up workers indefinitely.

Cap request body size

Add inside the TLS server block:

client_max_body_size 2m;

If you expect larger prompts, raise it gradually (for example 10m). Don’t leave it unlimited.

If you start seeing failures after tightening this, they look like standard Nginx upload errors. This guide is a handy reference: Nginx 413 Request Entity Too Large Fix On VPS.

Keep timeouts honest

We already set proxy_read_timeout to 300s. Increase it if your hardware is slower or you allow longer generations.

If users routinely need 10+ minute generations, treat it as a capacity issue. Don’t paper over it with a timeout tweak.

Step 9: Add access logs that help (and don’t leak secrets)

Nginx logs help you spot abuse and debug patterns. Don’t log the Authorization header.

By default, Nginx access logs won’t include it unless you customize log_format. Keep it that way.

For this API server, add a dedicated log file inside your TLS server block:

access_log /var/log/nginx/vllm-api.access.log;
error_log  /var/log/nginx/vllm-api.error.log;

Tail logs while testing:

sudo tail -f /var/log/nginx/vllm-api.access.log /var/log/nginx/vllm-api.error.log

Step 10: Lock down the host firewall to only what you need

Binding vLLM to localhost removes direct network exposure. Still, keep the host firewall strict.

Allow SSH, HTTP, and HTTPS. Block the rest unless you have a specific reason.

On Ubuntu with UFW:

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

If you run a database or vector store on the same server, don’t open it to the public internet.

Connect locally or over a private network.

Step 11: Quick client test using OpenAI-compatible calls

With Nginx enforcing HTTPS and the API key, point your client at your base URL.

Here’s a minimal curl chat call (the exact JSON depends on your model and vLLM config):

curl https://llm-api.example.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_KEY_HERE" \
  -d '{
    "model": "your-model-name",
    "messages": [
      {"role": "user", "content": "Give me a one-sentence status update."}
    ],
    "temperature": 0.2
  }'

If you get 502 Bad Gateway, Nginx can’t reach vLLM.

Confirm vLLM is listening on 127.0.0.1:8000 and that your upstream target matches.

This troubleshooting flow helps: Nginx 502 Bad Gateway Fix on Ubuntu VPS.

Honest limitations (what this setup does not solve)

• One shared key is not user identity. If the key leaks, rotate it and review logs. For multi-tenant usage, issue per-user keys at the application layer and enforce per-key quotas.
• IP allowlists break mobile users. Roaming clients will trip denies. For those cases, use a VPN or an auth-aware proxy.
• Rate limits are coarse. Token-based limits (tokens/minute) track LLM cost better, but they require app-layer accounting.
• Nginx can’t prevent “expensive prompts”. You still need sane model choices, max tokens, and timeouts in your vLLM/app config.

Operational checklist for production

• Bind vLLM to 127.0.0.1, not a public interface.
• Enforce HTTPS redirects and renew certificates automatically (Certbot timer).
• Require an API key (Bearer token) at Nginx.
• Add an IP allowlist for admin-only usage where possible.
• Rate limit requests and cap request body size.
• Keep logs, but don’t log auth headers.
• Firewall: only 22/80/443 open to the world.

FAQ: securing a self-hosted vLLM endpoint

Can I put vLLM directly on the internet and rely on its key support?

You can, but it’s easy to bind the service to the wrong interface. It’s also easy to end up with auth that isn’t consistently enforced.

Putting Nginx in front gives you a single gate (TLS + key + limits). That gate stays in place regardless of vLLM flags.

Should I use Basic Auth instead of a Bearer token?

Basic Auth works, but most OpenAI-compatible clients already send a Bearer token. Sticking with that convention keeps client-side changes minimal.

What rate limit should I start with?

Start conservative (for example 1–2 requests/sec per IP with a burst of 10). Watch GPU utilization and Nginx 429s. Then adjust based on real traffic.

How do I rotate the API key without downtime?

Add a second accepted key temporarily (for example with a map and multiple comparisons). Reload Nginx. Update clients. Then remove the old key and reload again.

If you need help, document the rollout in a runbook.

Summary: a practical way to secure vLLM without rewriting your app

You now have a public endpoint that’s meaningfully hardened.

vLLM stays private on localhost. Nginx enforces HTTPS, checks an API key, can restrict by IP, and applies basic rate limiting.

That’s the line between “demo on the internet” and something you can operate safely.

If you expect to run this long-term, size the machine for your model. Leave headroom for concurrency.

A HostMyCode VPS is a practical starting point, and moving to dedicated hardware is straightforward once traffic and model requirements stabilize.