Run Ollama on a VPS With Nginx Reverse Proxy

Most “self-hosted AI” projects stumble on the unglamorous parts. You need to expose the model safely, keep it running after reboots, and avoid publishing an open endpoint to the internet.

This walkthrough shows how to run Ollama on a VPS as a system service. Then you’ll put it behind Nginx with basic auth and real TLS.

You’ll end up with a private HTTPS endpoint for internal tools, an n8n workflow, a staging setup, or a small team. This is not a hyperscale inference platform.

It’s a clean, predictable deployment you can operate without babysitting it.

What you’ll build (and what it costs in resources)

• Ollama running as a systemd service on Linux
• One pulled model for testing and real usage
• Nginx reverse proxy in front of Ollama
• HTTP basic auth to stop casual abuse
• HTTPS via Let’s Encrypt so tokens and prompts don’t travel in cleartext

Recommended server sizing (CPU-only):

• Minimum: 2 vCPU, 4 GB RAM (ok for tiny models, slow responses)
• Comfortable baseline: 4 vCPU, 8–16 GB RAM (better concurrency, fewer out-of-memory surprises)
• Disk: 30–60 GB depending on how many models you keep (models can be several GB each)

GPU note: This guide assumes CPU inference because most VPS plans don’t include GPUs. If your server has a GPU, Ollama can use it.

The proxy and security steps below stay the same.

Need a clean Linux box for this? A HostMyCode VPS works well for a private endpoint.

If you’d rather not handle patching and baseline security yourself, managed VPS hosting is the lower-maintenance option.

Prerequisites

• A VPS running Ubuntu or Debian (this guide uses Ubuntu-style commands)
• A domain or subdomain pointing to the VPS (example: ollama.example.com)
• SSH access with sudo

If you’re moving DNS and want to test without downtime, follow the approach in our DNS cutover tutorial for migrations.

If you still need a domain, you can register and manage it via HostMyCode domains.

Step 1: Update the VPS and install required packages

Start from a sane baseline. You’re about to expose an API endpoint, so patch first.

sudo apt update
sudo apt -y upgrade
sudo apt -y install curl nginx apache2-utils

apache2-utils provides htpasswd for basic auth.

Step 2: Install Ollama and verify it runs locally

Install Ollama with the official script. The installer often sets up a service for you.

Even so, verify the process. Then take control over how it binds.

curl -fsSL https://ollama.com/install.sh | sh

Confirm the service is present and running:

systemctl status ollama --no-pager

Next, hit the local API. Ollama typically listens on 127.0.0.1:11434 or 0.0.0.0:11434, depending on configuration.

For now, you just want a local response.

curl -s http://127.0.0.1:11434/api/tags | head

If you see JSON (even an empty list), the API is responding.

Step 3: Pull a model (choose something realistic for your RAM)

This is where sizing stops being theoretical. Pull one model, test it, and only then decide whether to add more.

ollama pull llama3.1

If that’s too heavy for your VPS, pick a smaller model from Ollama’s library.

Rule of thumb: when model size plus overhead crowds your RAM, the server will swap. Latency will get ugly.

Run a quick local prompt test:

ollama run llama3.1 "Write a 3-bullet checklist for hardening an API server."

Honest limitation: CPU-only inference works for low-volume internal usage. With several concurrent users, you’ll saturate CPU quickly.

Keep this private and controlled. Don’t point public traffic at it and hope for the best.

Step 4: Make sure Ollama only listens locally (recommended)

The safest layout is simple. Ollama binds to localhost, and only Nginx can reach it.

Even if you slip up elsewhere, the API won’t be directly exposed to the internet.

Start by checking what address Ollama is listening on:

sudo ss -ltnp | grep 11434

If you see 0.0.0.0:11434 or [::]:11434, it’s reachable from outside unless something blocks it.

Switch to localhost binding using a systemd override.

Create an override file:

sudo systemctl edit ollama

In the editor, add:

[Service]
Environment="OLLAMA_HOST=127.0.0.1:11434"

Reload systemd and restart Ollama:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Check the listener again:

sudo ss -ltnp | grep 11434

You should now see 127.0.0.1:11434.

Step 5: Add basic authentication for the endpoint

Basic auth won’t stop a determined attacker. It does reliably block random scans and “oops, I exposed it” mistakes.

You’ll add TLS next. That keeps credentials out of cleartext.

Create an htpasswd file:

sudo mkdir -p /etc/nginx/auth
sudo htpasswd -c /etc/nginx/auth/ollama.htpasswd ollamaadmin

Pick a strong password and store it in your password manager. To add users later, omit -c:

sudo htpasswd /etc/nginx/auth/ollama.htpasswd developer1

Step 6: Configure Nginx reverse proxy for Ollama

Create an Nginx server block for your subdomain. Swap ollama.example.com for your real hostname.

sudo nano /etc/nginx/sites-available/ollama

Paste this configuration (HTTP first; TLS comes next):

server {
    listen 80;
    server_name ollama.example.com;

    # Keep request bodies reasonable; adjust if you send large contexts.
    client_max_body_size 25m;

    location / {
        auth_basic "Restricted";
        auth_basic_user_file /etc/nginx/auth/ollama.htpasswd;

        proxy_pass http://127.0.0.1:11434;
        proxy_http_version 1.1;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        # Helps with long responses/streams.
        proxy_read_timeout 300;
        proxy_send_timeout 300;
    }
}

Enable the site and reload Nginx:

sudo ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/ollama
sudo nginx -t
sudo systemctl reload nginx

Test from your laptop (or any client) with basic auth:

curl -u ollamaadmin 'http://ollama.example.com/api/tags'

If DNS hasn’t propagated, test from the server itself. Or temporarily add an entry to /etc/hosts on your machine.

Step 7: Add TLS with Let’s Encrypt

Basic auth without TLS exposes credentials to anyone who can intercept traffic. Fix that by issuing a certificate for the subdomain.

If you want a full HTTPS walkthrough (plus common failure cases), use: TLS setup guide with Let’s Encrypt on a VPS.

Install Certbot for Nginx:

sudo apt -y install certbot python3-certbot-nginx

Request the certificate and let Certbot update Nginx:

sudo certbot --nginx -d ollama.example.com

When prompted, choose the option to redirect HTTP to HTTPS.

Verify renewals:

sudo systemctl status certbot.timer --no-pager
sudo certbot renew --dry-run

Re-test over HTTPS:

curl -u ollamaadmin 'https://ollama.example.com/api/tags'

Step 8: Lock down the firewall (only 22, 80, 443)

Even with localhost binding, keep inbound ports tight. Allow SSH and web traffic only.

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable
sudo ufw status verbose

If you want the rationale and safe defaults for web servers, follow: Set up UFW firewall on Ubuntu for a web server.

Step 9: Test an end-to-end chat call through Nginx

Ollama exposes an HTTP API. /api/generate is one of the common endpoints.

Payloads vary by client, but this curl is a solid sanity check from your workstation.

curl -u ollamaadmin https://ollama.example.com/api/generate \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "llama3.1",
    "prompt": "Write a short Nginx checklist for protecting an internal API.",
    "stream": false
  }'

If you get JSON back with generated text, the reverse proxy, auth, and TLS are all doing their jobs.

Operational checklist: keep it stable on a production VPS

This is the difference between a quick demo and something you can run for months.

• Watch memory: once the box swaps, latency spikes fast.
• Limit who can reach it: keep it private (VPN), or at least use auth and rate limits.
• Patch regularly: Nginx is internet-facing, so updates matter.
• Log intentionally: prompts can contain sensitive data; don’t log request bodies.

For monitoring on a VPS or dedicated server, set up alerting early.

Our server monitoring setup guide with Netdata is a strong starting point.

Optional hardening: rate limit and restrict methods in Nginx

If your endpoint gets used internally, someone will eventually aim a loop at it. Rate limiting helps keep one client from consuming all your CPU.

Add a rate-limit zone in /etc/nginx/nginx.conf inside the http {} block:

limit_req_zone $binary_remote_addr zone=ollama_ratelimit:10m rate=10r/s;

Then apply it in your Ollama server block:

location / {
    limit_req zone=ollama_ratelimit burst=20 nodelay;

    auth_basic "Restricted";
    auth_basic_user_file /etc/nginx/auth/ollama.htpasswd;

    proxy_pass http://127.0.0.1:11434;
    proxy_http_version 1.1;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    proxy_read_timeout 300;
    proxy_send_timeout 300;
}

Reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

For a deeper treatment (and patterns that translate well to AI APIs), see: Nginx rate limiting tutorial.

Quantization and model choice: practical guidance for CPU VPS

On CPU, the biggest wins usually come from picking a smaller model or a quantized variant. You trade quality for speed and lower RAM use.

• If you need speed and low RAM: choose smaller parameter counts and quantized builds.
• If you need better answers: expect higher RAM use and more CPU time per token.
• If you need concurrency: plan for more vCPU and RAM, or move to a GPU-capable server.

While you test, keep a reality-check open:

htop

If ollama pegs CPU and your websites share the same VPS, separate workloads. Put your web stack and the AI endpoint on different instances.

Or move inference to a dedicated server.

Troubleshooting: common failures and fast fixes

Nginx returns 502 Bad Gateway

• Confirm Ollama is running: systemctl status ollama
• Confirm the listener: ss -ltnp | grep 11434
• Confirm Nginx points to the right address: proxy_pass http://127.0.0.1:11434;
• Check logs: sudo tail -n 200 /var/log/nginx/error.log

Certbot fails to issue a certificate

• Make sure DNS A/AAAA records point to the VPS IP.
• Confirm ports 80 and 443 are reachable from the internet.
• Temporarily disable conflicting server blocks or duplicate server_name entries.

Ollama is running, but responses are painfully slow

• Check swapping: free -h (look at Swap)
• Check load: uptime and htop
• Try a smaller model; then re-test the same prompt.

“413 Request Entity Too Large” in Nginx

• Increase client_max_body_size in the server block.
• Reload Nginx: sudo systemctl reload nginx

Production safety notes (read before exposing this broadly)

An LLM endpoint isn’t just another JSON API. People paste credentials, customer data, and internal context into prompts.

Assume sensitive input will hit this service.

• Don’t log prompts by default. Keep Nginx access logs, but avoid request-body logging.
• Use separate credentials per team if you need accountability. Basic auth supports multiple users.
• Prefer private network access (VPN, IP allow-lists) for anything beyond a small trusted group.
• Backups: decide whether you need to back up pulled models. Often you don’t; you can re-pull.

If this VPS also runs websites, apply the same hardening you’d apply to any production host.

Our web server hardening tutorial for Nginx + PHP-FPM covers headers and practical guardrails.

Summary: what you have now

You’ve installed Ollama as a service, pulled a model, and exposed a protected HTTPS endpoint through Nginx. That’s enough for internal tooling, a staging environment, or a small production workflow where you control who can call it.

If you outgrow CPU inference, don’t spend weeks trying to “tune” your way out of a hardware limit. Split workloads or upgrade.

For steadier performance, place the endpoint on a right-sized HostMyCode VPS (or move to a dedicated box if it becomes core). Keep your web apps isolated from inference spikes.

FAQ

Can I run Ollama and my website on the same VPS?

You can, but it’s easy to regret. Inference spikes CPU and RAM, which turns into slow pages and timeouts.

If the website matters, separate them, or enforce strict limits and watch metrics closely.

Is basic auth enough to secure an Ollama endpoint?

Basic auth + TLS is a solid baseline for a small, trusted team. For wider access, add IP allow-lists, VPN access, rate limits, and separate credentials per user.

Treat it like a sensitive internal API.

Why bind Ollama to 127.0.0.1 instead of 0.0.0.0?

Binding to localhost keeps the API reachable only from the VPS itself. Nginx becomes the single controlled entry point, which cuts exposure and simplifies firewall rules.

How do I update Ollama safely?

Update during a maintenance window, restart the service, then re-test one known prompt through Nginx.

For critical workflows, keep a second VPS as a warm standby and cut traffic over after validation.

What’s the fastest way to tell if the model is too big for my VPS?

Watch memory and swap while you run a few prompts: free -h and htop. If swap climbs or the system starts thrashing, move to a smaller/quantized model or add RAM.