Back to tutorials
Tutorial

Deploy Ollama on Debian With Docker and TLS

Run Ollama on Debian with Docker, HTTPS, systemd startup, and locked-down remote access for a safe self-hosted LLM endpoint.

By Anurag Singh
Updated on Oct 30, 2026
Category: Tutorial
Share article
Deploy Ollama on Debian With Docker and TLS

If you want to run Ollama on Debian in a repeatable, supportable way, use Docker with a reverse proxy. This setup keeps upgrades clean, terminates TLS, and gives you a stack you can manage with systemd.

This guide assumes a Debian server on a VPS or dedicated box. You’ll use Docker Compose for Ollama, Caddy for HTTPS, and systemd for reliable startup after reboots.

What you’ll build (and what you won’t)

  • Ollama running in Docker with a persistent model volume
  • TLS (HTTPS) on a real domain using Let’s Encrypt
  • Safe remote access by only exposing the reverse proxy, not Ollama directly
  • systemd-managed Docker Compose stack for predictable startup

Limitations to accept upfront:

  • A CPU-only Debian VPS can serve smaller models for low traffic, but latency will be noticeable on 7B+ models under load.
  • Ollama endpoints are not meant to be public, anonymous, high-traffic APIs without additional auth and rate-limiting.
  • Disk usage grows quickly. A few models can consume tens of GB.

Prerequisites and sizing for a Debian VPS

You can run this on a standard HostMyCode VPS. If you need more RAM or sustained throughput, use a dedicated server.

A practical baseline looks like this:

  • OS: Debian 12 (recommended), fresh install
  • CPU-only minimum: 4 vCPU / 8 GB RAM (good for small models and light testing)
  • Comfortable CPU-only: 8 vCPU / 16–32 GB RAM (better concurrency, larger context windows)
  • Disk: 80–160 GB SSD/NVMe (models + logs + OS; plan for growth)
  • Network: a stable IPv4 and a domain name pointed at it

Memory needs vary by model and quantization. The most common failure mode is swapping.

More RAM reduces swapping. Swapping can turn normal inference into a crawl.

If you only have 8 GB, start small. Keep concurrency low, then scale after you measure real usage.

Step 1: Create DNS records for your Ollama endpoint

Pick a hostname, for example: ollama.example.com. Create an A record pointing to your server’s public IPv4.

If you’re migrating an existing domain, lower the TTL first. That helps changes propagate faster.

This HostMyCode guide shows a clean cutover workflow: Migrate a Domain to Cloudflare DNS Without Downtime.

Step 2: Update Debian and install Docker Engine + Compose

SSH into your Debian server using a sudo-capable user.

sudo apt update
sudo apt -y upgrade
sudo apt -y install ca-certificates curl gnupg ufw

Install Docker using Debian packages.

Repo setup steps change over time. On Debian, the most reliable approach is Docker Engine from Docker’s official repository.

If Docker is already installed and working, keep your current method. Then continue.

Quick sanity checks once Docker is in place:

docker --version
docker compose version

Step 3: Lock down the firewall (only expose 22, 80, 443)

Keep Ollama’s port (11434) off the public internet. Expose only HTTP/HTTPS.

If possible, restrict SSH to known networks.

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

If you want a more detailed UFW walkthrough, see: Set Up UFW Firewall on Ubuntu for a Web Server. The same commands apply cleanly to Debian.

Step 4: Create the Ollama Docker Compose stack

This stack uses two containers:

  • ollama bound to localhost only (not reachable externally)
  • caddy exposed on 80/443, terminating TLS and proxying to Ollama

Create a working directory:

sudo mkdir -p /opt/ollama
sudo mkdir -p /opt/ollama/caddy
sudo chown -R root:root /opt/ollama

Create /opt/ollama/docker-compose.yml:

sudo tee /opt/ollama/docker-compose.yml > /dev/null <<'YAML'
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    # Bind only to localhost on the host. Do NOT expose to the internet.
    ports:
      - "127.0.0.1:11434:11434"
    volumes:
      - ollama:/root/.ollama

  caddy:
    image: caddy:latest
    container_name: ollama-caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./caddy/Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy_data:/data
      - caddy_config:/config
    depends_on:
      - ollama

volumes:
  ollama:
  caddy_data:
  caddy_config:
YAML

Step 5: Add TLS reverse proxy (Caddyfile)

Create /opt/ollama/caddy/Caddyfile. Replace the domain with your hostname.

sudo tee /opt/ollama/caddy/Caddyfile > /dev/null <<'CADDY'
ollama.example.com {
  encode gzip zstd

  # Basic hardening: don't let random browsers embed your endpoint.
  header {
    X-Content-Type-Options "nosniff"
    X-Frame-Options "DENY"
    Referrer-Policy "no-referrer"
  }

  # Optional: simple Basic Auth. Generate a bcrypt hash (see below).
  # basicauth /* {
  #   admin $2a$12$REPLACE_WITH_BCRYPT_HASH
  # }

  reverse_proxy 127.0.0.1:11434
}
CADDY

Optional but recommended: enable Basic Auth so the endpoint isn’t “anyone with the URL can use it.” Generate a bcrypt hash like this:

docker run --rm caddy:latest caddy hash-password --plaintext 'CHOOSE-A-LONG-PASSWORD'

Then uncomment the basicauth block. Set your username and paste in the bcrypt hash.

If you’d rather use Nginx than Caddy, this HostMyCode guide covers the same pattern (reverse proxy + Basic Auth) for Ollama: Run Ollama Behind Nginx With Basic Auth. The approach is the same on Debian, but file locations differ.

Step 6: Start the stack and verify HTTPS

Bring up the containers:

cd /opt/ollama
sudo docker compose up -d
sudo docker compose ps

If anything doesn’t look right, check logs next:

sudo docker compose logs -f --tail=200 caddy
sudo docker compose logs -f --tail=200 ollama

Test from your laptop (or any machine that can reach the domain). If you enabled Basic Auth, expect a 401 without credentials.

curl -sS https://ollama.example.com/api/tags | head

If TLS fails to issue, it’s usually one of these:

  • DNS still points to the old IP
  • Ports 80/443 blocked by firewall or upstream security group
  • Another web server already occupies 80/443

Step 7: Pull a model and run a first prompt

Pull a small model first. This validates the full path end-to-end.

It also keeps your first test from turning into a multi-GB download.

sudo docker exec -it ollama ollama pull llama3.2:3b
sudo docker exec -it ollama ollama list

Run a local prompt inside the container:

sudo docker exec -it ollama ollama run llama3.2:3b

Or test through the HTTPS endpoint (example using the generate API):

curl -sS https://ollama.example.com/api/generate \
  -H 'Content-Type: application/json' \
  -d '{"model":"llama3.2:3b","prompt":"Write a short maintenance checklist for a Debian VPS."}' \
  | head -n 20

If you enabled Basic Auth in Caddy, include credentials:

curl -sS -u 'admin:YOUR_PASSWORD' https://ollama.example.com/api/tags | head

Step 8: Add a systemd unit so the stack survives reboots

Docker can restart containers on its own. systemd gives you cleaner lifecycle control.

You also get predictable boot ordering, one status command, and a consistent stop/start path for maintenance.

Create /etc/systemd/system/ollama-compose.service:

sudo tee /etc/systemd/system/ollama-compose.service > /dev/null <<'UNIT'
[Unit]
Description=Ollama (Docker Compose) with Caddy TLS
Wants=network-online.target docker.service
After=network-online.target docker.service

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=/opt/ollama
ExecStart=/usr/bin/docker compose up -d
ExecStop=/usr/bin/docker compose down
TimeoutStartSec=0

[Install]
WantedBy=multi-user.target
UNIT

Enable and start it:

sudo systemctl daemon-reload
sudo systemctl enable --now ollama-compose.service
sudo systemctl status ollama-compose.service --no-pager

Reboot once to confirm it comes back cleanly:

sudo reboot

Step 9: Safe remote access patterns (choose one)

You’ve got HTTPS working. Next, decide who can reach it and how.

Pick the option that fits your environment and threat model.

Option A (simple): HTTPS + Basic Auth

This works for personal use or a small team. Use a strong password, and don’t share it widely.

If you expose it publicly, Basic Auth alone won’t stop password guessing.

Option B (stronger): Put the endpoint behind a private tunnel

If you don’t want the API reachable from the public internet, use a tunnel. It’s a clean operational choice.

HostMyCode covers this approach here: Secure Ollama API With Cloudflare Tunnel on VPS.

Option C (ops-friendly): Access over WireGuard VPN

For admin teams, a VPN is often the simplest control plane.

You can keep Ollama on a private IP and run Caddy internally.

This WireGuard setup works well on Debian too: Set Up WireGuard VPN on Ubuntu VPS for Admin Access.

Step 10: Operational checks (disk, memory, logs)

Most self-hosted AI outages aren’t mysterious. The disk fills, RAM runs hot, or a service starts flapping.

Check the basics early. Then keep an eye on them.

Disk usage

df -h
sudo docker system df
sudo du -sh /var/lib/docker 2>/dev/null || true

If you’re tight on space, remove unused images:

sudo docker image prune -a

Be careful: pruning removes unused images and can slow the next deployment.

Memory pressure

free -h
vmstat 1 5

If swapping shows up under load, you have three levers: smaller/quantized models, fewer concurrent requests, or more RAM.

On a busy system, swapping can turn a 3–10 second response into minutes.

Service health

sudo docker compose -f /opt/ollama/docker-compose.yml ps
sudo journalctl -u ollama-compose.service -n 200 --no-pager

Upgrading Ollama safely (without breaking the endpoint)

Docker makes upgrades boring, which is the goal. Use this routine:

cd /opt/ollama
sudo docker compose pull
sudo docker compose up -d
sudo docker image prune

After the update, re-test the endpoint:

curl -sS https://ollama.example.com/api/tags | head

Troubleshooting: the common failure cases

  • HTTPS won’t issue: confirm DNS points to the server, ports 80/443 are open, and nothing else is bound to 80/443 (ss -tulpn | grep -E ':80|:443').
  • 502/504 from the proxy: Ollama may be down, or it’s under heavy load. Check docker compose logs. If you’re also proxying other apps, keep this handy: Nginx 502 Bad Gateway Fix on Ubuntu VPS (concepts apply to Caddy too).
  • Requests hang: you’re likely swapping or saturating CPU. Verify with free -h and top.
  • Model download is slow: check VPS bandwidth, disk I/O, and that you have enough free space.

Where this fits in real hosting (websites, apps, and internal tools)

Once the HTTPS endpoint is stable, you can call it from:

  • a private admin dashboard that generates drafts or summaries
  • a coding assistant running inside your network
  • workflow automation (tickets, alerts, runbooks) that calls your own model instead of a third-party API

If you expect customer-facing traffic, plan for isolation. A dedicated VPS for inference helps avoid noisy-neighbor issues with your web stack.

If you’d rather not spend nights debugging OS updates, Docker runtime problems, and firewall edge cases, managed VPS hosting can be the cheaper option in the long run.

If you want a stable place to run private AI workloads, start with a HostMyCode VPS sized for your models and traffic. If you don’t want to own OS updates, Docker maintenance, and firewall tuning, managed VPS hosting is a practical step up.

FAQ

Is it safe to expose Ollama to the public internet?

Not by default. Put it behind HTTPS and authentication at minimum, and prefer a tunnel or VPN for anything sensitive. Also avoid exposing port 11434 directly.

Can I run Ollama on a small Debian VPS (2 vCPU, 4 GB RAM)?

You can install it, but you’ll hit RAM limits quickly and response times will be poor. It’s fine for learning, not for reliable usage.

Do I need a GPU?

No, but CPU-only inference is slower. If you need interactive speed for more users, move to a GPU machine or a larger dedicated server.

Where are Ollama models stored in this setup?

Inside the Docker named volume ollama, mounted at /root/.ollama in the container. Docker stores volumes under /var/lib/docker/volumes/ on the host.

Summary: a repeatable Debian deployment you can run in production

You now have a Debian stack that can run Ollama on Debian with Docker, terminate HTTPS on a real domain, and restart cleanly via systemd.

Keep the Ollama port private. Treat the endpoint like an internal service.

Watch disk and RAM before they turn into incidents.

If you’re ready to host this on real infrastructure, a properly sized HostMyCode VPS gives you the RAM and storage headroom models need, without sharing resources with your primary website.