Ollama Systemd Service on Ubuntu VPS

Running Ollama in a shell works until your VPS reboots or your SSH session drops. It also leaves you guessing when a model load hangs. This guide shows how to set up an Ollama systemd service on an Ubuntu VPS so it starts on boot, restarts predictably, logs cleanly, and exposes the API on your terms.

You’ll create a dedicated ollama user, a locked-down unit file, and an access pattern that holds up in real projects. The goal is uptime and safety, not clever tricks.

What you’ll build (and what you won’t)

• Build: A production-friendly systemd service that keeps Ollama running, limits privileges, writes logs to journald, and survives reboots.
• Build: An optional “localhost + reverse proxy” setup so your LLM endpoint isn’t exposed directly to the internet.
• Won’t: GPU tuning, multi-node serving, or Kubernetes. This stays focused on one VPS and a sane operator workflow.

Prerequisites and sizing guidance for a VPS

Ollama can run on CPU-only hardware. Your model choice decides whether it feels smooth or painful.

Use these baselines as a starting point:

• CPU-only (small models / light use): 4 vCPU, 8–16 GB RAM.
• CPU-only (medium models / dev team use): 8 vCPU, 32 GB RAM.
• Disk: NVMe helps. Keep /var/lib/ollama on fast storage; models can take multiple GB each.

If you expect multiple users to hit the endpoint, don’t underprovision. Sustained inference can pin CPU for long stretches.

A HostMyCode VPS is a common fit for self-hosted Ollama. You get full control over systemd, firewall rules, and storage locations.

OS assumption: Ubuntu Server on a fresh or maintained VPS with sudo access.

Step 1: Install Ollama (or confirm it’s installed)

If you haven’t installed Ollama yet, follow HostMyCode’s install workflow. Then return here for the systemd setup.

See: Install Ollama on Ubuntu VPS with Nginx and TLS.

To check an existing install:

command -v ollama
ollama --version || true

If ollama is present, capture the binary path:

which ollama

You’ll use that path in the systemd unit. It’s usually /usr/local/bin/ollama or /usr/bin/ollama.

Step 2: Create a dedicated system user and directories

Don’t run Ollama as root. Use a dedicated user instead.

This keeps permissions simpler and narrows the process scope.

sudo useradd --system --home /var/lib/ollama --create-home --shell /usr/sbin/nologin ollama

Create directories for models and runtime state:

sudo install -d -o ollama -g ollama -m 0750 /var/lib/ollama
sudo install -d -o ollama -g ollama -m 0750 /var/lib/ollama/models
sudo install -d -o ollama -g ollama -m 0755 /run/ollama

Why /run/ollama? On Ubuntu, /run is tmpfs. It’s meant for sockets, PID files, and other runtime-only data.

systemd can create /run/ollama automatically (and we’ll configure that). Creating it now just makes early testing easier.

Step 3: Decide how you’ll expose the Ollama API

Ollama serves an HTTP API. On most VPS setups, the safest default is straightforward.

Bind to localhost, then expose it through a reverse proxy with TLS and auth. If you prefer, keep it private behind a VPN.

• Best default for public servers: Bind to 127.0.0.1:11434 and proxy with Nginx/Caddy plus authentication.
• Acceptable for private networks: Bind to the server’s private IP and restrict inbound via firewall/VPN.
• Avoid: Listening on 0.0.0.0 with no auth. That’s an open invitation for unwanted inference bills.

If you want a ready-to-use proxy pattern with credentials, this guide matches what we’re doing here: Run Ollama Behind Nginx With Basic Auth.

In this tutorial, the systemd unit defaults to localhost binding.

Step 4: Create a hardened Ollama systemd service unit

Create a unit file:

sudo nano /etc/systemd/system/ollama.service

Paste the unit below. If your ollama binary lives elsewhere, update ExecStart:

[Unit]
Description=Ollama LLM API Server
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=ollama
Group=ollama

# Where models live
Environment=OLLAMA_HOME=/var/lib/ollama
Environment=OLLAMA_MODELS=/var/lib/ollama/models

# Bind only to localhost by default. Proxy externally via Nginx/Caddy or keep private.
Environment=OLLAMA_HOST=127.0.0.1:11434

# If you run into slow pulls or want bigger responses, you can set these later.
# Environment=OLLAMA_NUM_PARALLEL=1
# Environment=OLLAMA_KEEP_ALIVE=5m

WorkingDirectory=/var/lib/ollama
ExecStart=/usr/local/bin/ollama serve

# Reliability
Restart=on-failure
RestartSec=2
TimeoutStartSec=120
TimeoutStopSec=30

# systemd creates /run/ollama on each boot with correct ownership
RuntimeDirectory=ollama
RuntimeDirectoryMode=0755

# Hardening: keep it strict, but not so strict that updates break
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true

# Allow writing only where needed
ReadWritePaths=/var/lib/ollama /run/ollama

# Reduce kernel attack surface
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
LockPersonality=true
RestrictSUIDSGID=true
RestrictNamespaces=true
RestrictRealtime=true

# Networking is required (model pulls, API serving)
IPAddressDeny=any
IPAddressAllow=localhost

# If you need outbound internet to pull models directly from this VPS,
# comment out the IPAddressDeny/Allow lines above.

[Install]
WantedBy=multi-user.target

Outbound access warning: The IPAddressDeny/Allow hardening above restricts networking to localhost only. That’s ideal if you preload models and only serve behind a local reverse proxy.

It also blocks model downloads. If you want the service to pull models directly, comment out those two lines. Then control exposure at the firewall/proxy instead.

Step 5: Enable and start the service

Reload systemd, enable autostart, and start Ollama:

sudo systemctl daemon-reload
sudo systemctl enable --now ollama

Check the service state:

sudo systemctl status ollama --no-pager

If it’s running and bound to localhost, you should see it listening on port 11434:

sudo ss -lntp | grep 11434 || true

Step 6: Test the API from the server

Start with a quick local CLI call. This confirms the daemon can talk to its own API.

ollama list

If you haven’t pulled a model yet, pull something small to validate the full loop. The example below tries a small model first, then falls back:

ollama pull llama3.2:3b || ollama pull llama3:8b || true

Then run a short prompt:

ollama run llama3.2:3b "Give me a 5-line checklist to harden an Ubuntu VPS."

If you enabled the localhost-only networking restriction in the unit (IPAddressDeny), pulls will fail.

Pull models before locking networking down, or remove that restriction.

Step 7: Operate it like production (logs, restarts, upgrades)

With systemd managing the process, you’re no longer guessing what happened after a dropped SSH session. These are the commands you’ll use day-to-day.

View live logs

sudo journalctl -u ollama -f

See logs for the last boot

sudo journalctl -u ollama -b --no-pager

Restart safely after config changes

sudo systemctl restart ollama
sudo systemctl status ollama --no-pager

Confirm it starts after reboot

sudo reboot
# reconnect, then:
sudo systemctl is-enabled ollama
sudo systemctl status ollama --no-pager

For basic server hygiene, pair this with unattended security updates: Set Up Automated Security Updates on Ubuntu VPS.

Step 8: Put a reverse proxy in front (recommended)

Most apps want a stable HTTPS URL and a consistent auth layer. The common pattern looks like this:

• Ollama listens on 127.0.0.1:11434
• Nginx listens on 443 (TLS) and proxies to localhost
• You enforce Basic Auth, IP allowlists, or both

If you already run Nginx on the VPS for your app, add a dedicated server block. For a complete TLS runbook, use: TLS setup guide tutorial (2026): Configure HTTPS with Let’s Encrypt.

Here’s a minimal Nginx snippet (drop into a server block) that proxies to Ollama and enforces Basic Auth. Create a password file first:

sudo apt update
sudo apt install -y apache2-utils
sudo htpasswd -c /etc/nginx/.htpasswd-ollama ollamaadmin

Then in your Nginx site config (commonly /etc/nginx/sites-available/ollama.conf):

location / {
    auth_basic "Ollama";
    auth_basic_user_file /etc/nginx/.htpasswd-ollama;

    proxy_pass http://127.0.0.1:11434;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    # Optional: limit request body if you don’t want huge uploads
    client_max_body_size 25m;
}

Test and reload:

sudo nginx -t
sudo systemctl reload nginx

If you need rate limiting to prevent abuse from chewing through CPU, do it at the proxy. This guide covers patterns that reduce load without breaking legitimate traffic: Nginx Rate Limiting Tutorial (2026): Stop Bots and Brute-Force Traffic.

Step 9: Lock down firewall rules (UFW) for an Ollama VPS

If Ollama binds to 127.0.0.1, don’t open port 11434. Expose only your proxy (usually 80/443).

A typical UFW baseline:

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status verbose

If you want a refresher that avoids the classic “locked myself out” failure mode, follow: Set Up UFW Firewall on Ubuntu for a Web Server.

Step 10: Common failures and quick fixes

These issues show up most often when you move from “run it manually” to a systemd-managed service.

Ollama won’t start: “ExecStart=… no such file or directory”

• Fix the path in ExecStart. Run which ollama and update the unit.
• Reload and restart:

sudo systemctl daemon-reload
sudo systemctl restart ollama

Model pulls fail after hardening

If you set IPAddressDeny=any / IPAddressAllow=localhost, the service can’t reach the internet. That’s expected.

Your options:

• Remove those lines and rely on proxy + firewall for safety, or
• Temporarily comment them out, pull models, then re-enable them.

“Permission denied” writing models

Verify directory ownership and permissions:

sudo ls -ld /var/lib/ollama /var/lib/ollama/models
sudo chown -R ollama:ollama /var/lib/ollama
sudo chmod 0750 /var/lib/ollama /var/lib/ollama/models

High CPU and slow responses under load

• CPU-only inference saturates fast. Keep the endpoint private and authenticated.
• For interactive use, switch to a smaller or more heavily quantized model.
• Apply rate limits at Nginx, and cache at the application layer where it makes sense.

Optional: add a smoke test script for uptime checks

If you run monitoring, use an HTTP check that proves the API responds. Create a basic script:

sudo nano /usr/local/bin/ollama-healthcheck.sh

#!/usr/bin/env bash
set -euo pipefail
curl -fsS http://127.0.0.1:11434/api/tags > /dev/null

Make it executable:

sudo chmod +x /usr/local/bin/ollama-healthcheck.sh

Call that from your monitoring stack. For a solid baseline, see: Server Monitoring Setup Guide Tutorial (2026): Build Alerts for Your VPS or Dedicated Server.

Where to go next (practical extensions)

Once the service is stable, you can add tools without fighting the server:

• Chat UI: Use Open WebUI for team access (then put it behind HTTPS and login).
• Coding assistant: If your use case is “AI for devs,” pair Ollama with Continue.dev: Self-Host Continue.dev With Ollama on a VPS.
• RAG (search over docs): Add pgvector or Qdrant. HostMyCode has a full pgvector setup if your app already uses PostgreSQL: Set Up pgvector for RAG on a PostgreSQL VPS.

FAQ

Should I expose port 11434 to the internet?

Usually no. Bind Ollama to 127.0.0.1 and proxy through Nginx/Caddy with TLS and authentication. If you must expose it, restrict by IP and rate-limit aggressively.

Where does Ollama store models on Ubuntu?

In this tutorial, models live in /var/lib/ollama/models via OLLAMA_MODELS. Keeping it explicit avoids surprises after upgrades or user changes.

How do I change the bind address later?

Edit /etc/systemd/system/ollama.service and change OLLAMA_HOST. Then run sudo systemctl daemon-reload and sudo systemctl restart ollama.

How do I back up my Ollama setup?

Back up /var/lib/ollama (models) and your reverse proxy config (/etc/nginx or /etc/caddy). Models can be large, so decide whether you want to back them up or re-pull them after a rebuild.

Summary

An Ollama systemd service turns “run it in a terminal” into something you can operate. It starts on boot, restarts on failure, and keeps logs in journald.

Keep the API on localhost. Put TLS and auth at the proxy. Default to private access.

If you’re building self-hosted AI features for a site or internal app, a properly sized HostMyCode VPS gives you the control you need for systemd policy, firewall rules, and stable storage.