Self-Host Text Embeddings API With Ollama

Embeddings power “search that actually finds things” and most RAG pipelines. If you don’t want your content, queries, or user prompts leaving your network, the simplest option is to self-host embeddings API on infrastructure you control.

This walkthrough shows how to deploy Ollama on a VPS, put it behind HTTPS, and validate it with a small client you can drop into your app.

By the end, you’ll have an HTTPS endpoint that accepts text and returns vectors. You can store those vectors in pgvector or Qdrant.

You’ll also add safety rails—auth, rate limits, and sensible logging. That way you don’t accidentally publish an open embeddings service to the internet.

What you’re building (and where it fits)

Architecture for this tutorial:

Ollama running as a local service on your VPS (CPU-friendly embeddings).
Nginx as a reverse proxy for TLS termination, basic auth, and sane timeouts.
An /api/embeddings HTTPS endpoint you can call from your website/app.

This setup is a good fit for:

site search across docs, blog posts, tickets, product pages
RAG ingestion pipelines (chunk text → embed → store vectors)
internal apps that need semantic similarity without third-party APIs

If you later want a UI, add Open WebUI. For vector storage, pair this with Postgres + pgvector or Qdrant.

Server requirements (honest sizing for embeddings)

Embeddings are usually lighter than chat inference. You can run a reliable embeddings endpoint on CPU.

Size for concurrency, and keep latency expectations realistic.

OS: Ubuntu 22.04/24.04 LTS (commands below assume Ubuntu)
CPU: 2–4 vCPU minimum; 4–8 vCPU if you expect multiple concurrent requests
RAM: 4 GB minimum; 8 GB recommended (model + OS + buffers)
Disk: 30+ GB (model files can be several GB)
GPU: optional for embeddings; start with CPU unless you already have a GPU box

If you need consistently low-latency embeddings at higher throughput, scale up. The same applies if you’re also hosting a chat model.

For production workloads, consider a HostMyCode VPS so you can allocate stable CPU/RAM and keep your models close to your app.

Step 1: Prep the VPS (packages, firewall, and a domain)

SSH into the server as a sudo user.

sudo apt update
sudo apt -y upgrade
sudo apt -y install curl ca-certificates gnupg nginx apache2-utils ufw

Allow SSH and web traffic. If you already use UFW, just confirm these rules exist.

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw --force enable
sudo ufw status

You’ll need DNS pointing a hostname at this VPS, for example:

A record: embeddings.example.com → your VPS public IP

If your domain isn’t set up yet, use HostMyCode domains and point DNS wherever you manage it.

If you use Cloudflare DNS, add the A record there using your normal workflow.

Related reading (DNS): migrate a domain to Cloudflare DNS without downtime.

Step 2: Install Ollama and run it as a service

Install Ollama using the official script. It installs the binary and typically sets up a system service.

curl -fsSL https://ollama.com/install.sh | sh

Verify the service is up:

sudo systemctl status ollama --no-pager

By default, Ollama listens on 127.0.0.1:11434 (local only). Keep it that way.

You’ll expose it through Nginx with HTTPS and auth.

If you want to manage systemd settings explicitly (environment variables, limits, etc.), follow this guide: Ollama systemd service on Ubuntu VPS.

Step 3: Pull an embeddings model in Ollama

Ollama generates embeddings via its embeddings endpoint. You still need an embedding-capable model installed on the server.

Pick one and pull it down.

Example:

ollama pull nomic-embed-text

Confirm it’s installed:

ollama list

Limitation to plan for: model names and availability change over time. If a pull fails, choose another embedding-capable model.

The rest of this setup doesn’t depend on the specific model name.

Step 4: Confirm the local embeddings endpoint works

Before you add TLS, test the API locally on the VPS. Ollama serves HTTP on port 11434.

curl -s http://127.0.0.1:11434/api/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"nomic-embed-text","prompt":"hostmycode affordable reliable hosting"}' | head

You should get JSON with an embedding array (a long list of floats).

If you hit an error, check the basics:

ollama list shows the model
systemctl status ollama is active
you typed the model name exactly as pulled

Step 5: Put Ollama behind Nginx with HTTPS + basic auth

Exposing port 11434 to the internet is a common (and avoidable) mistake. Leave Ollama bound to localhost.

Put Nginx on 443. Then add auth and sane timeouts.

Create credentials

Create a password file for basic auth. You can add more users later.

sudo htpasswd -c /etc/nginx/.htpasswd embeddingsuser

Create an Nginx site

Create a server block for your hostname:

sudo nano /etc/nginx/sites-available/embeddings.example.com

Paste this configuration (replace the hostname):

server {
  listen 80;
  server_name embeddings.example.com;

  location / {
    return 301 https://$host$request_uri;
  }
}

server {
  listen 443 ssl http2;
  server_name embeddings.example.com;

  # TLS certs are added by Certbot in the next step

  # Basic auth gate
  auth_basic "Embeddings API";
  auth_basic_user_file /etc/nginx/.htpasswd;

  # Avoid giant request bodies; adjust if embedding large chunks
  client_max_body_size 2m;

  # Proxy to Ollama on localhost
  location / {
    proxy_pass http://127.0.0.1:11434;
    proxy_http_version 1.1;

    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    # Timeouts: embeddings can take longer on CPU under load
    proxy_connect_timeout 10s;
    proxy_send_timeout 120s;
    proxy_read_timeout 120s;
  }
}

Enable the site and test Nginx:

sudo ln -s /etc/nginx/sites-available/embeddings.example.com /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx

Issue a TLS certificate (Let’s Encrypt)

Install Certbot for Nginx and request a certificate:

sudo apt -y install certbot python3-certbot-nginx
sudo certbot --nginx -d embeddings.example.com

Certbot updates your Nginx config with the ssl_certificate paths and secure defaults.

Confirm renewal is scheduled:

sudo systemctl status certbot.timer --no-pager

If you prefer Caddy for automatic TLS, this tutorial is a good reference: Set Up Let’s Encrypt With Caddy for a VPS Site.

Also relevant (auth patterns): run Ollama behind Nginx with basic auth.

Step 6: Test the HTTPS embeddings endpoint from your laptop

From your local machine (or anywhere with access), hit the endpoint with curl and basic auth:

curl -s https://embeddings.example.com/api/embeddings \
  -u embeddingsuser \
  -H 'Content-Type: application/json' \
  -d '{"model":"nomic-embed-text","prompt":"refund policy for subscription"}' | head

If it returns JSON, you’re set.

If not, these are the usual culprits:

401 Unauthorized: wrong username/password, or missing -u
502 Bad Gateway: Ollama not running, or Nginx can’t reach 127.0.0.1:11434
TLS errors: DNS mismatch, cert not issued, or the hostname points at the wrong IP

For a 502 troubleshooting flow, this is useful even outside WordPress contexts: Nginx 502 Bad Gateway fix on Ubuntu VPS.

Step 7: Add a simple Python test client (drop-in for your app)

This client is intentionally small. It focuses on the checks you’ll actually care about:

calls your HTTPS embeddings endpoint
prints vector length (fast sanity check)
computes cosine similarity between two texts (a realistic “search-ish” operation)

Install Python venv tooling:

sudo apt -y install python3-venv

Create a folder and script on your laptop or a separate app server:

mkdir -p embeddings-client && cd embeddings-client
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install requests

Create client.py:

cat > client.py <<'PY'
import os
import math
import requests
from requests.auth import HTTPBasicAuth

BASE_URL = os.environ.get('EMBEDDINGS_URL', 'https://embeddings.example.com')
USER = os.environ.get('EMBEDDINGS_USER', 'embeddingsuser')
PASS = os.environ.get('EMBEDDINGS_PASS', '')
MODEL = os.environ.get('EMBEDDINGS_MODEL', 'nomic-embed-text')


def embed(text: str):
    r = requests.post(
        f"{BASE_URL}/api/embeddings",
        json={"model": MODEL, "prompt": text},
        auth=HTTPBasicAuth(USER, PASS),
        timeout=120,
    )
    r.raise_for_status()
    data = r.json()
    return data["embedding"]


def cosine(a, b):
    dot = sum(x*y for x, y in zip(a, b))
    na = math.sqrt(sum(x*x for x in a))
    nb = math.sqrt(sum(y*y for y in b))
    return dot / (na * nb)


if __name__ == "__main__":
    a = "reset my password"
    b = "I forgot my login and need to change my password"

    va = embed(a)
    vb = embed(b)

    print(f"Vector length: {len(va)}")
    print(f"Cosine similarity: {cosine(va, vb):.4f}")
PY

Run it (set your password in an env var so you don’t drop it into shell history):

export EMBEDDINGS_URL='https://embeddings.example.com'
export EMBEDDINGS_USER='embeddingsuser'
export EMBEDDINGS_PASS='YOUR_PASSWORD'
python client.py

If you see timeouts under load, bump Nginx proxy_read_timeout. Also watch CPU saturation on the VPS.

CPU embeddings queue quickly when requests pile up.

Step 8: Operational limits you should set now (before production)

Even if you only use embeddings internally, treat this like a real API. One misconfiguration can hand out free compute to the internet.

1) Keep the service private if you can

Basic auth is a gate, not a full security model. If only your app servers need embeddings, tighten access with one of these options:

restrict access by IP (office/VPN/app subnet) in Nginx
place the endpoint on a private network and call it internally
use a tunnel-based approach if you can’t open 443 publicly

If you want a tunnel approach for Ollama-style APIs, this guide shows the pattern: secure an Ollama API with Cloudflare Tunnel.

2) Add simple rate limiting

Embedding requests cost CPU. Without limits, a handful of abusive clients can pin your box.

Edit /etc/nginx/nginx.conf and add a limit zone inside the http {} block:

limit_req_zone $binary_remote_addr zone=embeddings_rl:10m rate=5r/s;

Then add this inside the location / block for your embeddings site:

limit_req zone=embeddings_rl burst=20 nodelay;

Reload Nginx:

sudo nginx -t
sudo systemctl reload nginx

3) Log enough to debug, not enough to leak data

Be careful with logging around embedding endpoints. Nginx won’t log the basic auth password.

Request bodies can still include sensitive text.

Avoid custom logging that captures bodies. Stick to default access logs unless you have a specific reason to change them. Keep the VPS patched.

For ongoing patching on Ubuntu VPS, this is a solid baseline: set up automated security updates on Ubuntu.

Step 9: Using embeddings for RAG search (what’s next)

After you can generate vectors, the next step is storing and querying them.

Two common paths:

PostgreSQL + pgvector: straightforward and dependable, especially if you already run Postgres
Qdrant: a dedicated vector DB with strong filtering and performance patterns

If you want Postgres-based RAG, follow: set up pgvector for RAG on a PostgreSQL VPS. If you prefer Qdrant, start here: run Qdrant on a VPS with Docker and Nginx TLS.

Hosting details matter once your index grows. Vector search becomes steady, read-heavy IO.

NVMe-backed VPS plans show their value once you’re indexing tens or hundreds of thousands of chunks.

Quick diagnostics checklist (common breakpoints)

Ollama API reachable locally?
curl -s http://127.0.0.1:11434/api/tags | head
Nginx can reach Ollama?
Check: sudo tail -n 80 /var/log/nginx/error.log
TLS good?
curl -I https://embeddings.example.com should return 401 (because of basic auth), not a TLS error.
CPU pegged during embeds?
top or htop; consider scaling vCPU/RAM or batching requests.
Requests hanging?
Increase Nginx timeouts and verify your client timeout is > your worst-case embed latency.

Summary: a production-friendly embeddings endpoint you control

You now have a working endpoint on infrastructure you own. Ollama stays on localhost.

Nginx listens on 443 with HTTPS and basic auth. You also have a client you can wire into ingestion or search code.

Keep it private where possible, rate limit it, and watch CPU as traffic grows.

If you want this to run without constant maintenance, choose a VPS sized for steady compute and storage.

A managed VPS hosting plan makes sense if you’d rather avoid patching and firewall chores. If you prefer hands-on control and predictable costs, start with a HostMyCode VPS and scale up as embedding traffic increases.

If you’re building RAG search or an internal knowledge assistant, keep embeddings close to your data and app servers. HostMyCode makes it easy to run Ollama and secure it behind HTTPS on a HostMyCode VPS, or you can offload routine maintenance with managed VPS hosting.

FAQ

Should I run embeddings on CPU or GPU?

Start with CPU for embeddings unless you already operate a GPU box. CPU embeddings handle typical site search ingestion fine. Move to GPU if you need high throughput or you’re also hosting a chat model.

Can I expose this endpoint to the public internet?

You can, but treat it like any paid API. Use HTTPS, strong auth, rate limiting, and preferably IP allowlists. Many operators keep embeddings private and only call it from app servers.

What chunk size should I embed for RAG?

For website/docs RAG, start with 300–800 tokens per chunk with small overlap. The best size depends on your content structure and how you retrieve (top-k, hybrid search, filters).

How do I store vectors after generating embeddings?

Use pgvector if you already run PostgreSQL, or Qdrant if you want a dedicated vector database. Both work well on a VPS; choose based on your filtering and operational preferences.

Why am I getting 502 from Nginx?

Most of the time Ollama isn’t running, is bound to a different address, or Nginx can’t reach 127.0.0.1:11434. Check systemctl status ollama and the Nginx error log.