Self-Host SearXNG With Ollama for AI Search

You can give your team “AI search” without sending queries, IP addresses, or internal URLs to a hosted SaaS. This walkthrough shows how to self-host SearXNG with Ollama on a VPS.

You’ll run both behind Nginx with HTTPS. You’ll also lock the endpoint down so it doesn’t become a public scraping target.

The goal is simple. SearXNG handles metasearch (privacy controls, multiple public engines). Ollama runs local LLM inference for summaries and Q&A using search results as context.

This is private by default. It’s also not magic. It has real constraints.

What you’ll build (architecture + honest limitations)

SearXNG in Docker, reachable only through Nginx.
Ollama in Docker, reachable only on the internal Docker network.
Nginx as the single public entrypoint with Let’s Encrypt TLS.
Access control: Basic Auth (simple and effective) and optional IP allowlisting.

Limitations you should expect:

On CPU-only VPS instances, LLM responses will be slower. That’s expected. For interactive use, pick smaller models.
SearXNG is a metasearch engine. Upstream engines can rate-limit or block you if you run it publicly. Keep it private.
This setup summarizes search results. It’s not a crawler + index like Elasticsearch/Meilisearch unless you build that separately.

Prerequisites (server sizing, domain, firewall)

Recommended baseline (CPU-only):

Ubuntu 24.04 LTS or Debian 12
4 vCPU, 8–16 GB RAM
60+ GB NVMe storage (models + Docker layers add up)

If you want fast responses: use a GPU machine. A dedicated server or GPU VPS is a better fit for larger models and multiple users.

For production deployments, start with a HostMyCode VPS. Move up to dedicated servers if concurrency grows.

You’ll also need:

A domain/subdomain (example: search.example.com) pointing to your server IP
Ports 80/443 open to the internet; SSH locked to your admin IP/VPN

If DNS is still settling, plan the cutover. You don’t want to break certificate issuance during the move.

This guide pairs well with migrating a domain to Cloudflare DNS without downtime.

Step 1: Prepare Ubuntu/Debian and install Docker

SSH into the server as a sudo user.

sudo apt update
sudo apt -y install ca-certificates curl gnupg git ufw apache2-utils

Install Docker using the distro packages (good enough for most VPS builds) or the official repo.

On Ubuntu/Debian, the packaged Docker usually works well for this stack.

sudo apt -y install docker.io docker-compose-plugin
sudo systemctl enable --now docker
sudo usermod -aG docker $USER
newgrp docker

Set a basic firewall policy (adjust SSH rules if you use a custom port):

sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw enable
sudo ufw status

If you want a fuller UFW setup with common web-server gotchas, see Set Up UFW Firewall on Ubuntu for a Web Server.

Step 2: Create directories and a private Docker network

Keep everything under /opt so updates and backups stay tidy.

sudo mkdir -p /opt/searxng-ollama/{searxng,nginx,certbot}
sudo chown -R $USER:$USER /opt/searxng-ollama
cd /opt/searxng-ollama

Create a Docker network so containers can talk to each other. This avoids publishing internal ports to the internet.

docker network create ai-internal

Step 3: Write a docker-compose.yml for SearXNG + Ollama + Nginx

Create /opt/searxng-ollama/docker-compose.yml:

cd /opt/searxng-ollama
cat > docker-compose.yml <<'YAML'
services:
  searxng:
    image: searxng/searxng:latest
    container_name: searxng
    networks:
      - ai-internal
    environment:
      - SEARXNG_BASE_URL=https://search.example.com/
    volumes:
      - ./searxng:/etc/searxng:rw
    restart: unless-stopped

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    networks:
      - ai-internal
    volumes:
      - ollama:/root/.ollama
    restart: unless-stopped

  nginx:
    image: nginx:stable
    container_name: nginx
    networks:
      - ai-internal
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/conf.d:/etc/nginx/conf.d:ro
      - ./nginx/snippets:/etc/nginx/snippets:ro
      - ./certbot/www:/var/www/certbot:ro
      - ./certbot/conf:/etc/letsencrypt:ro
    restart: unless-stopped

  certbot:
    image: certbot/certbot:latest
    container_name: certbot
    volumes:
      - ./certbot/www:/var/www/certbot
      - ./certbot/conf:/etc/letsencrypt
    entrypoint: /bin/sh -c
    command: "trap exit TERM; while :; do certbot renew --webroot -w /var/www/certbot --quiet; sleep 12h & wait $${!}; done"
    restart: unless-stopped

networks:
  ai-internal:
    external: true

volumes:
  ollama:
YAML

Replace search.example.com with your real domain.

Step 4: Configure SearXNG (settings.yml) and enable “AI answers” via Ollama

SearXNG reads /etc/searxng/settings.yml. Start with a minimal, locked-down config.

Keep the instance private.

cd /opt/searxng-ollama
cat > searxng/settings.yml <<'YAML'
use_default_settings: true

server:
  base_url: "https://search.example.com/"
  secret_key: "CHANGE_ME_TO_A_LONG_RANDOM_STRING"

ui:
  static_use_hash: true

search:
  safe_search: 1

outgoing:
  request_timeout: 6.0

# Keep it private: limit engines that tend to block heavy usage.
# You can tune this list later.
engines:
  - name: wikipedia
    engine: wikipedia
    shortcut: w
  - name: duckduckgo
    engine: duckduckgo
    shortcut: ddg
  - name: brave
    engine: brave
    shortcut: br

# Optional: enable results caching (memory). For multi-user deployments,
# use a dedicated cache service; keep it simple for a private instance.

# --- AI integration (Ollama) ---
# This section is intentionally conservative.
# If your SearXNG build uses different keys for LLM plugins, adapt accordingly.
# The proxy approach (Nginx + internal-only Ollama) is still valid.
llm:
  enabled: true
  provider: "ollama"
  ollama:
    base_url: "http://ollama:11434"
    model: "llama3"
    temperature: 0.2
    max_tokens: 512
YAML

Generate a real secret_key (don’t leave the placeholder):

python3 -c "import secrets; print(secrets.token_urlsafe(48))"

Paste the output into searxng/settings.yml.

About the LLM block: SearXNG extensions and config keys can vary by build and plugins.

If your container image doesn’t recognize llm:, don’t get stuck here.

You can still run SearXNG + Ollama and add a UI layer later (or connect a separate front-end). The networking, TLS, and access controls in this guide still apply.

Step 5: Add Nginx TLS reverse proxy + lock it down

Create Nginx directories:

mkdir -p /opt/searxng-ollama/nginx/{conf.d,snippets}

Create a Basic Auth password file (store it on the host, mount it read-only):

sudo mkdir -p /etc/nginx-auth
sudo htpasswd -c /etc/nginx-auth/searxng.htpasswd yourusername
sudo chmod 640 /etc/nginx-auth/searxng.htpasswd
sudo chown root:docker /etc/nginx-auth/searxng.htpasswd || true

If the chown command fails because your distro doesn’t create a docker group, skip that line. Keep the file owned by root.

You’ll mount it read-only into Nginx next.

Update docker-compose.yml to mount the auth file into the Nginx container. Edit the nginx service volumes and add:

- /etc/nginx-auth:/etc/nginx-auth:ro

Create an Nginx snippet for shared security headers: /opt/searxng-ollama/nginx/snippets/security-headers.conf

cat > /opt/searxng-ollama/nginx/snippets/security-headers.conf <<'NGINX'
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer" always;
add_header Permissions-Policy "geolocation=(), microphone=(), camera=()" always;
NGINX

Now create the site config: /opt/searxng-ollama/nginx/conf.d/search.conf

cat > /opt/searxng-ollama/nginx/conf.d/search.conf <<'NGINX'
server {
  listen 80;
  server_name search.example.com;

  location /.well-known/acme-challenge/ {
    root /var/www/certbot;
  }

  location / {
    return 301 https://$host$request_uri;
  }
}

server {
  listen 443 ssl http2;
  server_name search.example.com;

  ssl_certificate /etc/letsencrypt/live/search.example.com/fullchain.pem;
  ssl_certificate_key /etc/letsencrypt/live/search.example.com/privkey.pem;

  include /etc/nginx/snippets/security-headers.conf;

  # Basic access control
  auth_basic "Private search";
  auth_basic_user_file /etc/nginx-auth/searxng.htpasswd;

  # Optional: add an IP allowlist. Uncomment and adjust.
  # allow 203.0.113.10;
  # allow 198.51.100.0/24;
  # deny all;

  client_max_body_size 10m;

  location / {
    proxy_pass http://searxng:8080;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;
  }

  # Do NOT expose Ollama publicly.
  # If you later need it, proxy it behind separate auth and strict allowlist.
}
NGINX

Replace the domain again: search.example.com.

If you want a refresher on HTTPS and the certificate mistakes that waste the most time, HostMyCode has a full 2026 guide: Configure HTTPS with Let’s Encrypt on a VPS.

Step 6: Get a Let’s Encrypt certificate (webroot method)

Start Nginx first so the ACME challenge path is reachable:

cd /opt/searxng-ollama
docker compose up -d nginx
docker logs --tail=50 nginx

Request the certificate using Certbot in a one-off container run (replace email + domain):

docker run --rm \
  -v /opt/searxng-ollama/certbot/www:/var/www/certbot \
  -v /opt/searxng-ollama/certbot/conf:/etc/letsencrypt \
  certbot/certbot certonly \
  --webroot -w /var/www/certbot \
  -d search.example.com \
  --email you@example.com --agree-tos --no-eff-email

Restart Nginx so it picks up the new certificate files:

docker compose restart nginx

Step 7: Start SearXNG and Ollama, then pull a model

Bring up the full stack:

cd /opt/searxng-ollama
docker compose up -d
docker ps

Pull a model inside the Ollama container. On CPU-only servers, smaller models keep latency tolerable.

docker exec -it ollama ollama pull llama3

Model sizing reality check: “Small” still means gigabytes. You’ll also see real RAM use while generating.

If you only have 8 GB RAM and you’re running other services, keep models minimal. Consider swap, but do it carefully.

You can test Ollama locally (from the server):

docker exec -it ollama ollama run llama3 "Summarize the purpose of SearXNG in 2 sentences."

If you want Ollama-specific hardening patterns, this pairs well with running Ollama behind Nginx with Basic Auth and securing the Ollama API with Cloudflare Tunnel.

Step 8: Verify SearXNG behind HTTPS + auth

Open:

https://search.example.com

You should get a Basic Auth prompt. After you sign in, you should land on SearXNG.

Quick health checks from the server:

# Nginx config syntax (inside container)
docker exec -it nginx nginx -t

# SearXNG container logs
docker logs --tail=100 searxng

If you hit a proxy error (common on first run), debug it like any other upstream issue.

Confirm the container name, upstream port, and shared Docker network.

If you want a quick pattern-based walkthrough, this tutorial covers the usual checks: Nginx 502 Bad Gateway Fix on Ubuntu VPS.

Step 9: Wire SearXNG results into AI answers (two practical paths)

There are two realistic ways to get “AI answers” out of this stack:

Native integration if your SearXNG build/plugin supports calling Ollama directly (the llm: block). This keeps everything in one UI.
External front-end that queries SearXNG (as a search tool) and calls Ollama (as the model). This is often easier to customize.

Path A: test the native integration

If your SearXNG UI shows an “Answer” or “Chat” option and it actually uses the configured provider, test with something narrow.

Narrow queries make citations easy to spot.

Search: What is SPF DNS record purpose
Then use the AI answer option, and confirm it cites results rather than freewheeling.

If you don’t see any AI option, don’t waste time clicking around in production. Differences between builds are common. Use Path B.

Path B: use Open WebUI (recommended if you want a “chat” experience)

You can keep SearXNG private and add a chat UI that treats it as a tool. HostMyCode already has a clear walkthrough for the UI layer: Self-Host Open WebUI With Ollama on a VPS.

Deploy it on the same server or a separate VPS. Then point it at your Ollama endpoint (internal networking is best).

Configure SearXNG as the search tool if your chosen UI/plugin supports it.

Step 10: Keep it private (rate limits, bot pressure, and logs)

SearXNG works best as a private tool. The controls below keep you out of trouble fast:

Don’t expose Ollama publicly. In this setup, it has no published ports.
Leave Basic Auth enabled. It stops casual scanning and reduces upstream engine bans.
Add an IP allowlist if your users come from predictable locations (office IPs, VPN egress).
Check logs daily at first. You’ll spot probing quickly.

Useful log commands:

docker logs --since=1h nginx
docker logs --since=1h searxng
docker logs --since=1h ollama

If this VPS runs other services too, turn on automated security updates.

HostMyCode has a step-by-step: Set Up Automated Security Updates on Ubuntu VPS.

Step 11: Performance tuning that actually matters

Most “AI search is slow” complaints come down to two bottlenecks: model size and CPU contention.

Use a smaller model for summaries. Aim for “useful and quick,” not literary output.
Cap output tokens (we set 512). Long answers cost time and CPU.
Keep request timeouts short for upstream engines. A private search box that hangs is dead weight.

If you’re mixing web, database, and AI workloads on one VPS, plan to split them once load becomes steady.

A separate HostMyCode VPS for AI avoids noisy-neighbor problems between your site and model runtime.

Step 12: Backup the pieces that matter

Only two parts of this stack are truly stateful:

Ollama model store (Docker volume ollama)
SearXNG config (/opt/searxng-ollama/searxng/settings.yml)

Back up the SearXNG folder as regular files.

For the Ollama volume, the simplest option is backing up the Docker volume directory (path varies by distro).

Another common approach is to re-pull models during rebuilds and only back up config. That keeps backups small and predictable.

Troubleshooting checklist (the common failure modes)

502 from Nginx: container not healthy, wrong upstream port, or not on the same network. Run docker ps and check logs.
Certbot fails: DNS not pointing to the server, port 80 blocked, or wrong server_name. Verify with dig and curl -I http://search.example.com.
SearXNG loads but engines error: an upstream engine blocked or rate-limited you. Keep it private, trim the engine list, and don’t share the URL publicly.
Ollama is slow: the model is too large for your CPU/RAM. Switch to a smaller model and reduce token output.

Summary: private AI-assisted search that you control

You now have SearXNG serving private search over HTTPS, and Ollama running locally. You can add summaries/answers without shipping queries to a third party.

Keep it behind auth. Keep Ollama off the public internet.

Run it like any other service: patch, monitor, and review logs.

If you want this to hold up under real usage, size the server for sustained CPU/RAM load.

A managed VPS hosting plan is a good fit if you don’t want to babysit updates, TLS renewals, and base OS hardening.

If you’re starting small, a standard HostMyCode VPS is usually enough for a private team deployment.

Private AI search goes smoothly when your server is boring in the best way: consistent CPU, enough RAM, and clean networking. HostMyCode offers VPS hosting for self-managed builds and managed VPS hosting if you want help keeping the stack secure and running.

FAQ

Do I need a GPU to self-host SearXNG with Ollama?

No. A CPU-only VPS works for private use, but responses will be slower and model choice matters. If you need quick answers for multiple users, plan for a GPU server.

Should I expose the Ollama API to the internet?

Usually no. Keep it on an internal network and proxy it only if required, with authentication and an IP allowlist. Treat it like a database port.

Why am I getting blocked by search engines?

If the instance is public or heavily used, upstream engines will rate-limit. Keep it private, reduce the engine list, and avoid sharing the endpoint widely.

How do I add AI chat on top of this?

Deploy a chat UI such as Open WebUI and connect it to Ollama. Keep SearXNG as your search tool endpoint, and enforce the same access controls.