How to Install Khoj on Ubuntu 24.04 — Your Self-Hosted AI Knowledge Assistant

Most AI assistants can tell you about the world, but they know nothing about you — your notes, your journal entries, your research PDFs, the meeting minutes you took last March. Khoj flips that equation. It is a self-hosted AI assistant that sits on top of your personal knowledge base and answers questions using your own content as context. Instead of hallucinating generic answers, it grounds every response in the documents you have actually written or collected.

This guide walks you through installing Khoj on an Ubuntu 24.04 VPS using Docker Compose, connecting it to a local Ollama instance for zero-cost inference, and exposing it safely over HTTPS so you can reach it from your desktop, phone, and Obsidian vault.

Prefer a one-click deploy? Spin up Khoj + Ollama with our pre-configured AI Knowledge Server image. Launch an AI-ready VPS now and be chatting with your notes in under ten minutes.

Table of Contents

What is Khoj?

Khoj is an open-source AI application that gives you a ChatGPT-style interface on top of your personal data. You point it at your notes, documents, emails, and web pages, and it builds a semantic index using embeddings. When you ask a question, Khoj retrieves the most relevant chunks from your knowledge base and feeds them to a large language model along with your prompt. The model then produces an answer that cites the underlying sources — a pattern commonly called retrieval-augmented generation, or RAG.

The difference between Khoj and a generic RAG toolkit is polish. Khoj ships a complete product: a web UI, desktop apps for macOS/Windows/Linux, an Obsidian plugin, Emacs integration, Android and iOS mobile apps, and a clean HTTP API. The backend is a FastAPI Python service backed by PostgreSQL with the pgvector extension for vector search. All of this runs in a handful of Docker containers on a modest VPS.

Khoj was originally built as a personal search engine for Org-mode and Markdown notes. It has since grown into a fully featured agent that can chat about your content, run web searches, generate images, transcribe voice input, and execute Python for data analysis. You can run it with a managed cloud LLM (OpenAI, Anthropic, Google Gemini) or fully offline against a local Ollama server. The choice is yours on a per-agent basis.

Sync Sources and Supported Content

Khoj indexes a wide range of content types out of the box. On the document side it handles Markdown (.md), Org-mode (.org), plain text, PDFs, Microsoft Word (.docx), and images with OCR. It has first-class integrations with Obsidian vaults, Notion workspaces, GitHub repositories, and any folder synced via the desktop client. Emails can be indexed by exporting them as .eml or .mbox files and dropping them into a watched directory.

Once content is synced, Khoj chunks each document, generates embeddings using a sentence-transformer model (the default is thenlper/gte-small, which runs on CPU), and stores them in PostgreSQL/pgvector. A background worker keeps the index in sync whenever files change. The whole pipeline is incremental, so adding a single note does not re-index your entire vault.

Features at a Glance

• Chat over your data. Ask natural-language questions and get cited answers drawn from your notes.
• Web search. Khoj can reach out to the live web (via Serper, Jina, or a self-hosted SearXNG) when your personal data is not enough.
• Voice input and output. Built-in speech-to-text (Whisper) and text-to-speech for hands-free chat.
• Image generation. Generate images with Stable Diffusion, DALL-E, or a local ComfyUI endpoint.
• Agents. Create specialized personas with their own system prompts, tool access, and scoped knowledge bases.
• Automations. Scheduled prompts that deliver results by email (for example, a daily research digest).
• Multi-user. One Khoj instance can serve a whole team or family with isolated knowledge bases.

Why Self-Host Khoj?

Self-hosting Khoj on your own VPS is the only way to keep sensitive notes — journals, medical records, client documents, business strategy — out of third-party AI vendor logs. The public Khoj Cloud is convenient, but every query and document passes through someone else's infrastructure. On your own server, you decide which LLM sees your data. Pair Khoj with a local Ollama instance and the entire pipeline stays inside your VPS; nothing ever leaves the network.

Cost is the second driver. Cloud LLM APIs charge per token, and a single heavy RAG session can burn through dollars of tokens because each query stuffs retrieved context into the prompt. A self-hosted Khoj + Ollama stack on a CloudCore Professional VPS has a fixed monthly cost regardless of usage, and you can index millions of tokens of personal content without worrying about the meter.

The third reason is capability. Cloud Khoj imposes rate limits, content filters, and model caps. On your own box you choose the embedding model, the LLM, the context window size, and the number of indexed documents. There is no ceiling other than your hardware.

Prerequisites

Before you start, confirm you have:

• An Ubuntu 24.04 LTS VPS with at least 4 vCPU, 8 GB RAM, and 50 GB disk. A CloudCore Professional plan is a comfortable fit.
• A non-root user with sudo privileges.
• A domain name (for example khoj.example.com) with an A record pointing to your VPS IP. This is required for Step 11.
• Basic familiarity with the Linux command line and SSH.
• Optional but recommended: an existing Ollama server (see our Ollama install guide) for zero-cost local inference.

Step 1: Prepare the VPS

SSH into your server and start with a fresh package index and any pending security updates:

ssh admin@your-vps-ip
sudo apt update && sudo apt upgrade -y
sudo apt install -y curl ca-certificates gnupg ufw fail2ban

Enable the firewall and open only the ports you will actually use. SSH, HTTP, and HTTPS are enough; the Khoj container should never be exposed directly on a public port.

sudo ufw allow OpenSSH
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp
sudo ufw --force enable

Step 2: Install Docker and Docker Compose

Khoj is distributed as a set of official Docker images. Install Docker Engine from the upstream repository rather than the older Ubuntu package so you get the current Compose v2 plugin.

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker $USER
newgrp docker
docker --version
docker compose version

Step 3: Create the Khoj Directory Structure

Give Khoj a dedicated directory under /opt so persistent data is easy to back up.

sudo mkdir -p /opt/khoj/{data,models,config}
sudo chown -R $USER:$USER /opt/khoj
cd /opt/khoj

The data folder holds PostgreSQL state, the models folder caches embedding models downloaded on first run, and config keeps your .env and any custom YAML.

Step 4: Write the docker-compose.yml

Create /opt/khoj/docker-compose.yml with the following content. This stack includes Khoj itself and a dedicated PostgreSQL 16 + pgvector database.

services:
  database:
    image: pgvector/pgvector:pg16
    container_name: khoj-db
    restart: unless-stopped
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 10s
      timeout: 5s
      retries: 5
  server:
    image: ghcr.io/khoj-ai/khoj:latest
    container_name: khoj-server
    restart: unless-stopped
    depends_on:
      database:
        condition: service_healthy
    env_file:
      - .env
    ports:
      - "127.0.0.1:42110:42110"
    volumes:
      - ./models:/root/.khoj/search
      - ./data/khoj:/root/.khoj
      - ./config:/config
    command: >
      sh -c "python -m khoj --host 0.0.0.0 --port 42110 --anonymous-mode"
volumes:
  postgres-data:
  khoj-data:

Two details worth calling out. The server port 42110 is bound to 127.0.0.1 only, which means it is never reachable from the public internet — Nginx will proxy to it in Step 11. The --anonymous-mode flag is appropriate for a single-user setup; remove it if you plan to enable multi-user accounts with email/password or Google OAuth.

Step 5: Configure the .env File

Create /opt/khoj/.env with your secrets and API keys. Replace the placeholder values before saving.

# PostgreSQL
POSTGRES_USER=khoj
POSTGRES_PASSWORD=change-me-to-a-long-random-string
POSTGRES_DB=khoj
Khoj database connection
KHOJ_DJANGO_SECRET_KEY=generate-with-openssl-rand-hex-32
POSTGRES_HOST=database
POSTGRES_PORT=5432
Admin account (created on first launch)
[email protected]
KHOJ_ADMIN_PASSWORD=another-strong-password
LLM providers — uncomment whichever you want enabled
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GEMINI_API_KEY=...
Local Ollama (recommended for privacy + zero cost)
OPENAI_BASE_URL=http://host.docker.internal:11434/v1
OPENAI_API_KEY=ollama
Optional web search backend
SERPER_DEV_API_KEY=...
JINA_API_KEY=...

Generate the Django secret key with openssl rand -hex 32. Treat this file like any other secret — chmod 600 .env and never commit it to git.

Khoj supports four LLM providers out of the box: OpenAI, Anthropic (Claude), Google Gemini, and any OpenAI-compatible endpoint, which is how we will talk to Ollama. You can enable more than one and pick between them per-agent in the web UI.

Step 6: Connect Khoj to Local Ollama

If you already followed our Ollama install guide, you have an Ollama instance listening on 127.0.0.1:11434. To let the Khoj container reach the host's Ollama, add the extra_hosts directive to the server service in docker-compose.yml:

server:
    # ...existing config...
    extra_hosts:
      - "host.docker.internal:host-gateway"

Pull a chat model and an embedding model into Ollama:

ollama pull llama3.1:8b
ollama pull nomic-embed-text

Then, after your first Khoj launch, open the admin panel at https://khoj.example.com/server/admin/ and add a new Chat Model Option:

• Name: llama3.1:8b
• Model Type: openai
• Tokenizer: gpt-3.5-turbo (close enough for token counting)
• OpenAI Config: create a new config named Ollama, base URL http://host.docker.internal:11434/v1, api key ollama.

Step 7: Launch and Verify

With everything in place, bring the stack up:

cd /opt/khoj
docker compose pull
docker compose up -d
docker compose logs -f server

The first boot takes two to three minutes because Khoj downloads the default embedding model (~150 MB) and runs database migrations. When the log stream settles on Application startup complete, open http://127.0.0.1:42110 on the VPS (use curl or an SSH tunnel) to confirm the service is healthy:

curl -I http://127.0.0.1:42110/
HTTP/1.1 200 OK

If you see a 200 response, the backend is up. The web UI is not yet reachable from outside the VPS — we fix that in Step 11.

Step 8: Index Your Content

Khoj needs something to search before it can be useful. There are three ways to load content.

Upload via the web UI. Once Nginx is configured in Step 11, visit https://khoj.example.com, log in, and drag Markdown, PDF, Org, or DOCX files into the Documents tab. This is the easiest path for one-off collections.

Sync from the desktop client. The Khoj desktop app (installed in Step 9) watches a folder on your laptop and pushes any changes to the server. Point it at your entire Documents/Notes directory and every file gets indexed automatically.

Sync a GitHub repository. Under Settings → Content Sources → GitHub, paste a personal access token and the repo slug (for example yourname/notes). Khoj will clone the repo, index every Markdown and Org file, and re-pull on a schedule you configure.

Indexing progress is visible in the Documents tab. A vault of 5,000 notes and 200 PDFs typically completes in ten to fifteen minutes on a 4 vCPU VPS.

Step 9: Install the Desktop and Obsidian Clients

The desktop client turns a local folder into a live knowledge source and gives you a global hotkey for instant chat.

Download the appropriate build from khoj.dev/downloads. During setup, pick Self-Hosted and enter your server URL (https://khoj.example.com) along with the API token you generate under Settings → API Keys. Add one or more folders under Files; the client will sync them on change.

For Obsidian users, the experience is even tighter. Open Obsidian, go to Settings → Community plugins → Browse, search for Khoj, and install it. In the plugin settings, set the Khoj URL to your server and paste the same API token. You will now get:

• A chat pane that answers questions about your active vault.
• Inline semantic search with Ctrl+Alt+S.
• [[wiki-link]] citations in chat responses that open the source note in Obsidian when clicked.

Step 10: Mobile Apps and Remote Access

The official Khoj iOS and Android apps connect to any Khoj server. On first launch, tap the server icon and enter your HTTPS URL and API token. From that point on, you have a privacy-respecting AI assistant on your phone that talks to your own notes, not OpenAI's servers.

Because Khoj is bound to 127.0.0.1 on the VPS, remote access depends entirely on the reverse proxy we set up next. If you prefer to avoid a public domain, you can skip Step 11 and reach Khoj via a Tailscale or WireGuard tunnel — the mobile apps accept any URL, including http://100.x.x.x:42110 over the VPN.

Step 11: Put Khoj Behind Nginx with SSL

Install Nginx and Certbot, then request a Let's Encrypt certificate for your subdomain.

sudo apt install -y nginx certbot python3-certbot-nginx

Create /etc/nginx/sites-available/khoj.conf:

server {
    listen 80;
    server_name khoj.example.com;
    location / {
        proxy_pass http://127.0.0.1:42110;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_read_timeout 600s;
        proxy_send_timeout 600s;
        client_max_body_size 100M;
    }
}

The long read and send timeouts matter because Khoj can stream LLM responses that run for minutes on large local models. The 100 MB body size lets you upload big PDFs through the web UI.

Enable the site and issue the certificate:

sudo ln -s /etc/nginx/sites-available/khoj.conf /etc/nginx/sites-enabled/
sudo nginx -t
sudo systemctl reload nginx
sudo certbot --nginx -d khoj.example.com

Certbot rewrites the config to add HTTPS, redirects HTTP to HTTPS, and installs a renewal timer. Visit https://khoj.example.com and complete the first-run onboarding.

Backups

Khoj's state lives in three places: the PostgreSQL database (indexed embeddings, chat history, user accounts), the /opt/khoj/data/khoj directory (raw synced files and logs), and your .env. A daily cron job that dumps Postgres and archives the data folder covers everything:

sudo tee /etc/cron.daily/khoj-backup > /dev/null <<'EOF'
#!/bin/bash
set -e
BACKUP_DIR=/opt/khoj/backups
STAMP=$(date +%Y%m%d)
mkdir -p $BACKUP_DIR
docker exec khoj-db pg_dump -U khoj khoj | gzip > $BACKUP_DIR/khoj-db-$STAMP.sql.gz
tar czf $BACKUP_DIR/khoj-data-$STAMP.tar.gz -C /opt/khoj data/khoj config .env
find $BACKUP_DIR -type f -mtime +14 -delete
EOF
sudo chmod +x /etc/cron.daily/khoj-backup

Ship the backup directory offsite with rclone, restic, or your preferred tool.

Updating Khoj

Khoj ships frequent releases. To update, pull the latest image and recreate the container:

cd /opt/khoj
docker compose pull
docker compose up -d
docker compose logs -f server

Database migrations run automatically on boot. If you want to pin to a specific version rather than latest, change the image tag to ghcr.io/khoj-ai/khoj:1.x.y and bump it deliberately.

Troubleshooting

The server container restarts in a loop. Run docker compose logs server and look for the first error. The most common cause is an unreachable database — confirm POSTGRES_PASSWORD matches between the database and server services and that the khoj-db container reports ready to accept connections.

Chat responses say "No chat model configured". Log into /server/admin/, open Chat Model Options, and add at least one entry. If you are using Ollama, make sure extra_hosts: host.docker.internal:host-gateway is present in the compose file and that curl http://127.0.0.1:11434/api/tags works on the host.

Embeddings take forever on first sync. The default gte-small model runs on CPU and processes roughly 50 documents per minute on 4 vCPU. For large vaults, switch to all-MiniLM-L6-v2 (faster but slightly lower quality) or run the Khoj container on a box with a GPU.

Obsidian plugin can't connect. Verify the URL includes the scheme (https://) and that the API token was copied without trailing whitespace. Check the browser network tab for CORS errors — if you see any, confirm Nginx is forwarding the Origin header.

502 Bad Gateway from Nginx. The Khoj container probably crashed or is still booting. docker compose ps should show healthy; if not, tail the logs.

FAQ

Is Khoj really free? The self-hosted version is fully open source under the AGPL-3.0 license. You pay only for your VPS and, if you choose a cloud LLM, per-token API fees.

Can it work entirely offline? Yes. Use a local embedding model (the default gte-small is local) and a local chat model via Ollama. No outbound calls are made unless you explicitly enable web search or a cloud LLM provider.

How much RAM does it need? Khoj itself is modest — about 1.5 GB for the server plus the Postgres footprint. The real cost is the LLM you point it at. Llama 3.1 8B in Ollama needs roughly 6 GB of RAM or VRAM; smaller models like Phi-3-mini run in 2 GB.

Does it support multiple users? Yes. Disable --anonymous-mode, enable email or Google OAuth in the admin panel, and each user gets a scoped knowledge base.

Can I use Khoj with Claude or Gemini? Yes — set ANTHROPIC_API_KEY or GEMINI_API_KEY in .env, then add a Chat Model Option in the admin panel pointing at claude-3-5-sonnet-latest or gemini-1.5-pro.

How is this different from LibreChat or OpenWebUI? Those are primarily chat front-ends for LLMs. Khoj is a RAG platform: the chat UI is only one feature on top of a semantic index of your personal documents, with first-party sync to Obsidian, Notion, and GitHub.

Next Steps

Khoj is strongest when it is fed constantly. Set up automatic sync from your Obsidian vault, point it at your Notion workspace, and drop research PDFs into the watched folder as you collect them. Within a few weeks you will have a private AI assistant that knows your writing, your reading list, and your ongoing projects better than any cloud product ever could.

From here, consider:

• Layering Ollama for fully local inference if you have not already.
• Pairing Khoj with Gemma 3 on Ubuntu for a lightweight chat model that fits in 4 GB of RAM.
• Upgrading to a GPU-backed CloudCore plan for faster embeddings and larger local models.