How to Install AnythingLLM on Ubuntu 24.04 — Private Document RAG for Teams

AnythingLLM turns a folder of messy PDFs, Word documents, websites, and transcripts into a searchable AI assistant that your whole team can query in natural language. It is one of the most complete open-source retrieval-augmented generation (RAG) applications you can self-host: multi-user authentication, per-workspace document isolation, embeddings, vector storage, citation-aware responses, and a REST API are all bundled into a single Docker image. This guide walks through a production deployment on Ubuntu 24.04 — from Docker install to a fully TLS-protected reverse proxy — with AnythingLLM wired up to a local Ollama instance for fully private inference.

Table of Contents

What is AnythingLLM?

AnythingLLM is a full-stack, open-source AI application developed by Mintplex Labs that turns any collection of documents into a queryable, citation-aware assistant. Under the hood it bundles a React frontend, a Node.js API server, a document parsing and chunking pipeline, an embeddings worker, and a vector store — all behind a single Docker image (mintplexlabs/anythingllm). Users interact with it through a polished chat interface that supports threads, file uploads, agents, and per-workspace prompt customization.

What sets AnythingLLM apart from bare LLM chat UIs is its document-grounded RAG pipeline. Upload a PDF, a Word file, a CSV, a website URL, a YouTube transcript, or a Confluence export; AnythingLLM extracts the text, splits it into semantically sensible chunks, generates embeddings with the embedder of your choice, and stores the vectors in a vector database. Every chat message is answered against that corpus, with inline citations pointing back to the source document. The result is a private, auditable alternative to tools like NotebookLM or ChatGPT's "Custom GPTs" — but one where every byte of data stays on your hardware.

Teams use AnythingLLM for internal knowledge bases, legal and contract review, customer-support deflection, onboarding copilots, technical documentation search, and domain-specific agents over proprietary corpora. Because it supports dozens of LLM providers (OpenAI, Anthropic, Azure OpenAI, AWS Bedrock, Ollama, LM Studio, LocalAI, Together, Groq) and multiple vector stores (LanceDB, Qdrant, Weaviate, Chroma, Milvus, Pinecone), you can mix and match an architecture that matches your budget, compliance, and performance targets.

Why Self-Host AnythingLLM?

Hosted RAG products — NotebookLM, ChatGPT Enterprise, Claude Projects — are excellent, but they all require that your documents leave your network. For any business that handles client contracts, medical files, financial data, proprietary source code, or EU-personal data under GDPR, that single fact is often a hard blocker. Self-hosting AnythingLLM gives you:

• End-to-end data residency — documents, embeddings, chat logs, and model inference can all live on the same VPS. Nothing leaves the box unless you explicitly configure an external LLM.
• Flat, predictable cost — an EUR 19.99/month VPS handles an entire small team. Compare with per-seat SaaS pricing at USD 25-30/user/month.
• Zero vendor lock-in — swap LLMs, embedders, and vector stores via dropdown. No data migration required.
• Per-workspace isolation — different teams, clients, or projects get separate document corpora, prompts, and chat histories in the same instance.
• Full audit trail — every chat, citation, and document upload is stored in a SQLite database you own.
• API-first — the developer API lets you embed AnythingLLM chat into your own apps, Slack bots, or internal tooling.
• Composable with other open tools — pair with Ollama for inference, Open WebUI for a second chat front-end, Dify for agent workflows, LibreChat for multi-provider chat, and Qdrant or Weaviate for production-grade vector storage.

Recommended Plan: CloudCore Professional

AnythingLLM itself is lightweight, but RAG gets heavy when you stack embedder, vector store, and a local LLM on the same box. For a small team (5-15 users, a few thousand documents, 7B-9B parameter local model), we recommend the CloudCore Professional plan:

• 6 vCPU cores
• 12 GB RAM
• 100 GB NVMe SSD
• Unmetered bandwidth
• EUR 19.99/month

Prerequisites

Before you begin, make sure you have:

• A VPS running Ubuntu 24.04 LTS with root or sudo access
• SSH access to your server
• A domain or subdomain pointed at the server's public IP (for TLS) — for example chat.yourdomain.com
• At least 4 GB of RAM (12 GB+ recommended if you will run Ollama on the same host)
• 20 GB of free disk space minimum (more if you plan to ingest large corpora)
• Ollama installed on the same host if you want fully private inference. If you have not installed it yet, follow our Ollama on Ubuntu 24.04 guide first.

ssh root@your-server-ip

Step 1: Prepare the Ubuntu Host

Update the package index and upgrade installed packages to make sure you have current security patches and the Docker install does not trip over stale dependencies.

sudo apt update && sudo apt upgrade -y

Install a few utilities we will use throughout the guide:

sudo apt install -y curl ca-certificates gnupg lsb-release ufw

If the kernel was updated, reboot before continuing:

sudo reboot

Reconnect via SSH after a minute.

Step 2: Install Docker Engine

AnythingLLM ships as a Docker image, so we install Docker Engine (not Docker Desktop) on the host. Using Docker's official apt repository gives us newer releases than Ubuntu's default packages.

Add Docker's GPG key and repository:

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker Engine and the Compose plugin:

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io \
  docker-buildx-plugin docker-compose-plugin

Verify the install:

sudo docker run --rm hello-world

Expected (abbreviated) output:

Hello from Docker!
This message shows that your installation appears to be working correctly.

Optionally add your non-root user to the docker group so you can skip sudo on Docker commands:

sudo usermod -aG docker $USER
newgrp docker

Step 3: Create the Persistent Storage Directory

AnythingLLM stores its SQLite database, uploaded documents, parsed text, embeddings metadata, and (by default) its LanceDB vector store inside /app/server/storage in the container. We bind-mount a host directory into that path so all state survives container restarts and upgrades.

Create the storage directory and the env file location:

sudo mkdir -p /opt/anythingllm/storage
sudo touch /opt/anythingllm/.env

The container runs as UID 1000 (the anythingllm user baked into the image). Give that UID ownership so it can read and write the volume:

sudo chown -R 1000:1000 /opt/anythingllm
sudo chmod 600 /opt/anythingllm/.env

Production tip: if you are deploying on a VPS with block storage attached, mount the volume at /opt/anythingllm before running the container. Documents and embeddings grow over time — plan for 2-5 GB per 10k pages of ingested content.

Step 4: Configure the .env File

AnythingLLM reads its configuration from /app/server/.env inside the container. We pre-seed a minimal set of variables via the host-side env file so the container boots with sane defaults. Everything else can be changed from the web UI afterwards.

Open the env file:

sudo nano /opt/anythingllm/.env

Paste the following, changing SERVER_URL, JWT_SECRET, SIG_KEY, and SIG_SALT to your own values:

# Server
SERVER_URL="https://chat.yourdomain.com"
STORAGE_DIR="/app/server/storage"
Security — generate with: openssl rand -hex 32
JWT_SECRET="replace-with-a-long-random-string-at-least-32-chars"
SIG_KEY="replace-with-another-32-char-random-string-for-signing"
SIG_SALT="replace-with-a-16-char-random-salt"
LLM provider — we wire this to local Ollama
LLM_PROVIDER="ollama"
OLLAMA_BASE_PATH="http://host.docker.internal:11434"
OLLAMA_MODEL_PREF="llama3.1"
OLLAMA_MODEL_TOKEN_LIMIT="4096"
Embedder — use Ollama's nomic-embed-text for a fully local pipeline
EMBEDDING_ENGINE="ollama"
EMBEDDING_BASE_PATH="http://host.docker.internal:11434"
EMBEDDING_MODEL_PREF="nomic-embed-text"
EMBEDDING_MODEL_MAX_CHUNK_LENGTH="8192"
Vector database — LanceDB is built-in and requires no external service
VECTOR_DB="lancedb"
Disable telemetry (optional)
DISABLE_TELEMETRY="true"

Generate strong secrets quickly:

openssl rand -hex 32   # use for JWT_SECRET and SIG_KEY
openssl rand -hex 16   # use for SIG_SALT

Before running the container, make sure the embedding model is pulled in Ollama:

ollama pull nomic-embed-text
ollama pull llama3.1

Step 5: Run the AnythingLLM Container

Pull the latest image and launch it with the bind-mount volume, env file, host networking helper, and port mapping:

docker pull mintplexlabs/anythingllm:latest

docker run -d \
  --name anythingllm \
  --restart unless-stopped \
  --cap-add SYS_ADMIN \
  --add-host=host.docker.internal:host-gateway \
  -p 127.0.0.1:3001:3001 \
  -v /opt/anythingllm/storage:/app/server/storage \
  -v /opt/anythingllm/.env:/app/server/.env \
  -e STORAGE_DIR="/app/server/storage" \
  mintplexlabs/anythingllm:latest

Key flags explained:

• --cap-add SYS_ADMIN — required for the Chromium-based headless browser AnythingLLM uses when scraping URLs into workspaces.
• --add-host=host.docker.internal:host-gateway — lets the container reach the host's Ollama at http://host.docker.internal:11434. On Linux this is not automatic like it is on Docker Desktop.
• -p 127.0.0.1:3001:3001 — binds only to localhost. Nginx will handle public TLS traffic in Step 12. Never expose port 3001 directly to the internet.
• -v /opt/anythingllm/storage:/app/server/storage — persists all state.
• -v /opt/anythingllm/.env:/app/server/.env — supplies configuration.

docker ps --filter name=anythingllm
docker logs -f anythingllm

Expected (abbreviated) log output:

[backend] Primary server in HTTP mode listening on port 3001
[COLLECTOR] Document processor app listening on port 8888
[MIGRATIONS] Prisma migrations successfully applied

Press Ctrl+C to stop following the logs.

Confirm the API is answering locally:

curl http://127.0.0.1:3001/api/ping

Expected output:

{"online":true}

Step 6: Complete the First-Run Wizard

At this point AnythingLLM is running, but it is not reachable from your browser yet (we only bound to localhost). You have two options:

Option A — SSH tunnel (fastest for initial setup): from your laptop, open an SSH tunnel that forwards the remote port 3001 to your local 3001:

ssh -L 3001:127.0.0.1:3001 root@your-server-ip

Then open http://localhost:3001 in your browser.

Option B — skip ahead to Step 12 and complete the Nginx/TLS setup first, then return here using https://chat.yourdomain.com.

When the UI loads, AnythingLLM walks you through a short onboarding:

You now have a working RAG application. The next steps fine-tune the configuration.

Step 7: Connect AnythingLLM to Ollama

If you skipped the wizard defaults or want to change providers later, navigate to Settings → LLM Preference in the sidebar.

Under LLM Provider choose Ollama. Fill in:

• Ollama Base URL: http://host.docker.internal:11434
• Chat Model Selection: pick any model you have pulled — llama3.1, mistral, gemma2:9b, qwen2.5, etc.
• Token context window: 4096 is safe for most 7B-9B models. Bump to 8192 if your model supports it and you have headroom.
• Max output tokens: 1024 is a reasonable default; raise it for long-form generation.

Alternative LLM providers

AnythingLLM supports many hosted LLMs out of the box. Switch the provider to:

• OpenAI — paste your API key and choose gpt-4o-mini or gpt-4o for a high-quality hosted experience.
• Anthropic — Claude Haiku/Sonnet/Opus.
• Groq — ultra-fast Llama/Mixtral inference at cents per million tokens.
• Together AI, Fireworks, OpenRouter — broad open-model catalogs.
• Azure OpenAI, AWS Bedrock — enterprise compliance scenarios.

Step 8: Configure the Embedder

The embedder turns your document chunks into vectors. It does not have to match your LLM provider — many teams use a small local embedder (free, fast) paired with a hosted chat model (high quality).

Open Settings → Embedder Preference. For a fully private pipeline, select Ollama and nomic-embed-text (768 dimensions, 8192-token context window, strong general-purpose quality).

Alternative embedders worth considering:

• OpenAI text-embedding-3-small — cheap (USD 0.02 / 1M tokens), 1536 dimensions, very high quality. Good choice when data sensitivity allows.
• OpenAI text-embedding-3-large — 3072 dimensions, best retrieval quality for complex domains.
• Cohere Embed v3 — strong multilingual support.
• Azure OpenAI — same models as OpenAI with enterprise SLAs.
• LocalAI / LM Studio — any local embedding model exposed over an OpenAI-compatible API.

Important: every time you change the embedder, you must re-embed existing workspaces. AnythingLLM does not auto-migrate vectors between embedding models because the dimensions differ. Plan your choice before uploading large corpora.

Step 9: Choose a Vector Database

AnythingLLM ships with LanceDB enabled by default. LanceDB is an embedded, serverless, file-based vector store — no extra container, no network round-trip, and it scales comfortably to tens of millions of vectors on a single NVMe disk. For most self-hosted deployments, LanceDB is the right choice and you can safely skip this step.

When LanceDB is not enough:

• You run multiple AnythingLLM instances behind a load balancer.
• You want to share the same vector store between AnythingLLM and other tools (a custom agent, a second RAG app, reporting queries).
• You need advanced filtering, hybrid search, or managed backups.

Option A: External Qdrant

Qdrant is a high-performance Rust-based vector database with excellent filtering and hybrid search. If you already have it deployed (see our Qdrant on Ubuntu guide), switch AnythingLLM to use it.

In Settings → Vector Database, select Qdrant and fill in:

• Qdrant API Endpoint: http://host.docker.internal:6333 (or your Qdrant server URL)
• Qdrant API Key: your key if authentication is enabled

Option B: External Weaviate

Weaviate is another production-grade option with strong multi-tenant and hybrid BM25 + vector search. See our Weaviate on Ubuntu guide for deployment. Configure in AnythingLLM as:

• Weaviate Endpoint: http://host.docker.internal:8080
• API Key: your configured key

Option C: Chroma, Milvus, Pinecone

AnythingLLM also supports Chroma (simple dev workflows), Milvus (massive scale), and Pinecone (hosted). Pick the one that matches your operational preferences.

Step 10: Create a Workspace and Upload Documents

Workspaces are the core organizational unit in AnythingLLM. Each workspace has its own document set, vector collection, chat history, system prompt, and (optionally) user access list.

From the sidebar, click New Workspace and give it a descriptive name — "Support Handbook", "Client Contracts 2026", "Engineering Runbooks". AnythingLLM creates a dedicated vector collection and a fresh chat thread.

Uploading documents

Click the Upload icon on the workspace. AnythingLLM accepts:

• PDFs (including scanned PDFs — OCR is applied automatically)
• Word documents (.docx, .doc)
• PowerPoint and Excel (.pptx, .xlsx, .csv)
• Plain text and Markdown
• HTML files and website URLs (the built-in scraper crawls linked pages)
• YouTube URLs (auto-transcribed)
• GitHub, GitLab, Confluence, Obsidian connectors

Once a document is processed, click the folder icon and move it from "My Documents" into the workspace — this is the step that actually attaches the embeddings to the workspace. Unmoved documents remain idle in your library.

Chatting with citations

Type a question in the chat box. AnythingLLM:

Answers include numbered citations that expand to show the exact chunk and source filename. This is invaluable for compliance workflows — every AI answer is traceable back to a specific page of a specific document.

Tuning workspace behaviour

Under Workspace Settings you can control:

• System prompt — set the persona and constraints ("You are a legal research assistant. Never speculate beyond the provided documents.").
• Chat mode — chat uses documents + general model knowledge; query refuses to answer anything not supported by the documents.
• Top-K retrieval — how many chunks to retrieve. Higher values give more context but slower responses and a diluted answer.
• Similarity threshold — discard chunks below a cosine-similarity floor.
• Temperature — deterministic (0) for Q&A, creative (0.7) for brainstorming.

Step 11: Enable Multi-User Mode

By default AnythingLLM runs in single-user mode — one admin account, no login screen shown. To give your team access with isolated workspaces, turn on multi-user mode.

Go to Settings → Security → Multi-User Mode and toggle it on. You will be prompted to create a new admin account (the existing single-user credentials are migrated).

With multi-user mode enabled, you can:

• Invite users by email (or generate invite codes for signup).
• Assign roles: default, manager, admin.
• Create teams / groups for workspace sharing.
• Restrict workspaces to specific users — sensitive workspaces stay invisible to everyone else.
• Audit usage — see per-user token spend and chat history.

Step 12: Secure with Nginx and TLS

AnythingLLM should never be exposed directly on port 3001 over the public internet. We front it with Nginx for TLS termination, sane security headers, and request buffering tuned for streaming chat.

Install Nginx and Certbot:

sudo apt install -y nginx certbot python3-certbot-nginx

Point your DNS A record for chat.yourdomain.com at the VPS public IP and wait for propagation (usually a few minutes).

Create the site config:

sudo tee /etc/nginx/sites-available/anythingllm > /dev/null <<'EOF'
server {
    listen 80;
    server_name chat.yourdomain.com;
    return 301 https://$host$request_uri;
}
server {
    listen 443 ssl http2;
    server_name chat.yourdomain.com;
    ssl_certificate     /etc/letsencrypt/live/chat.yourdomain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/chat.yourdomain.com/privkey.pem;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;
    # Security headers
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
    add_header X-Content-Type-Options nosniff always;
    add_header X-Frame-Options SAMEORIGIN always;
    add_header Referrer-Policy strict-origin-when-cross-origin always;
    # File uploads up to 2 GB (tune to your corpus)
    client_max_body_size 2048m;
    location / {
        proxy_pass http://127.0.0.1:3001;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        # Streaming chat responses
        proxy_buffering off;
        proxy_cache off;
        proxy_read_timeout 600s;
        proxy_send_timeout 600s;
        # WebSocket (live document processing status)
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
EOF

Enable the site and issue the certificate:

sudo ln -s /etc/nginx/sites-available/anythingllm /etc/nginx/sites-enabled/
sudo nginx -t
sudo certbot --nginx -d chat.yourdomain.com --redirect --agree-tos -m [email protected]
sudo systemctl reload nginx

Update the firewall:

sudo ufw allow 'Nginx Full'
sudo ufw allow OpenSSH
sudo ufw --force enable

Visit https://chat.yourdomain.com — you should see the AnythingLLM login screen served over TLS. Certbot renews the certificate automatically via a systemd timer.

Using the Developer API

AnythingLLM exposes a REST API that mirrors most UI actions. Generate an API key under Settings → API Keys → Generate new API key. The resulting key authenticates all requests via a bearer header.

Chat with a workspace

curl -X POST "https://chat.yourdomain.com/api/v1/workspace/general-knowledge-base/chat" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "message": "Summarize our refund policy in three bullet points.",
    "mode": "chat"
  }'

Expected response:

{
  "id": "chat-uuid",
  "type": "textResponse",
  "textResponse": "- Refunds are available within 30 days...\n- ...",
  "sources": [
    {
      "title": "refund-policy-2026.pdf",
      "chunk": "Our refund policy covers...",
      "score": 0.87
    }
  ],
  "close": true
}

Upload a document programmatically

curl -X POST "https://chat.yourdomain.com/api/v1/document/upload" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -F "file=@./handbook.pdf"

Then attach it to a workspace:

curl -X POST "https://chat.yourdomain.com/api/v1/workspace/general-knowledge-base/update-embeddings" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"adds": ["custom-documents/handbook-xxx.json"]}'

OpenAI-compatible endpoint

AnythingLLM exposes an /api/v1/openai/chat/completions endpoint that speaks the OpenAI schema, so any library that can call OpenAI — LangChain, LlamaIndex, the openai SDK, Cursor, Continue.dev — can point at AnythingLLM with a one-line URL change and instantly gain workspace-grounded answers.

Full API documentation is available at https://chat.yourdomain.com/api/docs after login.

FAQ

Can AnythingLLM work without Ollama or an external LLM?

No. AnythingLLM is an application layer on top of an LLM — it needs a model to generate responses. However, you have broad freedom over where that model lives. For fully offline, private deployments, pair AnythingLLM with local Ollama or LM Studio. For faster, higher-quality answers with some data leaving the network, use OpenAI, Anthropic, or Groq. You can switch providers at any time from the settings UI without re-embedding your documents.

Do I need a GPU for AnythingLLM?

AnythingLLM itself is CPU-only — it runs comfortably on a 2 vCPU / 4 GB VPS when you point it at a hosted LLM. A GPU only matters if you are also running a local LLM (via Ollama) on the same host. For 7B-9B parameter models on CPU, expect 10-20 tokens per second on a 6 vCPU server, which is perfectly usable for chat. If you need sub-second latency or are serving dozens of concurrent users, move the LLM to a GPU-equipped VPS and keep AnythingLLM on the smaller plan.

How does AnythingLLM compare to LibreChat, Open WebUI, and Dify?

AnythingLLM is RAG-first: document ingestion, embeddings, and citations are the headline feature. Workspaces let you scope documents to teams or projects. Best fit for knowledge-base and document-Q&A use cases.

Open WebUI is a polished ChatGPT-style UI for Ollama with growing RAG support. Best fit for individual developers who want a clean chat UI on top of local models.

LibreChat is a multi-provider chat front-end that supports tools, plugins, and shared conversations. RAG is secondary. Best fit for teams that want ChatGPT-Plus parity across multiple LLM providers.

Dify is an agent and workflow builder — visual pipelines, tool use, multi-step reasoning. Best fit for building custom AI applications, not just chat.

The four are complementary: many teams run AnythingLLM for knowledge-base Q&A, Dify for automation workflows, and LibreChat or Open WebUI for general chat.

How much data can AnythingLLM handle?

In practice, the bottleneck is the vector store, not AnythingLLM itself. LanceDB comfortably handles 10-50 million vectors on a single NVMe disk; beyond that, move to Qdrant or Milvus. For most small-to-medium teams — 10k-100k documents, under 10 million chunks — LanceDB is more than enough and incurs zero operational overhead. Disk usage grows roughly linearly at 1-3 KB per chunk of raw vector plus metadata.

Is AnythingLLM production-ready for regulated industries?

AnythingLLM has been deployed in healthcare, legal, financial services, and government settings where data cannot leave the network. Combined with a local LLM and local embedder, the entire RAG pipeline stays on your VPS — no telemetry, no API calls to third parties. For regulatory rigor, pair the deployment with daily encrypted backups of /opt/anythingllm/storage, centralised logs, and access audits. The SOC 2 story is yours to own, but the architecture does not leak data outside your control. Review the project's security documentation at docs.anythingllm.com and validate that configuration decisions match your policy.

How do I back up my AnythingLLM instance?

All durable state lives in /opt/anythingllm/storage — SQLite database, uploaded documents, parsed text, LanceDB vectors, and chat history. A nightly tar + offsite upload is sufficient:

sudo tar czf /backup/anythingllm-$(date +%F).tgz /opt/anythingllm

Schedule it via cron or systemd timer, and push the archive to object storage (S3, Backblaze B2, Wasabi). Restoring is a matter of extracting the archive on a fresh host and running the same docker run command from Step 5.

How do I update to a new version of AnythingLLM?

Pull the latest image and recreate the container — your bind-mounted storage is preserved:

docker pull mintplexlabs/anythingllm:latest
docker stop anythingllm
docker rm anythingllm
then re-run the same docker run ... command from Step 5

Database migrations run automatically on first boot. Always take a fresh backup before upgrading major versions.

Next Steps

You now have a production AnythingLLM deployment on Ubuntu 24.04 with private inference, local embeddings, TLS, and a multi-user workspace model. Here are high-value next moves:

• Scale embeddings with Qdrant or Weaviate — swap LanceDB for a networked vector store when you exceed 10 million chunks or want to share the collection with other apps.
• Install Open WebUI or LibreChat alongside AnythingLLM — use AnythingLLM for document Q&A and a second front-end for open-ended chat.
• Orchestrate multi-step agents with Dify — wire AnythingLLM's OpenAI-compatible endpoint into Dify workflows to build retrieval-grounded automations.
• Connect Continue.dev or Cursor to the AnythingLLM OpenAI-compatible API so your IDE's AI assistant can cite your internal engineering handbook.
• Add monitoring — expose the Docker container's healthcheck to Uptime Kuma or Prometheus so you get paged before users notice.
• Read the docs — docs.anythingllm.com and useanything.com cover advanced topics like agent skills, custom data connectors, and enterprise SSO.

Need more headroom? AnythingLLM + Ollama + a vector store is a surprisingly heavy stack when usage scales. Our CloudCore Professional plan (6 vCPU, 12 GB RAM, 100 GB NVMe, EUR 19.99/month) is tuned for exactly this workload and ships with NVMe storage that keeps LanceDB queries sub-10ms.