Skip to main contentSkip to navigation
[email protected]
Client AreaSupport
Hosting Mammoth
HostingMammothYour Data, Our Responsibility
Home
Solutions
Hosting Services
Store
Pricing
About
Blog
API
Contact

Stay Ahead of the Curve

Get the latest insights on cybersecurity, AI innovations, and enterprise data solutions delivered to your inbox.

Hosting Mammoth
HostingMammothEnterprise Solutions

Enterprise-grade data solutions. Hosting, recovery, cybersecurity, and AI-powered services for businesses worldwide.

[email protected]
Sun - Fri, 9:00am - 5:00pm

Services

  • Cloud Hosting
  • Data Recovery
  • Cybersecurity
  • Legal Support
  • MSP Services
  • Web Development
  • AI Services
  • Free Server Migration

Hosting

  • VPS Hosting (NVMe SSD)
  • VDS Hosting (NVMe)
  • Storage VPS (High SSD)
  • GPU Servers
  • Managed Services
  • Cloud Firewall
  • Load Balancer
  • One-Click Apps
  • n8n Hosting
  • Object Storage
  • FAQ

Company

  • Store
  • Pricing
  • About Us
  • Locations
  • Blog
  • Testimonials
  • Contact
  • Affiliate Program
  • White-Label
  • Terms of Service
  • Privacy Policy
  • Browser Cookies
  • SLA

Support

  • Client Area
  • Submit Ticket
  • Knowledge Base
  • Server Status
  • API Documentation

© 2026 Hosting Mammoth. All rights reserved.

Knowledge Base
Getting StartedAccount ManagementVPS HostingGPU ServersStorage VPSCloud FirewallLoad BalancerServer ManagementBilling & PaymentsSupport & TicketsAffiliate ProgramReseller ProgramMarketplace & Appsn8n HostingManaged ServicesServer MigrationAPI & DevelopersSecurityTroubleshootingGlossaryInstall Guides
  1. Home
  2. /
  3. Support
  4. /
  5. Install Guides
  6. /
  7. How To Install Loki Ubuntu
GUIDEInstall Guides

How to Install Loki on Ubuntu 24.04 VPS — Self-Hosted Log Aggregation

31 min read

How to Install Loki on Ubuntu 24.04 VPS — Self-Hosted Log Aggregation

Centralized logging is one of the three pillars of observability, but most solutions are either painfully expensive (Datadog, Splunk) or operationally heavy (Elasticsearch, Graylog). Grafana Loki takes a different approach: index only metadata labels, store log content in cheap object storage, and query with a Prometheus-like language called LogQL. The result is a log aggregation system that costs a fraction of Elasticsearch-based stacks while scaling to terabytes of logs per day.

This guide walks you through installing Loki on an Ubuntu 24.04 LTS VPS, from the first SSH connection to a production deployment with S3-backed storage, Promtail agents forwarding logs from multiple servers, retention policies, and a Grafana dashboard visualizing everything. By the end you will have a working log pipeline you can point any application, container, or syslog source at.

Looking for a managed Grafana + Loki stack? Our Professional VPS plan provides the right balance of CPU, RAM, and NVMe storage for a single-node Loki deployment serving 10-50 GB of logs per day.

Table of Contents

  • What is Loki?
  • Why Self-Host Loki Instead of Elasticsearch, Graylog, or CloudWatch?
  • Prerequisites
  • Step 1: Update System Packages and Create the Loki User
  • Step 2: Download and Install the Loki Binary
  • Step 3: Choose a Deployment Mode — Monolithic vs Simple Scalable
  • Step 4: Configure loki.yaml
  • Step 5: Configure S3-Compatible Object Storage
  • Step 6: Create the systemd Service
  • Step 7: Install and Configure Promtail
  • Step 8: Connect Loki as a Grafana Datasource
  • Step 9: Query Logs with LogQL
  • Step 10: Configure Retention Policies
  • Performance Tuning
  • Troubleshooting
  • FAQ
  • Next Steps
  • What is Loki?

    Grafana Loki is an open-source, horizontally scalable log aggregation system inspired by Prometheus. Where Prometheus indexes metrics with labels, Loki indexes log streams with labels — but crucially, it does not index the log content itself. Instead, raw log lines are compressed and pushed to an object store (S3, GCS, Azure Blob, or self-hosted MinIO), while a small index of labels is kept fast-queryable.

    This design sounds counterintuitive at first — how can you search logs if you do not index the content? The answer is LogQL, Loki's query language, which uses labels to narrow down the relevant streams first and then greps through the compressed content on the fly. Because labels filter the dataset down to a few gigabytes before the full-text scan happens, queries stay fast without the storage and memory overhead of an inverted index.

    Loki is developed by Grafana Labs, the same team behind Grafana and the Grafana Tempo tracing backend, and is released under the AGPL-3.0 license. The three systems are designed to work together as a cohesive observability stack: Loki for logs, Prometheus for metrics, and Tempo for traces, all queried through a single Grafana UI.

    The ecosystem includes several companion tools. Promtail is the official log-shipping agent, responsible for discovering log files on a host, adding labels, and pushing them to Loki. Grafana Agent and Vector are alternative collectors. For Kubernetes, the Loki Helm chart ships with Promtail as a DaemonSet that auto-discovers pod logs. And the logcli CLI tool lets you query Loki from the terminal, which is useful for scripting and alerts.

    Why Self-Host Loki Instead of Elasticsearch, Graylog, or CloudWatch?

    Log aggregation is one of those problems where the default choice — send everything to a hosted SaaS — quickly becomes the most expensive line item on your infrastructure bill. Here is how Loki compares to the main alternatives.

    Loki vs Elasticsearch (ELK stack). Elasticsearch is the historical standard for log search, and it is extremely powerful — full-text indexing, fuzzy queries, aggregations, vector search. But that power comes at a cost: Elasticsearch typically needs 3-10x more RAM and disk than Loki for the same log volume, because it maintains a full inverted index. A 100 GB/day log pipeline on ELK often requires 32-64 GB RAM and NVMe SSDs; the same volume on Loki runs comfortably on 12 GB RAM and cheap S3 storage. Loki also has a much simpler operational model — a single binary, no JVM tuning, no shard rebalancing, no split brains.

    Loki vs Graylog. Graylog is a polished ELK-based product with built-in dashboards, alerting, and role management. It is a great choice if you need GELF input, complex stream routing, and a turnkey UI. But Graylog carries all of Elasticsearch's operational weight plus a MongoDB dependency for configuration. Loki is the better fit when you already use Grafana for metrics and want a unified observability UI without running a separate logging product.

    Loki vs AWS CloudWatch Logs. CloudWatch is convenient if your workload lives entirely in AWS, but the pricing scales brutally: $0.50 per GB ingested, plus storage, plus $0.005 per query-GB scanned. A team ingesting 50 GB/day pays roughly $750/month on ingestion alone before a single query runs. The equivalent self-hosted Loki setup on a Professional VPS with S3-backed storage costs under EUR 30/month, including bandwidth.

    Cost Comparison: Loki vs Alternatives at 50 GB/day Ingestion

    SolutionMonthly CostIndex SizeQuery LanguageOps Complexity
    Self-hosted Loki (VPS + S3)~EUR 29/moTiny (labels only)LogQLLow (single binary)
    Self-hosted Elasticsearch~EUR 79/mo (bigger VPS)Full inverted indexLucene/KQLHigh (JVM, shards)
    Self-hosted Graylog~EUR 89/moFull inverted indexGL SearchMedium (ES + Mongo)
    AWS CloudWatch Logs~$750/moManagedLogs InsightsZero (but locked-in)
    Datadog Logs~$1,200/moManagedDatadog queryZero (but locked-in)
    For teams generating more than a few GB of logs per day, the savings from self-hosting Loki typically pay for the entire VPS within the first week of the month.

    Prerequisites

    Before you begin, make sure you have:

    • A VPS running Ubuntu 24.04 LTS with root or sudo access
    • SSH access to your server
    • At least 8 GB of RAM (12 GB recommended for production workloads)
    • At least 50 GB of local NVMe storage for the write-ahead log, BoltDB index cache, and chunk buffers (200 GB recommended)
    • An S3-compatible object store — AWS S3, Backblaze B2, Wasabi, or a self-hosted MinIO instance
    • An existing Grafana installation (on the same or a different server) for querying logs
    Recommended Plan: CloudCore Professional
    >
    For a single-node Loki deployment ingesting 10-50 GB of logs per day, we recommend the Professional VPS plan:
    >
    - 6 vCPU cores
    - 12 GB RAM
    - 200 GB NVMe SSD
    - Unmetered bandwidth
    - EUR 19.99/month
    >
    This gives you enough headroom for Loki's ingester, BoltDB-shipper cache, and Promtail sidecar traffic, while leaving resources for Grafana on the same node. For larger ingestion rates (100+ GB/day) or high availability, move to the Simple Scalable mode across multiple VPS instances — our CloudCore Business plan (8 vCPU / 24 GB RAM) at EUR 29.99/month is a good fit for write nodes.

    CloudCore Pricing Tiers for Loki

    PlanSpecsIngestion CapacityMonthly Price
    CloudCore Starter2 vCPU / 4 GB / 50 GBUp to 5 GB/dayEUR 7.99
    CloudCore Professional6 vCPU / 12 GB / 200 GB5-50 GB/dayEUR 19.99
    CloudCore Business8 vCPU / 24 GB / 400 GB50-150 GB/dayEUR 29.99
    Connect to your server via SSH to get started:

    bash
    ssh root@your-server-ip

    Step 1: Update System Packages and Create the Loki User

    Start by updating your package index and installing a few dependencies we will use later (unzip, curl, wget).

    bash
    sudo apt update && sudo apt upgrade -y
    sudo apt install -y unzip curl wget gnupg ca-certificates

    Create a dedicated system user for running Loki. Running services under an unprivileged user is a basic security hygiene step — if the Loki process is ever compromised, the attacker cannot modify system binaries or read other users' files.

    bash
    sudo groupadd --system loki
    sudo useradd --system --no-create-home --shell /usr/sbin/nologin --gid loki loki

    Create the directories Loki will use for configuration, data, and logs:

    bash
    sudo mkdir -p /etc/loki /var/lib/loki /var/log/loki
    sudo chown -R loki:loki /var/lib/loki /var/log/loki
    sudo chmod 750 /etc/loki /var/lib/loki

    Step 2: Download and Install the Loki Binary

    Grafana Labs publishes pre-compiled binaries for every Loki release on GitHub. We will pin to a specific version to avoid surprise upgrades.

    Download the latest stable release (at the time of writing, 3.3.2 — check the releases page for the newest version):

    bash
    cd /tmp
    LOKI_VERSION="3.3.2"
    wget "https://github.com/grafana/loki/releases/download/v${LOKI_VERSION}/loki-linux-amd64.zip"
    unzip loki-linux-amd64.zip
    sudo mv loki-linux-amd64 /usr/local/bin/loki
    sudo chmod +x /usr/local/bin/loki
    sudo chown root:root /usr/local/bin/loki

    Verify the install:

    bash
    loki --version

    Expected output:

    text
    loki, version 3.3.2 (branch: HEAD, revision: abc1234)
      build user:       root@buildkitsandbox
      build date:       2026-03-04
      go version:       go1.22.9
      platform:         linux/amd64

    While we are at it, also install logcli, the Loki query tool. It is indispensable for debugging and scripting alerts.

    bash
    cd /tmp
    wget "https://github.com/grafana/loki/releases/download/v${LOKI_VERSION}/logcli-linux-amd64.zip"
    unzip logcli-linux-amd64.zip
    sudo mv logcli-linux-amd64 /usr/local/bin/logcli
    sudo chmod +x /usr/local/bin/logcli

    Step 3: Choose a Deployment Mode — Monolithic vs Simple Scalable

    Loki can run in three architectural modes. Picking the right one upfront saves a migration later.

    Monolithic mode runs all Loki components (distributor, ingester, querier, query-frontend, compactor) inside a single process. This is the simplest setup and what we will use for this guide. It scales vertically to roughly 100 GB/day of ingested logs on a well-provisioned VPS. Single-node monolithic is ideal for small teams, staging environments, and side projects where operational simplicity matters more than horizontal scale.

    Simple Scalable mode splits Loki into two roles: read (querier + query-frontend) and write (distributor + ingester + compactor). Each role runs as its own process, and you can scale them independently — more write nodes during peak ingestion, more read nodes during dashboard-heavy hours. This is the recommended mode for production deployments ingesting 100 GB - 1 TB per day. It adds some operational complexity (a load balancer, shared object store, memberlist gossip) but remains much simpler than the fully microservices mode.

    Microservices mode breaks every component into its own deployment. This is what Grafana Cloud uses internally, and it scales to petabytes per day. For almost everyone else, it is operational overkill.

    Decision Table

    Ingestion VolumeTeam SizeRecommended ModeCloudCore Plan
    < 10 GB/day1-5 engineersMonolithic on a single VPSStandard
    10-100 GB/day5-50 engineersMonolithicProfessional
    100 GB - 1 TB/day50+ engineersSimple Scalable (2 write, 2 read)Business x 4
    1 TB+/dayPlatform teamMicroservices / Grafana CloudEnterprise cluster
    For this tutorial we proceed with monolithic mode using S3-compatible object storage for chunks and the TSDB index shipper. This configuration scales comfortably up to 100 GB/day and can be migrated to Simple Scalable later without re-ingesting historical data — the object-store chunks carry over as-is.

    Step 4: Configure loki.yaml

    Loki's behavior is controlled by a single YAML file. Create /etc/loki/loki.yaml with the following configuration:

    bash
    sudo tee /etc/loki/loki.yaml > /dev/null <<'EOF'
    auth_enabled: false

    server: http_listen_port: 3100 grpc_listen_port: 9096 log_level: info

    common: instance_addr: 127.0.0.1 path_prefix: /var/lib/loki replication_factor: 1 ring: kvstore: store: inmemory

    ingester: wal: enabled: true dir: /var/lib/loki/wal lifecycler: address: 127.0.0.1 ring: kvstore: store: inmemory replication_factor: 1 final_sleep: 0s chunk_idle_period: 1h max_chunk_age: 1h chunk_target_size: 1572864 chunk_retain_period: 30s

    schema_config: configs: - from: 2024-01-01 store: tsdb object_store: s3 schema: v13 index: prefix: loki_index_ period: 24h

    storage_config: tsdb_shipper: active_index_directory: /var/lib/loki/tsdb-index cache_location: /var/lib/loki/tsdb-cache aws: s3: s3://AKIAEXAMPLE:[email protected]/loki-chunks s3forcepathstyle: true

    limits_config: reject_old_samples: true reject_old_samples_max_age: 168h retention_period: 744h max_query_series: 10000 max_query_parallelism: 32 ingestion_rate_mb: 16 ingestion_burst_size_mb: 32

    compactor: working_directory: /var/lib/loki/compactor compaction_interval: 10m retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 delete_request_store: s3

    ruler: storage: type: local local: directory: /etc/loki/rules rule_path: /var/lib/loki/rules-temp alertmanager_url: http://localhost:9093 ring: kvstore: store: inmemory enable_api: true

    analytics: reporting_enabled: false EOF

    sudo chown loki:loki /etc/loki/loki.yaml sudo chmod 640 /etc/loki/loki.yaml

    Let's break down the key sections:

    • auth_enabled: false — Disables multi-tenancy. For a single-team deployment this is fine; all logs share tenant ID fake. Enable this and put Loki behind an auth proxy (Nginx + basic auth, or OAuth2 Proxy) if you plan to host logs for multiple tenants.
    • server — HTTP on 3100 is the query and push endpoint; gRPC on 9096 is used for internal component communication. Keep these bound to localhost or your private network — do not expose them directly to the internet.
    • ingester.wal — The write-ahead log protects against data loss if Loki crashes mid-flush. Keep WAL on local NVMe.
    • schema_config — Tells Loki which storage schema to use starting from a given date. tsdb is the current recommended index format (replaces the older boltdb-shipper) and ships with Loki 3.x. The from date must be in the past, and you must never change or remove an existing schema period — add a new one if you need to migrate.
    • storage_config.aws — The S3 connection string. We will fill in real credentials in Step 5.
    • limits_config.retention_period: 744h — 31 days of retention. Loki only deletes logs older than this if compactor.retention_enabled is true.
    • compactor — Compacts index files and enforces retention. Without the compactor, logs accumulate forever.
    Validate the syntax before starting the service:

    bash
    sudo -u loki /usr/local/bin/loki -config.file=/etc/loki/loki.yaml -verify-config

    Expected output:

    text
    config is valid

    Step 5: Configure S3-Compatible Object Storage

    The S3 connection string in loki.yaml is the most sensitive part of the config. Put real credentials there, not placeholders.

    Option A: AWS S3

    Create a bucket (via the AWS console or CLI) named loki-chunks-yourcompany. Create an IAM user with a policy that grants s3:ListBucket, s3:GetObject, s3:PutObject, and s3:DeleteObject only on that bucket.

    Update the storage block:

    yaml
    storage_config:
      tsdb_shipper:
        active_index_directory: /var/lib/loki/tsdb-index
        cache_location: /var/lib/loki/tsdb-cache
      aws:
        s3: s3://AKIAYOURKEY:[email protected]/loki-chunks-yourcompany
        s3forcepathstyle: false

    Option B: Self-Hosted MinIO

    If you want full data sovereignty, run MinIO on another VPS and point Loki at it. Create a bucket called loki-chunks and an access key:

    yaml
    storage_config:
      aws:
        s3: s3://minioaccess:[email protected]:9000/loki-chunks
        s3forcepathstyle: true
        insecure: true

    The s3forcepathstyle: true flag is required for MinIO, Ceph, and most non-AWS S3 implementations — they use path-style addressing (host/bucket/object) rather than virtual-host-style (bucket.host/object).

    Option C: Backblaze B2 or Wasabi

    These are drop-in replacements for S3 at a fraction of the cost. Example for Backblaze:

    yaml
    storage_config:
      aws:
        s3: s3://B2KEYID:[email protected]/loki-chunks
        s3forcepathstyle: true

    Securing Credentials

    Hard-coding credentials in loki.yaml works but is not ideal. For production, prefer environment variables:

    yaml
    aws:
        s3: s3://${S3_ACCESS_KEY}:${S3_SECRET_KEY}@s3.eu-central-1.amazonaws.com/loki-chunks

    And pass them via the systemd unit (see next step).

    Step 6: Create the systemd Service

    Create /etc/systemd/system/loki.service:

    bash
    sudo tee /etc/systemd/system/loki.service > /dev/null <<'EOF'
    [Unit]
    Description=Grafana Loki log aggregation service
    Documentation=https://grafana.com/docs/loki/latest/
    Wants=network-online.target
    After=network-online.target

    [Service] Type=simple User=loki Group=loki ExecStart=/usr/local/bin/loki \ -config.file=/etc/loki/loki.yaml \ -config.expand-env=true Restart=on-failure RestartSec=10 LimitNOFILE=65536 EnvironmentFile=-/etc/loki/loki.env

    Hardening

    NoNewPrivileges=true ProtectSystem=full ProtectHome=true PrivateTmp=true ReadWritePaths=/var/lib/loki /var/log/loki

    [Install] WantedBy=multi-user.target EOF

    Create the environment file for secrets:

    bash
    sudo tee /etc/loki/loki.env > /dev/null <<'EOF'
    S3_ACCESS_KEY=AKIAYOURKEY
    S3_SECRET_KEY=YourSecretKey
    EOF
    sudo chown loki:loki /etc/loki/loki.env
    sudo chmod 600 /etc/loki/loki.env

    Reload systemd, enable, and start Loki:

    bash
    sudo systemctl daemon-reload
    sudo systemctl enable loki
    sudo systemctl start loki
    sudo systemctl status loki

    Expected output:

    text
    ● loki.service - Grafana Loki log aggregation service
         Loaded: loaded (/etc/systemd/system/loki.service; enabled; preset: enabled)
         Active: active (running) since Wed 2026-04-16 10:00:00 UTC; 5s ago
       Main PID: 1234 (loki)
          Tasks: 12
         Memory: 128.4M
         CGroup: /system.slice/loki.service
                 └─1234 /usr/local/bin/loki -config.file=/etc/loki/loki.yaml ...

    Verify the /ready endpoint returns 200:

    bash
    curl http://localhost:3100/ready

    Expected output:

    text
    ready

    And /metrics should return Prometheus-format metrics:

    bash
    curl -s http://localhost:3100/metrics | head -20

    Step 7: Install and Configure Promtail

    Loki does not collect logs itself — it is a server that receives logs pushed to it. Promtail is the official agent that tails log files on each host, attaches labels, and forwards them to Loki over HTTP.

    Install Promtail on the same server as Loki first. Later you will deploy Promtail to every host whose logs you want to collect.

    bash
    cd /tmp
    wget "https://github.com/grafana/loki/releases/download/v${LOKI_VERSION}/promtail-linux-amd64.zip"
    unzip promtail-linux-amd64.zip
    sudo mv promtail-linux-amd64 /usr/local/bin/promtail
    sudo chmod +x /usr/local/bin/promtail

    sudo groupadd --system promtail sudo useradd --system --no-create-home --shell /usr/sbin/nologin --gid promtail promtail sudo usermod -aG adm promtail # access to /var/log/syslog and similar sudo mkdir -p /etc/promtail /var/lib/promtail sudo chown -R promtail:promtail /var/lib/promtail

    Create /etc/promtail/promtail.yaml:

    bash
    sudo tee /etc/promtail/promtail.yaml > /dev/null <<'EOF'
    server:
      http_listen_port: 9080
      grpc_listen_port: 0

    positions: filename: /var/lib/promtail/positions.yaml

    clients: - url: http://localhost:3100/loki/api/v1/push

    scrape_configs: - job_name: system static_configs: - targets: - localhost labels: job: varlogs host: web-01 __path__: /var/log/*.log

    - job_name: journal journal: max_age: 12h labels: job: systemd-journal host: web-01 path: /var/log/journal relabel_configs: - source_labels: ['__journal__systemd_unit'] target_label: 'unit' - source_labels: ['__journal_priority_keyword'] target_label: 'level'

    - job_name: nginx static_configs: - targets: - localhost labels: job: nginx host: web-01 __path__: /var/log/nginx/*.log pipeline_stages: - match: selector: '{job="nginx"}' stages: - regex: expression: '^(?P<remote_addr>[\d\.]+) - (?P<remote_user>\S+) \[(?P<time_local>[^\]]+)\] "(?P<method>\S+) (?P<path>\S+) (?P<protocol>\S+)" (?P<status>\d+) (?P<body_bytes_sent>\d+)' - labels: method: status: EOF

    sudo chown promtail:promtail /etc/promtail/promtail.yaml

    The config has three scrape jobs:

  • system — Tails every .log file in /var/log/ and labels it job=varlogs.
  • journal — Reads directly from the systemd journal and extracts the unit name and priority as labels.
  • nginx — Parses Nginx access-log format with a regex pipeline and extracts HTTP method and status as labels.
  • Create the Promtail systemd unit:

    bash
    sudo tee /etc/systemd/system/promtail.service > /dev/null <<'EOF'
    [Unit]
    Description=Promtail log shipper
    Documentation=https://grafana.com/docs/loki/latest/send-data/promtail/
    Wants=network-online.target
    After=network-online.target

    [Service] Type=simple User=promtail Group=promtail ExecStart=/usr/local/bin/promtail -config.file=/etc/promtail/promtail.yaml Restart=on-failure RestartSec=10

    [Install] WantedBy=multi-user.target EOF

    sudo systemctl daemon-reload sudo systemctl enable --now promtail sudo systemctl status promtail

    Within 30 seconds, Promtail will start pushing logs to Loki. Verify:

    bash
    logcli --addr=http://localhost:3100 labels

    Expected output:

    text
    host
    job
    level
    method
    status
    unit

    Step 8: Connect Loki as a Grafana Datasource

    Assuming you already have Grafana running (on the same host or a different one), add Loki as a datasource.

    In the Grafana UI, navigate to Connections -> Data sources -> Add data source, select Loki, and configure:

    • Name: Loki
    • URL: http://localhost:3100 (or the internal IP of your Loki server)
    • HTTP Header (optional): X-Scope-OrgID: fake if auth_enabled: true
    Click Save & test. You should see a green "Data source connected and labels found" message.

    Alternatively, provision the datasource via YAML (the 12-factor approach):

    yaml
    # /etc/grafana/provisioning/datasources/loki.yaml
    apiVersion: 1

    datasources: - name: Loki type: loki access: proxy url: http://loki.internal:3100 jsonData: maxLines: 5000 timeout: 60

    Restart Grafana, and the datasource appears automatically. Open the Explore view, pick Loki, and run your first query — {job="varlogs"} should return log lines from /var/log.

    Step 9: Query Logs with LogQL

    LogQL is the query language for Loki, designed to feel familiar if you know PromQL. Every query has two parts: a log stream selector (label matchers) and an optional filter expression.

    Basic Stream Selection

    logql
    {job="varlogs"}

    Returns all log lines with the label job=varlogs.

    logql
    {job="nginx", status="500"}

    Returns all Nginx log lines where the parsed status was 500.

    Filtering Log Content

    After the selector, pipe the stream through line filters:

    logql
    {job="nginx"} |= "GET /api"

    |= means "line contains". Other operators:

    • != — line does not contain
    • |~ — line matches regex
    • !~ — line does not match regex
    Example: all Nginx 5xx errors mentioning /api/users:

    logql
    {job="nginx", status=~"5.."} |= "/api/users"

    Parser Stages

    Chain parsers to extract fields at query time:

    logql
    {job="app"}
      | json
      | duration_ms > 1000
      | line_format "{{.request_id}} took {{.duration_ms}}ms"

    Aggregation (Metric Queries)

    LogQL can convert log streams into time-series metrics. This is Loki's killer feature — you can alert on log patterns without running a separate SIEM.

    Count 500 errors per minute by host:

    logql
    sum by (host) (rate({job="nginx", status="500"}[1m]))

    95th percentile request latency if you emit a duration_ms field:

    logql
    quantile_over_time(0.95,
      {job="nginx"}
        | json
        | unwrap duration_ms [5m]
    ) by (path)

    Common Useful Queries

    logql
    # All error-level logs in the last hour
    {job="systemd-journal", level="err"}

    SSH failed-login attempts

    {unit="ssh.service"} |= "Failed password"

    Top 10 noisiest systemd units

    topk(10, sum by (unit) (count_over_time({job="systemd-journal"}[1h])))

    Any log with stack traces

    {job=~".+"} |~ "Traceback|Exception|panic:"

    Run any of these from the command line with logcli:

    bash
    logcli --addr=http://localhost:3100 query '{job="nginx", status="500"}' --limit 50

    Step 10: Configure Retention Policies

    By default, the limits_config.retention_period in loki.yaml applies globally. To set per-tenant or per-stream retention, use the retention_stream section of limits_config.

    Edit /etc/loki/loki.yaml:

    yaml
    limits_config:
      retention_period: 744h   # 31 days default
      retention_stream:
        - selector: '{job="nginx"}'
          priority: 1
          period: 2160h        # 90 days for Nginx
        - selector: '{level="debug"}'
          priority: 2
          period: 72h          # 3 days for debug logs
        - selector: '{job="audit"}'
          priority: 1
          period: 8760h        # 1 year for audit logs

    compactor: working_directory: /var/lib/loki/compactor compaction_interval: 10m retention_enabled: true retention_delete_delay: 2h retention_delete_worker_count: 150 delete_request_store: s3

    Lower priority wins when multiple stream selectors match. After changes, restart Loki:

    bash
    sudo systemctl restart loki

    The compactor runs every 10 minutes and marks chunks older than the retention window for deletion. Actual deletion from S3 happens after retention_delete_delay, giving you a 2-hour safety window to recover from misconfiguration.

    Monitor the compactor:

    bash
    curl -s http://localhost:3100/metrics | grep loki_compactor_

    Key metrics to watch:

    • loki_compactor_oldest_pending_delete_request_age_seconds — should be low; high values mean the compactor is falling behind.
    • loki_compactor_deleted_chunks_total — counter of chunks successfully deleted.

    Performance Tuning

    A monolithic Loki on a Professional VPS (6 vCPU / 12 GB) comfortably handles 50-100 GB/day. Push beyond that and you need to tune.

    Ingester Flush Behavior

    The ingester buffers log chunks in memory and flushes them to S3 when any of these conditions trigger:

    • Chunk reaches chunk_target_size (default 1.5 MB)
    • Chunk is idle for chunk_idle_period
    • Chunk is older than max_chunk_age
    Larger chunks are cheaper to store (fewer S3 PUT requests) but slower to query in recent-time windows. For chatty applications, increase chunk_target_size to 3 MB. For ultra-low-latency tail -f in Grafana, decrease chunk_idle_period to 5 minutes.

    Query Parallelism

    The query-frontend splits long queries into sub-queries and runs them in parallel:

    yaml
    limits_config:
      max_query_parallelism: 32
      split_queries_by_interval: 30m

    For a query spanning 24 hours with split_queries_by_interval: 30m, Loki runs 48 sub-queries capped at 32 in parallel. Higher parallelism speeds up long-range queries but increases S3 request costs.

    Caching

    Enable results caching for repeated dashboard queries:

    yaml
    query_range:
      align_queries_with_step: true
      cache_results: true
      results_cache:
        cache:
          embedded_cache:
            enabled: true
            max_size_mb: 512
            ttl: 24h

    On high-traffic deployments, swap the embedded cache for external Memcached or Redis.

    Ingestion Rate Limits

    Protect Loki from runaway log sources:

    yaml
    limits_config:
      ingestion_rate_mb: 16        # per tenant MB/s
      ingestion_burst_size_mb: 32  # short-burst allowance
      per_stream_rate_limit: 5MB
      per_stream_rate_limit_burst: 15MB

    If Promtail starts reporting 429 Too Many Requests, raise these limits — but also investigate whether a misbehaving application is the real cause.

    Troubleshooting

    ProblemCauseSolution
    context deadline exceeded on S3 writesNetwork latency or wrong endpointVerify the S3 endpoint URL. Test with aws s3 ls s3://loki-chunks --endpoint-url=.... Increase storage_config.aws.http_config.idle_conn_timeout.
    no such host for S3 endpointDNS resolution or typoResolve the hostname manually: dig s3.eu-central-1.amazonaws.com. Check /etc/resolv.conf.
    Promtail shows 429 Too Many RequestsLoki ingestion rate limit hitIncrease limits_config.ingestion_rate_mb and per_stream_rate_limit. Check for runaway log producers.
    Queries return empty results but labels existTime range mismatchIn Grafana, set the time range to include a period when logs were pushed. Check positions.yaml in Promtail — it may be tailing from the end of the file.
    failed to flush user in Loki logsS3 credentials invalid or bucket unreachableCheck credentials in /etc/loki/loki.env. Verify bucket policy allows the IAM user to PUT objects.
    Disk fills up on /var/lib/lokiWAL or tsdb-shipper cache growingCheck df -h. If WAL is large, it means S3 writes are failing. Fix S3 connectivity; then WAL drains automatically.
    Retention not deleting old logsCompactor disabled or not runningVerify compactor.retention_enabled: true. Check loki_compactor_runs_completed_total is incrementing.
    msg="failed to create index client"Wrong schema versionSchema v13 requires Loki 3.x. Older Loki uses v11 or v12. Never delete an existing schema period.
    High memory usageToo many active streams (label cardinality)Check loki_ingester_memory_streams. Reduce label cardinality — avoid labels with high-cardinality values like user IDs or request paths.
    level=error msg="error writing WAL"Disk full or permissionsCheck df -h and ownership of /var/lib/loki/wal. Must be writable by the loki user.

    Viewing Logs

    Loki's own logs are your best debugging tool:

    bash
    sudo journalctl -u loki -f
    sudo journalctl -u promtail -f

    FAQ

    How does Loki compare to Elasticsearch for log search?

    Loki indexes only labels (host, job, namespace, level) and stores log content as compressed chunks in object storage. Elasticsearch indexes every word in every log line as part of an inverted index, enabling fast full-text search, fuzzy matching, and aggregations over arbitrary fields. Loki is 3-10x cheaper to run for the same ingestion volume and simpler to operate, but Elasticsearch wins for ad-hoc exploratory search where you do not know which labels to query in advance. For most observability use cases — "show me errors from service X between 10:00 and 10:15" — LogQL with label selectors and grep-style filters is plenty fast and dramatically cheaper.

    Can I migrate from Graylog or ELK to Loki without data loss?

    Historical logs in Graylog or Elasticsearch cannot be directly imported into Loki — the storage formats are incompatible. The practical migration path is to run both systems in parallel for the retention period of your old system (typically 30-90 days), forward new logs to both Loki and the legacy system during that window, and decommission the legacy system once its retention window has rolled over. Graylog users often appreciate Loki's simpler operational model — no Elasticsearch, no MongoDB, no JVM tuning — once the migration completes.

    Does Loki require S3, or can I use local disk only?

    Loki supports local filesystem storage via the filesystem backend, which is useful for development and testing. For production, always use object storage (S3, GCS, Azure Blob, or self-hosted MinIO). Local disk storage does not scale horizontally, does not survive instance replacement, and limits retention to whatever fits on the local volume. Even on a single-node deployment, pointing at an S3-compatible object store gives you durability, cheap long-term retention, and a trivial upgrade path to Simple Scalable mode.

    How do I secure the Loki HTTP endpoint?

    Loki has no built-in authentication. In production, put it behind a reverse proxy (Nginx, Traefik, or Caddy) with TLS and basic auth or OAuth2 Proxy. For multi-tenant deployments, set auth_enabled: true in loki.yaml and have the auth proxy inject the X-Scope-OrgID header based on the authenticated user. Also firewall ports 3100 and 9096 so only Promtail agents and Grafana can reach them — never expose Loki directly to the public internet.

    What labels should I use in Promtail?

    Keep label cardinality low. Good labels: host, job, env, namespace, cluster, unit. Bad labels: request_id, user_id, trace_id, path, ip — any value that changes often creates a new stream, and Loki's ingester keeps one in-memory buffer per stream. A deployment with 10,000+ active streams per tenant typically indicates a cardinality problem that will eventually OOM the ingester. Put high-cardinality values inside the log line itself (as JSON or key=value pairs) and extract them at query time with LogQL's | json and | logfmt parsers, which do not create new streams.

    Can Loki alert on log patterns like Elasticsearch Watcher?

    Yes. Loki's ruler component supports alerting rules written in LogQL, compatible with Prometheus Alertmanager. For example, you can alert when the rate of 500 errors exceeds 5 per minute, or when a specific error message appears more than 10 times in 5 minutes. Rules live in YAML files in the directory pointed to by ruler.storage.local.directory and Loki evaluates them on a configurable interval. Alerts are pushed to the same Alertmanager you use for metric alerts, giving you unified alert routing and deduplication across logs and metrics.

    How much does a self-hosted Loki deployment actually cost?

    For a team ingesting around 50 GB/day, you need a CloudCore Professional VPS (EUR 19.99/month) plus object storage costs. On AWS S3 with 30-day retention and lz4 compression, that is roughly 450 GB stored at $0.023/GB = ~$10/month plus PUT requests (~$3/month). Backblaze B2 or Wasabi drop that to under $3/month. Total monthly cost: EUR 25-30. An equivalent Elasticsearch-based stack needs at least a Business-tier VPS (EUR 39.99/month) with no way to offload cold data to cheap storage without additional complexity.

    Next Steps

    Now that Loki is running on your VPS, here are recommended follow-ups to build out a complete observability stack:

    • Install Grafana and build log dashboards — If you have not already, install Grafana on the same VPS and build dashboards that combine Loki log panels with Prometheus metric panels for side-by-side troubleshooting.
    • Deploy Promtail to every host — Install Promtail on every server, container host, and Kubernetes cluster whose logs you want to aggregate. Use the same label scheme across all Promtails so queries can span hosts.
    • Add Prometheus for metrics — Install Prometheus alongside Loki and scrape the /metrics endpoints of both Loki and Promtail. This gives you visibility into ingestion rates, query latency, and compaction progress, and lets you alert on Loki itself falling behind.
    • Set up MinIO for self-hosted object storage — If AWS S3 egress costs concern you, deploy MinIO on a separate VPS with bulk storage and point Loki at it. You get S3-compatible storage with zero egress fees.
    • Migrate from Elasticsearch or Graylog — If you are running Elasticsearch or Graylog, start dual-writing logs to Loki today and retire the heavier stack once your legacy retention window rolls over.
    • Read the official Loki docs — The Grafana Labs documentation at grafana.com/docs/loki/latest/ is the authoritative reference for configuration options, LogQL functions, and scaling guidance.

    Need more storage or CPU? Upgrade your VPS at any time from the Professional plan to a higher tier without re-provisioning. Loki's chunk storage lives in S3, so scaling up takes only a resize and a reboot.

    Was this article helpful?

    ← Back to Install GuidesBrowse all categories →

    Still have questions?

    Contact Support →Submit a Ticket