How to Install Grafana Tempo on Ubuntu 24.04 VPS: Self-Hosted Distributed Tracing
Distributed tracing is the missing signal in most self-hosted observability stacks. You may already run Prometheus for metrics and Loki for logs, but without traces you cannot see which service call caused a slow checkout or pinpoint where a 500 error originated across a chain of microservices. Grafana Tempo fills that gap with a lightweight, object-storage-native tracing backend that speaks every major protocol (OTLP, Jaeger, Zipkin) and integrates cleanly with Grafana. This guide walks you through installing Tempo on Ubuntu 24.04, wiring up receivers, switching from local disk to S3/MinIO, writing TraceQL queries, enabling service graphs, and hooking up the metrics-generator.
Skip the setup? Deploy a pre-configured observability stack with Grafana, Loki, and Tempo on our CloudCore Professional plan and start correlating traces with logs in minutes.
Table of Contents
What is Grafana Tempo?
Grafana Tempo is an open-source, high-scale distributed tracing backend built by Grafana Labs. It was designed around one central insight: most tracing backends get expensive because they index every span across multiple attributes. Tempo instead indexes only by trace ID and stores the raw span data as compressed blocks in object storage (S3, GCS, Azure Blob, MinIO, or local disk). Full-text querying happens at read time through TraceQL, Grafana's purpose-built trace query language.
This architecture delivers three practical benefits. First, storage is cheap: at typical compression ratios, one terabyte of object storage holds hundreds of millions of spans. Second, ingestion is simple: Tempo runs as a single binary with no external dependencies (no Cassandra, no Elasticsearch, no ZooKeeper). Third, correlation is native: Tempo is built to live alongside Grafana Loki (logs) and Grafana Mimir or Prometheus (metrics), so you can jump from a log line to the trace that emitted it to the metric that alerted on it, all through a single Grafana instance.
Tempo accepts traces over every major protocol:
- OpenTelemetry Protocol (OTLP) over gRPC (port 4317) and HTTP (port 4318) — the modern standard
- Jaeger over gRPC (14250), Thrift HTTP (14268), Thrift compact (6831/UDP), and Thrift binary (6832/UDP)
- Zipkin over HTTP (9411) with JSON payloads
- OpenCensus over gRPC (55678) — legacy but still supported
Why Self-Host Distributed Tracing?
Managed tracing services (Datadog APM, New Relic, Honeycomb, Lightstep) are powerful but priced per-span or per-host in ways that become prohibitive the moment your request volume grows. Self-hosting Tempo on a VPS turns that variable cost into a predictable flat rate:
- Flat-rate economics. A CloudCore Professional VPS at EUR 19.99/month ingests tens of millions of spans per day comfortably. The same volume on Datadog APM often lands in the four-digit monthly range once retention and host counts are factored in.
- Data sovereignty. Traces contain request paths, user IDs, SQL statements, and sometimes payload fragments. Keeping them on infrastructure you control is the cleanest path to GDPR, HIPAA, and SOC 2 compliance.
- No sampling pressure. SaaS vendors push aggressive head-based sampling to cap ingestion costs. On your own Tempo, you can keep 100% of production traces and sample only if disk forces it.
- Correlation with self-hosted logs and metrics. If you already run Grafana, Loki, or Prometheus, Tempo drops in beside them with shared authentication and dashboards.
- TraceQL without per-query fees. Structural queries across millions of traces run against your VPS disk or your own S3 bucket, not a metered API.
Cost Comparison: Self-Hosted Tempo vs. Managed APM
| Scenario | Datadog APM | Honeycomb Pro | Self-Hosted Tempo (VPS) |
|---|---|---|---|
| 10 hosts, 5M spans/day | ~$310/mo | ~$130/mo | EUR 19.99/mo |
| 30-day retention | Extra charge | Included (limited) | Your disk/S3 quota |
| Data leaves your infra? | Yes | Yes | No |
| Full sampling | Rare (cost-prohibitive) | Dynamic | 100% possible |
| TraceQL / span query | Proprietary DSL | Honeycomb Query | Native TraceQL |
| Correlation with logs | Extra module | Extra module | Built-in with Loki |
Prerequisites
Before you begin, make sure you have:
- A VPS running Ubuntu 24.04 LTS with root or sudo access
- SSH access to your server
- At least 4 GB of RAM (8 GB+ recommended for production)
- At least 40 GB of free disk space if using local storage (or any size if using S3/MinIO)
- An existing Grafana instance (or follow our Grafana install guide first)
- Optional: an S3-compatible bucket for long-term trace storage
Recommended Plan: CloudCore Professional>
For an observability stack running Tempo, Loki, and Grafana together on one server, we recommend CloudCore Professional:>
- 6 vCPU cores
- 12 GB RAM
- 100 GB NVMe SSD
- Unmetered bandwidth
- EUR 19.99/month>
This gives you headroom for Tempo's ingesters, Loki's chunks, and Grafana's panels without resource contention.
Connect to your server via SSH:
ssh root@your-server-ipStep 1: Update System Packages
Start with a clean package index and apply any pending security updates. This prevents dependency mismatches during installation.
sudo apt update && sudo apt upgrade -yInstall the small set of tools we need:
sudo apt install -y curl wget tar ca-certificates gnupgIf the kernel was updated, reboot before continuing:
sudo rebootStep 2: Create Tempo User and Directories
Running Tempo as a dedicated unprivileged user is the standard hardening step for any daemon. Create the user, the configuration directory, and the data directory:
sudo useradd --system --no-create-home --shell /usr/sbin/nologin tempo
sudo mkdir -p /etc/tempo /var/lib/tempo /var/log/tempo
sudo chown -R tempo:tempo /var/lib/tempo /var/log/tempoExplanation:
/etc/tempoholdstempo.yaml(configuration)/var/lib/tempoholds the WAL (write-ahead log) and local trace blocks/var/log/tempois reserved for any local log forwarding you add later
Step 3: Download and Install Tempo
Tempo is distributed as a single statically compiled Go binary from the official GitHub releases. Fetch the latest stable release:
TEMPO_VERSION="2.6.1"
cd /tmp
wget "https://github.com/grafana/tempo/releases/download/v${TEMPO_VERSION}/tempo_${TEMPO_VERSION}_linux_amd64.tar.gz"
tar -xzf "tempo_${TEMPO_VERSION}_linux_amd64.tar.gz"
sudo mv tempo /usr/local/bin/tempo
sudo chmod +x /usr/local/bin/tempoVerify the binary:
tempo --versionExpected output:
tempo, version 2.6.1 (branch: HEAD, revision: ...)
build user: root@...
build date: 2026-...
go version: go1.22.7Step 4: Configure Tempo
Tempo is driven by a single YAML file. Create /etc/tempo/tempo.yaml with a monolithic configuration that runs all components (distributor, ingester, compactor, querier, query-frontend) in one process — ideal for single-server deployments up to several thousand spans per second.
sudo tee /etc/tempo/tempo.yaml > /dev/null <<'EOF'Tempo 2.x single-binary configuration
stream_over_http_enabled: true
server: http_listen_port: 3200 grpc_listen_port: 9095 log_level: info
Multi-protocol receivers
distributor: receivers: otlp: protocols: grpc: endpoint: "0.0.0.0:4317" http: endpoint: "0.0.0.0:4318" jaeger: protocols: grpc: endpoint: "0.0.0.0:14250" thrift_http: endpoint: "0.0.0.0:14268" thrift_compact: endpoint: "0.0.0.0:6831" thrift_binary: endpoint: "0.0.0.0:6832" zipkin: endpoint: "0.0.0.0:9411" opencensus: endpoint: "0.0.0.0:55678"ingester: max_block_duration: 5m trace_idle_period: 10s
compactor: compaction: block_retention: 720h # 30 days
Local storage for now; we will switch to S3 in Step 8
storage: trace: backend: local wal: path: /var/lib/tempo/wal local: path: /var/lib/tempo/blocks pool: max_workers: 100 queue_depth: 10000Metrics-generator: produces RED metrics + service graph edges
metrics_generator: registry: external_labels: source: tempo cluster: self-hosted storage: path: /var/lib/tempo/generator/wal remote_write: - url: http://localhost:9090/api/v1/write send_exemplars: true processor: service_graphs: dimensions: [http.method, http.status_code] span_metrics: dimensions: [http.method, http.status_code, http.route]
overrides: defaults: metrics_generator: processors: [service-graphs, span-metrics] collection_interval: 15s ingestion: max_traces_per_user: 50000 burst_size_bytes: 20000000 rate_limit_bytes: 15000000 EOF sudo chown root:tempo /etc/tempo/tempo.yaml sudo chmod 640 /etc/tempo/tempo.yaml
Key sections explained:
distributor.receivers— declares every ingestion port. OTLP gRPC (4317) and HTTP (4318) are the modern defaults. Jaeger and Zipkin receivers are included so legacy clients can point at Tempo without code changes.ingester— flushes in-memory blocks to the backend. The 5-minute block duration balances freshness and compaction load.compactor.compaction.block_retention— 30 days (720h) of retention. Lower it to 72h for development or raise it for long-term audits.storage.trace.backend—localfor now. We swap this to S3/MinIO in Step 8.metrics_generator.storage.remote_write— where RED metrics and service graph edges are pushed. Point this at your Prometheus, Mimir, or VictoriaMetrics endpoint. If you do not have one yet, leave the entry and fix it later.overrides— per-tenant limits. Tempo is multi-tenant by design;defaultsapplies to the syntheticsingle-tenantuser used in non-multi-tenant mode.
Step 5: Create the systemd Service
Write a systemd unit so Tempo starts at boot and restarts on failure:
sudo tee /etc/systemd/system/tempo.service > /dev/null <<'EOF' [Unit] Description=Grafana Tempo Documentation=https://grafana.com/docs/tempo/ After=network-online.target Wants=network-online.target[Service] Type=simple User=tempo Group=tempo ExecStart=/usr/local/bin/tempo -config.file=/etc/tempo/tempo.yaml Restart=on-failure RestartSec=5 LimitNOFILE=65536
Hardening
NoNewPrivileges=true ProtectSystem=strict ProtectHome=true ReadWritePaths=/var/lib/tempo /var/log/tempo PrivateTmp=true PrivateDevices=true
[Install] WantedBy=multi-user.target EOF
Reload systemd, enable, and start the service:
sudo systemctl daemon-reload
sudo systemctl enable --now tempo
sudo systemctl status tempoExpected status output:
● tempo.service - Grafana Tempo
Loaded: loaded (/etc/systemd/system/tempo.service; enabled; preset: enabled)
Active: active (running) since Wed 2026-04-16 10:00:00 UTC; 3s ago
Main PID: 2345 (tempo)
Tasks: 14
Memory: 62.0MCheck the startup log for receiver confirmation:
sudo journalctl -u tempo -n 40 --no-pagerYou should see lines like listening on 0.0.0.0:4317, listening on 0.0.0.0:4318, and listening on 0.0.0.0:9411.
Confirm the HTTP admin endpoint is reachable:
curl -s http://localhost:3200/readyExpected output:
readyStep 6: Send Your First Trace
Let's verify the OTLP HTTP receiver by sending a handcrafted trace. Install curl and uuidgen if not present, then push a minimal span:
TRACE_ID=$(openssl rand -hex 16) SPAN_ID=$(openssl rand -hex 8) NOW_NS=$(date +%s%N)
curl -X POST http://localhost:4318/v1/traces \ -H "Content-Type: application/json" \ -d @- <<EOF { "resourceSpans": [{ "resource": { "attributes": [ {"key": "service.name", "value": {"stringValue": "test-service"}} ] }, "scopeSpans": [{ "scope": {"name": "manual"}, "spans": [{ "traceId": "${TRACE_ID}", "spanId": "${SPAN_ID}", "name": "hello-tempo", "kind": 1, "startTimeUnixNano": "${NOW_NS}", "endTimeUnixNano": "$((NOW_NS + 1000000))", "attributes": [ {"key": "http.method", "value": {"stringValue": "GET"}}, {"key": "http.status_code", "value": {"intValue": "200"}} ] }] }] }] } EOF
A successful push returns an empty JSON object {}. Give Tempo ~10 seconds to flush the WAL, then query the trace back by ID:
curl -s "http://localhost:3200/api/traces/${TRACE_ID}" | head -c 500You should see the span wrapped in a Tempo JSON envelope. If you do, ingestion, storage, and querier are all working end to end.
For a real instrumentation test, point an OpenTelemetry Collector at http://<tempo-host>:4318/v1/traces and forward traces from your applications.
Step 7: Connect Grafana as a Data Source
Open your Grafana instance and go to Connections -> Data sources -> Add data source. Choose Tempo and configure:
- Name:
Tempo - URL:
http://<tempo-host>:3200 - Auth: none (for now — we harden in Step 11)
trace_id tag so you can click from a trace span to its matching log lines.Click Save & test. A green "Data source is working" confirms the connection.
Now go to Explore, pick the Tempo data source, and run a TraceQL query:
{ resource.service.name = "test-service" }The trace you sent in Step 6 should appear. Click into it to see the waterfall view.
Provisioning the Data Source as Code
If you prefer declarative provisioning, add this to /etc/grafana/provisioning/datasources/tempo.yaml:
apiVersion: 1
datasources:
- name: Tempo
type: tempo
access: proxy
url: http://localhost:3200
uid: tempo
jsonData:
tracesToLogsV2:
datasourceUid: loki
spanStartTimeShift: -5m
spanEndTimeShift: 5m
tags: [{ key: 'service.name', value: 'service' }]
filterByTraceID: true
serviceMap:
datasourceUid: prometheus
nodeGraph:
enabled: true
lokiSearch:
datasourceUid: lokiRestart Grafana after provisioning:
sudo systemctl restart grafana-serverStep 8: Move Storage to S3 or MinIO
Local disk is fine for development but unsustainable for production retention. Tempo speaks S3 natively and works with any S3-compatible backend — AWS S3, Backblaze B2, Cloudflare R2, Wasabi, and self-hosted MinIO.
Option A: MinIO on the Same VPS
Install MinIO as a companion service:
wget https://dl.min.io/server/minio/release/linux-amd64/minio
sudo mv minio /usr/local/bin/
sudo chmod +x /usr/local/bin/minio
sudo useradd --system --no-create-home --shell /usr/sbin/nologin minio
sudo mkdir -p /var/lib/minio
sudo chown minio:minio /var/lib/minioCreate a systemd unit at /etc/systemd/system/minio.service:
sudo tee /etc/systemd/system/minio.service > /dev/null <<'EOF' [Unit] Description=MinIO Object Storage After=network-online.target[Service] User=minio Group=minio Environment="MINIO_ROOT_USER=tempoadmin" Environment="MINIO_ROOT_PASSWORD=change-me-to-a-long-random-string" ExecStart=/usr/local/bin/minio server /var/lib/minio --address :9000 --console-address :9001 Restart=on-failure LimitNOFILE=65536
[Install] WantedBy=multi-user.target EOF sudo systemctl daemon-reload sudo systemctl enable --now minio
Create a bucket named tempo-traces via the MinIO console at http://<server-ip>:9001 or with the mc CLI.
Option B: External S3 Bucket
Create an IAM user in AWS with s3:PutObject, s3:GetObject, s3:DeleteObject, and s3:ListBucket on a dedicated bucket (for example my-company-tempo). Record the access key and secret.
Update Tempo Configuration
Replace the storage block in /etc/tempo/tempo.yaml:
storage:
trace:
backend: s3
wal:
path: /var/lib/tempo/wal
s3:
bucket: tempo-traces
endpoint: 127.0.0.1:9000 # or s3.amazonaws.com
access_key: tempoadmin # or AWS_ACCESS_KEY_ID
secret_key: change-me-... # or AWS_SECRET_ACCESS_KEY
insecure: true # true for local MinIO over HTTP
forcepathstyle: true # required for MinIO
pool:
max_workers: 100
queue_depth: 10000For production, pull credentials from environment variables instead of hardcoding them. Create /etc/tempo/tempo.env:
sudo tee /etc/tempo/tempo.env > /dev/null <<'EOF'
AWS_ACCESS_KEY_ID=tempoadmin
AWS_SECRET_ACCESS_KEY=change-me-to-a-long-random-string
EOF
sudo chown root:tempo /etc/tempo/tempo.env
sudo chmod 640 /etc/tempo/tempo.envAdd EnvironmentFile=/etc/tempo/tempo.env to the [Service] section of the systemd unit, then reload and restart:
sudo systemctl daemon-reload
sudo systemctl restart tempoVerify the backend switched successfully:
sudo journalctl -u tempo -n 50 --no-pager | grep -i backendYou should see lines referencing s3 instead of local.
Step 9: Enable Metrics-Generator and Service Graphs
The metrics-generator is already configured in Step 4 but only works if the remote_write URL points at a live Prometheus-compatible receiver. The generator produces two data streams:
- Span metrics (RED):
traces_spanmetrics_calls_total,traces_spanmetrics_duration_seconds_bucket, etc. These give you rate, errors, and duration per service and operation — everything needed for an SLO dashboard. - Service graph metrics:
traces_service_graph_request_total,traces_service_graph_request_failed_total,traces_service_graph_request_client_seconds_bucket. These power the Node Graph visualization in Grafana.
Prometheus Remote-Write
If you run Prometheus 2.x locally, enable the remote-write receiver by starting Prometheus with:
--web.enable-remote-write-receiverThen point Tempo at http://localhost:9090/api/v1/write (already configured above). Reload Prometheus and restart Tempo:
sudo systemctl restart prometheus tempoWithin a minute of ingesting traces, metrics like traces_spanmetrics_calls_total start appearing in Prometheus.
Viewing the Service Graph in Grafana
In Grafana, go to Explore, select the Tempo data source, and switch the query type from Search to Service Graph. The node graph renders services as nodes and request flows as edges, colored by error rate and sized by request volume.
Tuning Cardinality
The dimensions field under each processor adds label dimensions to emitted metrics. Each extra dimension multiplies cardinality, so keep the list tight:
metrics_generator:
processor:
span_metrics:
dimensions: [http.method, http.status_code, http.route]Avoid adding high-cardinality attributes like user.id, http.url, or trace.id — they will explode the Prometheus time series count.
Step 10: Query Traces with TraceQL
TraceQL is Tempo's query language. It runs across raw span data and supports attribute filters, duration filters, and structural operators. Use it in the Grafana Explore pane (query type: TraceQL) or directly against the HTTP API.
Basic Attribute Filters
Find all traces from a specific service:
{ resource.service.name = "checkout" }Find all failed HTTP 500 spans:
{ span.http.status_code = 500 }Find slow database queries:
{ span.db.system = "postgresql" && duration > 500ms }Combining Conditions
Intersect (&&) and union (||) within the same span:
{ resource.service.name = "api-gateway" && span.http.status_code >= 400 && duration > 1s }Structural Queries
Find traces where a checkout service span is an ancestor of a payment service span (parent/child chain):
{ resource.service.name = "checkout" } >> { resource.service.name = "payment" }Find traces where a 500-error span exists anywhere:
{ } >> { span.http.status_code = 500 }HTTP API Queries
Call TraceQL directly via the search endpoint:
curl -s -G "http://localhost:3200/api/search" \
--data-urlencode 'q={ resource.service.name = "checkout" && duration > 500ms }' \
--data-urlencode 'limit=20' | jqFetch a specific trace by ID:
curl -s "http://localhost:3200/api/traces/${TRACE_ID}" | jqList available tag names:
curl -s "http://localhost:3200/api/search/tags" | jqList values for a specific tag:
curl -s "http://localhost:3200/api/search/tag/service.name/values" | jqStep 11: Secure the Endpoints
Tempo ships without built-in authentication — it assumes you place it behind a reverse proxy or a network boundary. For a production deployment on a public VPS, three layers of defense are recommended.
Layer 1: Firewall
Only expose the receiver ports you actually need, and only to trusted sources. With UFW:
sudo ufw default deny incoming
sudo ufw allow ssh
Allow OTLP HTTP only from your application VPS
sudo ufw allow from 203.0.113.50 to any port 4318
Allow Grafana access to the query port from localhost only (if same host)
(No rule needed if Grafana is on the same server)
sudo ufw enableLayer 2: Nginx Reverse Proxy with TLS and Basic Auth
Install Nginx and the Apache htpasswd tool:
sudo apt install -y nginx apache2-utils certbot python3-certbot-nginx
sudo htpasswd -c /etc/nginx/.tempo-htpasswd tempouserCreate /etc/nginx/sites-available/tempo:
server { listen 443 ssl; server_name tempo.yourdomain.com;ssl_certificate /etc/letsencrypt/live/tempo.yourdomain.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/tempo.yourdomain.com/privkey.pem;
# Tempo query API location /api/ { auth_basic "Tempo"; auth_basic_user_file /etc/nginx/.tempo-htpasswd; proxy_pass http://127.0.0.1:3200; proxy_http_version 1.1; proxy_set_header Host $host; proxy_read_timeout 300s; }
# OTLP HTTP ingestion location /v1/traces { auth_basic "Tempo Ingest"; auth_basic_user_file /etc/nginx/.tempo-htpasswd; proxy_pass http://127.0.0.1:4318; proxy_http_version 1.1; proxy_set_header Host $host; client_max_body_size 10m; } }
Enable the site and get a certificate:
sudo ln -s /etc/nginx/sites-available/tempo /etc/nginx/sites-enabled/
sudo certbot --nginx -d tempo.yourdomain.com
sudo nginx -t && sudo systemctl reload nginxGrafana and OTLP clients now authenticate with tempouser:<password> over HTTPS.
Layer 3: Multi-Tenant Headers
Tempo supports multi-tenancy via the X-Scope-OrgID header. Enable it in tempo.yaml:
multitenancy_enabled: trueNow every request must include X-Scope-OrgID: tenant-a (or similar). Use this to isolate traces per customer on a shared Tempo instance.
Performance Tuning
Memory and CPU
Tempo's ingester holds active traces in memory until blocks flush. Rough sizing:
| Ingestion rate | Recommended RAM | CPU |
|---|---|---|
| < 1,000 spans/sec | 2 GB | 2 vCPU |
| 1,000 - 10,000 spans/sec | 4 GB | 4 vCPU |
| 10,000 - 50,000 spans/sec | 8 - 16 GB | 6 - 8 vCPU |
| > 50,000 spans/sec | Switch to microservices mode | multi-host |
Block Duration vs. Query Latency
The ingester.max_block_duration default of 5 minutes balances ingestion throughput and freshness. Shorten to 1m for lower trace-to-query lag; lengthen to 15m to reduce compactor load on very high-volume instances.
Compactor Retention Window
Lower compactor.compaction.block_retention to control disk (or S3) cost. At the same ingestion rate, 7-day retention uses roughly a quarter of the storage of 30-day retention.
Ingester Rate Limits
The overrides.defaults.ingestion.rate_limit_bytes and burst_size_bytes protect the ingester from runaway clients. Start at 15 MB/s sustained and 20 MB burst, then raise as needed.
Troubleshooting
| Problem | Cause | Solution |
|---|---|---|
level=error msg="error pushing spans to backend" | S3/MinIO credentials wrong or bucket missing | Verify credentials in /etc/tempo/tempo.env, confirm bucket exists, check forcepathstyle: true for MinIO |
| Traces not appearing in Grafana | Ingester has not flushed yet | Wait up to max_block_duration + trace_idle_period (~5m10s). For faster feedback set trace_idle_period: 2s in dev |
HTTP 413 Request Entity Too Large from Nginx | Large batch OTLP payload | Raise client_max_body_size in the Nginx server block to 50m |
| Metrics-generator writes nothing | remote_write.url unreachable or Prometheus not in receiver mode | Start Prometheus with --web.enable-remote-write-receiver; test with curl http://localhost:9090/api/v1/write -X POST |
| Service graph empty | No traces have completed inter-service hops yet | Ensure at least two distinct service.name values and parent/child span IDs are present |
connection refused on port 4317 | Distributor did not start OTLP gRPC | Check journalctl -u tempo for receiver startup errors; confirm no other process owns the port (sudo lsof -i :4317) |
| Disk filling up quickly | Retention window too long or ingestion too high | Lower block_retention, switch to S3/MinIO, or raise rate limits to push back on clients |
| TraceQL query returns nothing | Tag name mismatch — TraceQL distinguishes resource. (resource attributes) from span. (span attributes) | Use /api/search/tags to list available tag scopes; adjust the prefix |
Useful Log Commands
# Live stream Tempo logs
sudo journalctl -u tempo -fLast 100 lines
sudo journalctl -u tempo -n 100 --no-pagerOnly errors
sudo journalctl -u tempo | grep -i errorFAQ
What is Grafana Tempo and how is it different from Jaeger?
Tempo is a trace storage and query backend that indexes only by trace ID. Jaeger indexes across tags and service names, which means Jaeger needs a heavy backing store (Cassandra or Elasticsearch) and gets expensive fast. Tempo pushes all query complexity into TraceQL at read time and stores raw data in cheap object storage. For most teams, Tempo is much cheaper to operate at the same trace volume, and it integrates natively with Grafana Loki and Prometheus for correlated logs/metrics/traces.
How much disk space do I need for Tempo traces?
Compressed spans typically land between 10 and 50 bytes each. A service emitting 1,000 spans/second generates roughly 4 to 17 GB per day. A 40 GB disk holds 3 to 14 days of that volume. For longer retention or higher throughput, switch to S3 or MinIO where per-GB costs are an order of magnitude lower than SSD.
Can Tempo accept Jaeger and Zipkin traces?
Yes, natively. Tempo ships receivers for OTLP (gRPC + HTTP), Jaeger (gRPC, Thrift HTTP, Thrift compact, Thrift binary), Zipkin (HTTP JSON), and OpenCensus. Existing instrumentation — Jaeger agents, Zipkin-instrumented Spring Boot apps, OpenTelemetry SDKs — can all point at Tempo without code changes. This makes Tempo a practical drop-in replacement for legacy Jaeger stacks.
Do I need Prometheus or Mimir to use Tempo's metrics-generator?
The metrics-generator produces RED metrics and service graph edges from spans, then writes them out via Prometheus remote-write. You need some Prometheus-compatible receiver — Prometheus with --web.enable-remote-write-receiver, Grafana Mimir, Thanos Receive, or VictoriaMetrics. Without a live remote-write target the generator runs but emits no persisted metrics.
How do I query traces with TraceQL?
TraceQL runs in the Grafana Explore pane or directly against /api/search. The basic form is { <conditions> }, for example { resource.service.name = "checkout" && span.http.status_code = 500 }. Structural operators let you trace parent/child relationships: { resource.service.name = "api" } >> { resource.service.name = "db" } finds traces where the api service calls the db service (descendant relationship). TraceQL also supports &&, ||, numeric comparisons, regex matches, and duration filters.
Should I run Tempo in monolithic or microservices mode?
Monolithic (single binary) mode — what this guide installs — scales to tens of thousands of spans per second on a single VPS. Switch to microservices mode (separate distributor, ingester, compactor, querier, query-frontend processes) only when you need horizontal scaling beyond a single host, typically above 50,000 spans/second sustained or multi-terabyte block storage workloads.
How does Tempo compare to Honeycomb and Datadog APM?
Honeycomb and Datadog offer polished UIs and advanced analytics (BubbleUp, Watchdog). Tempo trades that polish for flat-rate economics and data sovereignty. If you're already invested in Grafana and want unified dashboards across logs, metrics, and traces on infrastructure you control, Tempo is the natural fit. For teams that value managed analytics over raw cost and control, SaaS APM remains a valid choice.
Next Steps
With Tempo running, these are the highest-leverage follow-ups:
- Install the OpenTelemetry Collector — a production-ready relay in front of Tempo that does tail-based sampling, batching, and multi-destination fan-out. Most teams run the Collector between their apps and Tempo.
- Install Grafana Loki — pair traces with structured logs. Grafana's Trace to logs feature lets you click from a slow span directly to the log lines that emitted it.
- Install Grafana — if you haven't already, Grafana is the front door to Tempo. The TraceQL query builder, service graph, and node graph all live there.
- Instrument your apps with OpenTelemetry — the OpenTelemetry SDKs for Node.js, Python, Go, Java, and .NET export OTLP directly to Tempo's port 4318. Add one initialization block to your apps and traces start flowing.
- Build an SLO dashboard from span metrics — use
traces_spanmetrics_calls_totalandtraces_spanmetrics_duration_seconds_bucketto compute error rate and p95 latency per service. Add alerts for SLO burn rate.
- Explore exemplars — with
send_exemplars: truein the metrics-generator, Prometheus stores trace IDs alongside metric samples. In Grafana dashboards, click an exemplar dot on a latency graph to jump straight to the slow trace.
- Read the official Tempo docs for advanced topics: multi-tenancy, search on metadata, generic forwarders, and deploying Tempo microservices on Kubernetes.
Skip the Manual Install — Deploy a Full Observability Stack>
Our CloudCore Professional plan gives you the headroom to run Tempo, Loki, Grafana, and Prometheus side by side on a single VPS — enough for most small-to-medium observability workloads.>
- 6 vCPU, 12 GB RAM, 100 GB NVMe
- Unmetered bandwidth
- EUR 19.99/month>
Deploy your observability VPS and have Tempo ingesting traces within the hour.