Jaeger Overview

Jaeger is an open‑source distributed tracing platform that began as an Uber project and later became a graduated CNCF pr

Jaeger is an open‑source distributed tracing platform that began as an Uber project and later became a graduated CNCF project. It captures, stores, queries, and visualizes request traces across microservice architectures, enabling developers to pinpoint latency hotspots, error propagation, and service‑dependency issues. With OpenTelemetry emerging as the de‑facto standard for telemetry collection, Jaeger is commonly used as the backend and UI for trace data.

Core Concepts

  • Trace – A complete request journey through a distributed system. A trace is identified by a unique Trace ID and consists of multiple Spans.
  • Span – The fundamental unit of work in a trace. Each span records an operation or logical block, including its name, start time, duration, tags, logs, and a reference to its parent span.
  • SpanContext – The context that travels across process boundaries. It carries the Trace ID, Span ID, sampling flag, and optional Baggage.
  • Baggage – Key‑value pairs attached to a trace and propagated with every request. They are useful for passing lightweight business context but should not contain sensitive or large data.
  • Sampling – A strategy that determines which traces are recorded and exported. Proper sampling reduces storage and network overhead while still providing actionable insights.

Architecture & Components

Jaeger’s typical deployment includes the following building blocks:

Component Role Notes
Client / SDK Instruments application code, creating spans and injecting/extracting context. Modern projects favor the OpenTelemetry SDK.
Agent Legacy local proxy that batches data from the client and forwards it to the Collector. Being phased out in favor of direct Collector or OTLP endpoints.
Collector Receives trace data, validates, applies sampling, and writes to storage. Handles protocol conversion and back‑pressure.
Query Exposes a REST API for retrieving traces and serves the web UI. Reads directly from the chosen storage backend.
Storage Persistent backend for trace data. Options include in‑memory, Badger, Cassandra, Elasticsearch, and others. Production workloads usually use Elasticsearch or Cassandra.
UI Web interface for searching traces, visualizing call graphs, and analyzing service dependencies. Accessible on port 16686 by default.

Typical Data Flow

  1. Request Arrival – The application receives an incoming request and starts a root span, generating a new Trace ID.
  2. Context Propagation – When calling downstream services, the current SpanContext is injected into HTTP headers or message attributes.
  3. Child Spans – Downstream services extract the context, create child spans, and establish parent‑child relationships.
  4. Export – Finished spans are exported via OTLP or the native Jaeger protocol to the Collector.
  5. Storage & Query – The Collector writes the data to the configured storage. The Query service reads from storage and feeds the UI for exploration.

Quick Deployment with Docker

A single Docker image bundles all components and is ideal for experimentation:

docker run -d --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest

After the container starts, open http://localhost:16686 to access the Jaeger UI.

  • 4317 is the OTLP gRPC port.
  • 4318 is the OTLP HTTP port.

Integrating with OpenTelemetry

Below is a minimal Python example that sends spans to Jaeger via OTLP:

from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter

provider = TracerProvider()
exporter = OTLPSpanExporter(endpoint="http://localhost:4317", insecure=True)
provider.add_span_processor(BatchSpanProcessor(exporter))
trace.set_tracer_provider(provider)

tracer = trace.get_tracer(__name__)

with tracer.start_as_current_span("parent-operation"):
    with tracer.start_as_current_span("child-operation"):
        print("doing work")

Install the required packages:

pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp

OpenTelemetry handles instrumentation and export, while Jaeger stores and visualizes the data.

Best Practices

  • Sampling Strategy – Configure a sampling rate that balances visibility and cost. Avoid full‑trace collection in high‑traffic environments.
  • Meaningful Tags – Add tags such as http.method, http.status_code, db.statement, and custom business keys to enrich spans.
  • Correct Context Propagation – Ensure that every service injects and extracts the SpanContext; missing propagation leads to broken traces.
  • Secure Baggage – Never place passwords, tokens, or large payloads in baggage or span tags.
  • Scalable Storage – Choose a storage backend that can grow with your data volume and set an appropriate retention policy.
  • Monitor Collector Health – The Collector is a single point of failure; consider high‑availability setups for production.

Conclusion

Jaeger provides a comprehensive solution for distributed tracing, from instrumentation to storage and visualization. By coupling it with OpenTelemetry, teams can adopt a standardized telemetry pipeline while leveraging Jaeger’s mature query and UI capabilities. Mastering the fundamentals—traces, spans, context propagation, and sampling—is essential for effective observability and performance troubleshooting in modern microservice ecosystems.