OpenTelemetry

Distributed tracing with Jaeger, Tempo, or Datadog

OpenTelemetry Tracing

Export distributed traces to Jaeger, Grafana Tempo, Datadog, or any OTLP-compatible backend. Requires an Enterprise license.

Enable

settings:
  otel:
    enabled: true
    endpoint: "tempo:4317"        # OTLP gRPC endpoint
    insecure: true                # set to false for TLS
    sample_rate: 1.0              # 1.0 = trace every request

What’s Traced

Attributes include: model name, provider, status code, token counts, duration, and request ID.

Trace Propagation

Trace context (traceparent header) is propagated to upstream providers for end-to-end distributed tracing. If your LLM provider supports OpenTelemetry, you get a complete trace from client -> VoidLLM -> provider.

Log Correlation

When tracing is active, every log line automatically includes trace_id and span_id. This makes it easy to find related logs for a specific trace in tools like Grafana Loki or Elasticsearch.

Example log entry:

{
  "time": "2026-04-01T12:00:00Z",
  "level": "INFO",
  "msg": "proxy request",
  "trace_id": "abc123...",
  "span_id": "def456...",
  "model": "gpt-4o",
  "status": 200
}

Sample Rate

For high-traffic deployments, reduce the sample rate to control tracing volume:

settings:
  otel:
    sample_rate: 0.1    # trace 10% of requests

A sample rate of 1.0 traces every request. 0.1 traces 10%. 0.0 disables tracing while keeping the configuration in place.

Backend Setup Examples

Grafana Tempo

settings:
  otel:
    enabled: true
    endpoint: "tempo:4317"
    insecure: true

Jaeger

settings:
  otel:
    enabled: true
    endpoint: "jaeger:4317"
    insecure: true

Datadog

settings:
  otel:
    enabled: true
    endpoint: "datadog-agent:4317"
    insecure: true

Datadog requires the OTLP ingest to be enabled on the Datadog Agent.