Unified Tracing with OpenTelemetry GenAI Semantic Conventions

Module 7 logged security and cost events. This module instruments quality and performance, and the first question is what schema to log them in. OpenTelemetry’s GenAI semantic conventions answer that: a CNCF-backed, vendor-neutral standard for exactly this kind of telemetry, and for a gateway routing across many providers, using it instead of an ad hoc schema is the difference between one dashboard that works for every provider and a different query for each one.

The Core Attributes

The GenAI semantic conventions define a standard set of span attributes for any LLM call:

Recommended span naming follows the pattern {gen_ai.operation.name} {gen_ai.request.model}, e.g. chat gpt-4o, which is what makes traces from completely different providers instantly comparable in a trace viewer without translating naming schemes in your head.

Emitting a Real Span

from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode

tracer = trace.get_tracer("ai-gateway")

def traced_completion(gateway_client, model: str, messages: list[dict], provider_hint: str):
    with tracer.start_as_current_span(f"chat {model}") as span:
        span.set_attribute("gen_ai.operation.name", "chat")
        span.set_attribute("gen_ai.request.model", model)
        span.set_attribute("gen_ai.provider.name", provider_hint)  # known once routing (Module 4) resolves it

        try:
            response = gateway_client.chat.completions.create(model=model, messages=messages)
        except Exception as e:
            span.set_status(Status(StatusCode.ERROR, str(e)))
            raise

        usage = response.usage
        span.set_attribute("gen_ai.usage.input_tokens", usage.prompt_tokens)
        span.set_attribute("gen_ai.usage.output_tokens", usage.completion_tokens)
        span.set_attribute(
            "gen_ai.response.finish_reasons",
            [response.choices[0].finish_reason],
        )
        return response

Every field set here is a real, spec-defined GenAI attribute name, not an invented one, which is exactly what lets this span show up correctly in any OpenTelemetry-compatible backend (Jaeger, Grafana Tempo, an OTLP-compatible SaaS) without custom parsing logic on the backend side.

Why This Matters More for a Gateway Than a Single-Provider App

A single-provider application’s tracing only ever needs to describe one provider’s shape, custom field names would work fine, if awkwardly, since there’s nothing to compare them against. A gateway is different: the exact same logical request (model="production-chat") can be served by Anthropic today and a self-hosted vLLM deployment tomorrow, depending on the routing decisions from Module 4, retry and fallback behavior from Module 5, or the task classification from Lesson 12. A dashboard built on standard gen_ai.* attributes handles that transparently, a query grouping by gen_ai.provider.name just works regardless of how many providers are in the model_list; a dashboard built on provider-specific field names would need new logic every time a new provider is added.

flowchart LR
    Req["Gateway request span:\nchat production-chat"] --> Route["Routing decision\n(Module 4)"]
    Route -->|"served by Anthropic"| SpanA["gen_ai.provider.name: anthropic\ngen_ai.request.model: claude-sonnet-4-5"]
    Route -->|"served by self-hosted vLLM"| SpanB["gen_ai.provider.name: vllm\ngen_ai.request.model: llama-3.1-8b"]
    SpanA --> Dashboard["One dashboard,\none query,\ngen_ai.* attributes"]
    SpanB --> Dashboard

    style Req fill:#4338CA,stroke:#4338CA,color:#ffffff,rx:8px,ry:8px
    style SpanA fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style SpanB fill:#EEF0F7,stroke:#6366F1,color:#0F172A,rx:8px,ry:8px
    style Dashboard fill:#F7F8FC,stroke:#0D9488,color:#0F172A,rx:8px,ry:8px

One caveat worth flagging: the GenAI conventions are explicitly still under active development (Development stability, per OpenTelemetry’s own project status), so attribute names and structure can evolve. The practical response is to isolate the mapping from a raw provider response to gen_ai.* attributes in one place in the gateway’s codebase, much like the adapter pattern from Lesson 6, so a future spec revision means updating one function, not every call site that emits a span. With unified traces flowing, Lesson 23 uses this same data to answer a harder question than “what happened”: whether the answers themselves are still good.