OpenInference vs OpenTelemetry GenAI: LLM Tracing Spec
Source: mortalapps.com- OpenInference is purpose-built for AI workloads, providing deeply nested tracing structures for RAG, evaluations, and multi-agent systems.
- OpenTelemetry GenAI is a generalized, official CNCF-backed extension designed to standardize basic LLM attributes (tokens, models) across all of APM.
- OpenInference has a richer ecosystem of auto-instrumentors for frameworks like LlamaIndex, LangChain, and DSPy.
- OpenTelemetry GenAI offers seamless integration with legacy enterprise APM backends but currently lacks deep semantic definitions for RAG-specific steps.
- The choice dictates whether your telemetry sits in an AI-specific evaluation platform or a unified corporate APM dashboard.
The Contenders
OpenInference
OpenInference is an open-source semantic convention and set of auto-instrumentation libraries designed specifically for LLM-based applications, originally created by Arize AI. It extends OpenTelemetry by defining a rich, highly specialized set of attributes for tracing generative AI tasks, including retrieval-augmented generation (RAG), vector search parameters, prompt templates, agent tool calls, and LLM evaluations. The design philosophy of OpenInference is AI-first: it assumes that LLM systems are fundamentally different from traditional microservices and therefore require deep, nested metadata to facilitate effective debugging and evaluation.
Its primary strength lies in its extensive ecosystem of auto-instrumentation wrappers for popular AI frameworks like LlamaIndex, LangChain, CrewAI, and DSPy, allowing developers to instrument complex pipelines with minimal code. However, a known limitation is its tight alignment with the Arize/Phoenix ecosystem; while it outputs standard OTel traces, visualizing its rich, specialized attributes (such as retrieval chunk contents or LLM-as-a-judge scores) often requires specialized backends, making it less plug-and-play with standard APM platforms like Dynatrace or Datadog without custom mapping.
OpenTelemetry GenAI Semantic Conventions
The OpenTelemetry GenAI Semantic Conventions represent the official, community-driven effort by the Cloud Native Computing Foundation (CNCF) to standardize LLM observability under the broader OpenTelemetry umbrella. It defines a baseline set of semantic conventions for generative AI operations, focusing on core metrics such as prompt/completion tokens, model names, temperature, top_p, and basic latency metrics. The design philosophy is unified and vendor-neutral: LLM calls are treated as external dependencies (similar to database queries or HTTP calls) that should seamlessly integrate into an enterprise's existing, unified APM infrastructure.
Its primary strength is its universal compatibility and backing by the entire CNCF ecosystem, ensuring that any OTel-compliant backend (such as Datadog, New Relic, Honeycomb, or Grafana) can ingest and visualize these metrics out of the box without proprietary plugins. Its major limitation is its current lack of depth for advanced agentic and RAG workflows; it does not natively define standardized semantic conventions for complex retrieval steps, rerankers, multi-agent orchestrations, or online evaluation scores, requiring developers to fall back on custom span attributes for these advanced use cases.
OpenInference vs OpenTelemetry GenAI Semantic Conventions: At a Glance
| Dimension | OpenInference | OpenTelemetry GenAI Semantic Conventions |
|---|---|---|
| Architecture paradigm | AI-Native / Specialized | General-purpose APM Extension |
| Learning curve | Low (Auto-instrumentation) | Moderate (Requires OTel SDK setup) |
| Determinism | ✓ (Strict schema for AI structures) | ✓ (Strict spec schema) |
| State management | Context-propagated trace state | W3C Trace Context / Baggage |
| Enterprise readiness | High (Via Arize platform) | Very High (CNCF Standard) |
| Primary use case | Complex RAG and Agentic Workflows | Unified LLM Gateway & APM Monitoring |
| Maintenance overhead | Low (Maintained SDKs per framework) | Moderate (Manual span attribute mapping) |
| Best For | Deep AI engineering & evaluations | Corporate compliance & multi-vendor APM |
Why This Decision Matters
In the rapidly evolving landscape of LLM observability, selecting the right instrumentation standard is a critical architectural decision. Comparing openinference vs opentelemetry genai highlights a fundamental tension: should you adopt an AI-native specification built specifically for complex agentic pipelines, or align with the broader, vendor-neutral CNCF OpenTelemetry ecosystem? The cost of making the wrong choice is high. Re-instrumenting a production-grade multi-agent system or RAG application involves modifying thousands of lines of context-propagation code, updating collector configurations, and rebuilding monitoring dashboards.
For production engineers, observability is not just about logging tokens and latency; it is about debugging non-deterministic agent loops, evaluating retrieval quality (RAG), and tracking tool execution chains. OpenInference (incubated by Arize AI) provides out-of-the-box semantic definitions for these complex AI concepts. Conversely, OpenTelemetry (OTel) GenAI Semantic Conventions seek to standardize LLM metrics across the entire software engineering industry, ensuring that LLM calls are monitored alongside database queries and HTTP requests in traditional APM tools. Choosing between them determines whether your team will spend its engineering cycles building custom collectors and dashboards in Datadog or managing a separate, specialized AI evaluation backend.
Head-to-Head Analysis
OpenInference provides a highly granular schema designed specifically for AI workflows. It defines distinct span types for RETRIEVAL, RERANK, EMBEDDING, TOOL, and LLM. For example, a RETRIEVAL span in OpenInference includes standardized attributes for query strings, retrieved document contents, document IDs, and similarity scores. This level of detail is crucial for computing RAG metrics like faithfulness and relevancy.
In contrast, the OpenTelemetry GenAI Semantic Conventions focus primarily on the core LLM exchange. While it excels at standardizing basic parameters like gen_ai.request.model, gen_ai.response.tokens, and gen_ai.request.temperature, it does not yet have standardized, deeply nested schemas for RAG or multi-agent orchestrations. Standardizing these complex structures within the CNCF governance process takes time, meaning engineers using OTel GenAI must often define their own custom namespaces to capture the same level of detail that OpenInference provides out of the box. Winner: OpenInference.
When it comes to developer velocity, OpenInference offers a massive advantage through its pre-built auto-instrumentation packages. Under the openinference-instrumentation umbrella, there are dedicated, actively maintained packages for LlamaIndex, LangChain, OpenAI, Anthropic, DSPy, and Guardrails. Enabling tracing is typically a matter of calling a single register() function, which automatically hooks into the framework's internal event systems to emit perfectly structured spans.
OpenTelemetry GenAI relies on a mix of official OTel instrumentations (such as the official OTel OpenAI instrumentation) and manual span creation. Because the GenAI conventions are newer and broader, many framework-specific instrumentations are either in their infancy or require manual mapping. If you are building an agent using a framework like LangGraph or CrewAI, OpenInference will automatically capture the internal state transitions and tool execution graphs, whereas OTel GenAI will require significant manual boilerplate code to achieve the same visibility. Winner: OpenInference.
This is where OpenTelemetry GenAI holds a massive enterprise advantage. Because it is the official CNCF standard, major APM vendors like Datadog, Dynatrace, New Relic, Splunk, and Honeycomb are actively building native dashboards, alerts, and cost-tracking tools specifically tailored to the gen_ai.* semantic conventions. If your organization already has a centralized observability team and a multi-million dollar APM contract, using OTel GenAI allows you to route your LLM telemetry directly into your existing infrastructure with zero changes to your collector pipelines.
OpenInference traces are technically standard OTel traces under the hood, but they use the openinference.* namespace. While you can send these traces to any OTel collector, general-purpose APM vendors do not natively understand OpenInference's custom span attributes. To visualize them effectively, you must either build custom dashboards from scratch in your APM or route the traces to an AI-native observability backend like Arize Phoenix, LangFuse, or Arize Enterprise. This introduces architectural fragmentation and vendor lock-in to specialized AI platforms. Winner: OpenTelemetry GenAI.
OpenTelemetry is a CNCF Graduated project with strict governance, backward compatibility guarantees, and a slow, consensus-driven process for modifying schemas. While this ensures long-term stability and protection against breaking changes, it makes the specification slow to adapt to new AI engineering paradigms (such as multi-modal inputs, computer use, or real-time audio streams).
OpenInference, being managed primarily by Arize AI and a focused community of AI engineers, moves at the speed of the AI ecosystem. When a new framework or model capability emerges, OpenInference can update its semantic conventions and release new auto-instrumentation packages within days. However, this agility comes with a higher risk of breaking changes and a specification that is less formally audited than CNCF standards. For fast-moving startups, OpenInference’s speed is an asset; for conservative enterprises, OTel's slow, stable governance is preferred. Winner: Tie (depends on organizational risk tolerance).
Same Task, Two Approaches
This implementation uses OpenInference auto-instrumentation for OpenAI along with the Arize Phoenix tracer provider to capture rich, AI-specific span attributes. The OpenInference SDK automatically maps the OpenAI API request and response to the openinference.span.attribute namespace, capturing detailed token counts and input/output parameters.
import os
from openai import OpenAI
from opentelemetry import trace as otel_trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
# Set up OpenTelemetry tracer provider
provider = TracerProvider()
processor = SimpleSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
otel_trace.set_tracer_provider(provider)
# Instrument OpenAI with OpenInference
OpenAIInstrumentor().instrument()
# Initialize OpenAI client (API key fetched from environment)
api_key = os.environ.get("OPENAI_API_KEY", "mock-key-for-testing")
client = OpenAI(api_key=api_key)
# Perform a simple LLM call
try:
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain OpenInference in one sentence."}
],
temperature=0.2,
max_tokens=50
)
print("LLM Response:", response.choices[0].message.content)
except Exception as e:
print("Call failed (expected if mock key is used):", e)This implementation uses the official OpenTelemetry GenAI Semantic Conventions to manually instrument an LLM call. It demonstrates how to explicitly map parameters to the standard CNCF gen_ai attributes, ensuring total compatibility with any standard OTel collector and APM backend.
import os
import time
from openai import OpenAI
from opentelemetry import trace as otel_trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor, ConsoleSpanExporter
# Set up OpenTelemetry tracer provider
provider = TracerProvider()
processor = SimpleSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
otel_trace.set_tracer_provider(provider)
tracer = otel_trace.get_tracer("manual-genai-tracer")
# Initialize OpenAI client
api_key = os.environ.get("OPENAI_API_KEY", "mock-key-for-testing")
client = OpenAI(api_key=api_key)
# Manually instrument using OTel GenAI Semantic Conventions
with tracer.start_as_current_span("chat_completion") as span:
span.set_attribute("gen_ai.system", "openai")
span.set_attribute("gen_ai.request.model", "gpt-4o-mini")
span.set_attribute("gen_ai.request.temperature", 0.2)
span.set_attribute("gen_ai.request.max_tokens", 50)
try:
start_time = time.time()
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Explain OpenTelemetry GenAI in one sentence."}
],
temperature=0.2,
max_tokens=50
)
latency = time.time() - start_time
span.set_attribute("gen_ai.response.model", response.model)
span.set_attribute("gen_ai.usage.output_tokens", response.usage.completion_tokens)
span.set_attribute("gen_ai.usage.input_tokens", response.usage.prompt_tokens)
span.set_attribute("gen_ai.response.finish_reasons", [choice.finish_reason for choice in response.choices])
print("LLM Response:", response.choices[0].message.content)
except Exception as e:
span.record_exception(e)
span.set_status(otel_trace.StatusCode.ERROR, str(e))
print("Call failed (expected if mock key is used):", e)The key structural difference lies in the level of abstraction and standardization. OpenInference uses an SDK wrapper (OpenAIInstrumentor) to automatically capture and structure all parameters into a deeply rich schema under the hood, requiring zero manual span management. The OpenTelemetry GenAI implementation requires manual span creation and explicit attribute mapping using standard CNCF keys. While OpenInference is highly automated and AI-centric, the OTel GenAI approach offers complete control and strict adherence to the universal CNCF specification.
When to Choose Each
Can You Use Both?
Yes, OpenInference and OpenTelemetry GenAI can coexist within the same architecture, and in many enterprise scenarios, this hybrid approach is highly recommended. Because OpenInference is built on top of the OpenTelemetry SDK, they share the same underlying tracing infrastructure, including tracer providers, samplers, and exporters.
The most common integration pattern is to use OpenInference's auto-instrumentation libraries to capture rich, AI-specific metadata (such as retrieval documents and tool inputs) at the application level, and then use an OpenTelemetry Collector to process and transform these spans. In the OTel Collector, you can use the Transform Processor to map OpenInference attributes (e.g., llm.token.count.prompt → gen_ai.usage.input_tokens, llm.token.count.completion → gen_ai.usage.output_tokens, llm.model_name → gen_ai.request.model) to standard OpenTelemetry GenAI semantic attributes before exporting them to a general-purpose APM backend like Datadog. This allows your AI team to use specialized tools like Arize Phoenix for deep debugging and evaluations, while your central platform engineering team gets standard GenAI metrics in their global dashboards.
Migration Path
Migrating from OpenInference to OpenTelemetry GenAI Semantic Conventions (or vice versa) is a moderate engineering effort, primarily because both tools share the OpenTelemetry core SDK.
If you are migrating from OpenInference to OTel GenAI, the major change is the loss of framework-specific auto-instrumentation. You will need to replace openinference-instrumentation packages with manual span instrumentation or the emerging, official OTel instrumentation libraries. Your core trace propagation code (W3C traceparent headers) will remain completely intact, but you will need to rewrite your dashboard queries and alert rules to point to the new gen_ai.* attribute names instead of openinference.*.
The risk of migration is data loss during the transition; you will lose historical continuity in your dashboards as the schema changes. The migration of a standard microservice calling LLMs typically takes 2 to 5 engineering days. However, if you have a deeply nested RAG pipeline with custom evaluators, migrating off OpenInference will require writing custom span processors to manually inject retrieval and evaluation data into OTel spans, which can extend the timeline to several weeks.