LLM observability with OpenTelemetry: trace OpenAI, Claude and Gemini calls in a Node.js backend

LLM observability is the part of an AI feature most teams add last: a user reports that the "summarize reviews" button is slow or wrong, and the only record is a line in the provider's usage dashboard. OpenTelemetry fixes that with a trace for every model call, nested under the request that caused it, carrying the model, tokens, latency and error. This month it got much easier. On October 14, 2026 Anthropic launched OpenTelemetry export in beta, and the Claude TypeScript SDK now creates spans on its own once your app registers a tracer provider.

This guide is for developers running a Node.js backend behind an AI extension or SaaS product. It covers what to record, the OpenTelemetry GenAI conventions that make spans readable in any backend, the new Claude SDK and API tracing, a small wrapper for OpenAI and Gemini calls, the token-counting traps, and what not to put in a span. Every code sample was run on October 18, 2026 against a local mock server with @anthropic-ai/sdk 0.135.0, openai 7.38.0 and @opentelemetry/sdk-trace-node 2.12.0.

Key takeaways

• Use OpenTelemetry for LLM observability: one CLIENT span per model call, named {operation} {model}, with gen_ai.* attributes. Any OTLP backend can store it.
• The Claude TypeScript SDK emits spans automatically from version 0.134.0 (October 14, 2026). Register a tracer provider and you are done.
• The official OpenAI instrumentation for Node.js supports openai below version 7, so on the current SDK you add spans yourself. Gemini also needs a wrapper.
• gen_ai.usage.input_tokens should include cached tokens. Anthropic's raw usage.input_tokens does not, so never mix raw provider numbers with span attributes.
• Prompts and responses stay out of spans by default. Keep it that way for user content such as reviews and emails.

What LLM observability should answer#

A provider dashboard tells you how many tokens an API key used. It cannot tell you which feature, which user or which request was slow. For a product built on model calls, LLM observability should answer five questions:

1. Which requests call a model, and how long does each call take? Latency per model and per feature, at the median and the 95th percentile.
2. How many tokens did each call use, and how many came from the cache? This is where cost per feature comes from.
3. Which calls failed, and why? Rate limits, overloads, timeouts, content refusals and bad requests are different problems.
4. Which model actually answered? The model you request and the snapshot that responds are not always the same string.
5. What happened around the call? A slow model call inside a slow database query is a different fix from a slow model call alone.

Tracing answers all five, because a trace keeps the model call inside the request that caused it. If you only need per-user cost, our guide to tracking LLM cost per user builds a usage ledger without tracing. The two work well together: the ledger for billing, the traces for debugging.

The OpenTelemetry GenAI conventions in five minutes#

OpenTelemetry is a vendor-neutral standard for traces, metrics and logs. The part that matters here is the GenAI semantic conventions, which now live in their own repository. They are still marked Development, so attribute names can change, but providers and observability vendors already use them. The client inference spec says:

• Span name should be {gen_ai.operation.name} {gen_ai.request.model}, for example chat claude-haiku-5-5.
• Span kind should be CLIENT.
• Required attributes are gen_ai.operation.name (such as chat or generate_content) and gen_ai.provider.name.

The attributes worth setting on every call:

Two notes in the spec save a lot of confusion. Input tokens "SHOULD include all types of input tokens, including cached tokens", and reasoning tokens "SHOULD be included in gen_ai.usage.output_tokens". A dashboard that sums input_tokens then shows the real volume, and the cache attributes show how much of it was cheap.

For providers, the list of well-known values includes anthropic, openai, gcp.gemini (the AI Studio endpoint at generativelanguage.googleapis.com), gcp.vertex_ai, aws.bedrock and azure.ai.openai. Use those strings exactly, because backends key their LLM views on them.

Claude: SDK spans and API spans#

Anthropic's OpenTelemetry overview describes two sources of spans, and you can use either or both.

SDK tracing: three lines of setup#

The TypeScript SDK added "native support for OpenTelemetry tracing" in version 0.134.0, released October 14, 2026. The changelog is explicit: "If your app registers a global tracer provider, the SDK will create spans with it after upgrading." Requests also get W3C trace context headers.

// tracing.js: load this before anything else in the process.
import { NodeTracerProvider, BatchSpanProcessor } from '@opentelemetry/sdk-trace-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';

const provider = new NodeTracerProvider({
  spanProcessors: [new BatchSpanProcessor(new OTLPTraceExporter())],
});
// register() also installs the context manager, so SDK spans nest under yours.
provider.register();

The exporter reads OTEL_EXPORTER_OTLP_ENDPOINT and OTEL_EXPORTER_OTLP_HEADERS, so the same code sends to a self-hosted Collector, Grafana, Honeycomb, Datadog, Sentry, PostHog or any other OTLP backend. Anthropic's docs list a dozen partners with their own setup pages.

With that in place, a normal call needs nothing else:

import Anthropic from '@anthropic-ai/sdk';
import { trace } from '@opentelemetry/api';

const anthropic = new Anthropic();
const tracer = trace.getTracer('reviews-service');

export async function summarizeReviews(listingId, reviews) {
  return tracer.startActiveSpan('summarize_reviews', async (span) => {
    span.setAttribute('app.listing_id', listingId);
    try {
      return await anthropic.messages.create({
        model: 'claude-haiku-5-5',
        max_tokens: 800,
        messages: [{ role: 'user', content: buildPrompt(reviews) }],
      });
    } finally {
      span.end();
    }
  });
}

In our run, the SDK produced an anthropic.messages.create span of kind CLIENT under summarize_reviews, with gen_ai.provider.name: anthropic, gen_ai.request.model, gen_ai.response.id, gen_ai.response.model, the usage attributes and anthropic.message.stop_reason. A request that came back with HTTP 429 got an error status and error.type: rate_limit_error, which is the provider's own error type and makes a good dashboard filter. Our guide to handling LLM API rate limits covers what to do once you can see them.

Client options from the SDK setup page:

• openTelemetry: { traces: { contentMode: 'content' } } adds prompts and responses to spans. The default is metadata_only.
• openTelemetry: false, or ANTHROPIC_OPEN_TELEMETRY=false, turns spans off for a client.
• conversationId writes gen_ai.conversation.id. The docs warn that it "is exported in both content modes, so don't put personal data or secrets in it."

The SDK page also warns that during the beta, "a new version of the SDK can rename a span, an attribute, an option, or an environment variable." Pin the SDK version, and check your dashboards after each upgrade.

API tracing: spans from Anthropic's side#

The second source is the API itself. An organization admin adds an OTLP/HTTP destination in the Claude Console, and the API sends spans for each request, each model run inside it and each server tool such as web search. The API tracing page lists the requirements: the admin role, an HTTPS OTLP/HTTP endpoint on port 443 that accepts protobuf, and credentials for it. If you use the SDK as well, the traceparent header it sends lets the API spans nest under the SDK span, so one trace shows your handler, the client call and the server-side model run.

Delivery of API spans is best effort, and the docs say so plainly: exported spans "aren't guaranteed to be a complete record of your API usage." Use them for debugging and trends, and the usage reports for billing.

Count Claude tokens once#

With both sources on, the same request has a CLIENT span from the SDK and a SERVER span from the API, both named anthropic.messages.create, and both carry token counts. Anthropic's counting rule is to sum one source per request: the API spans when they exist, otherwise the SDK spans. Sum either the anthropic.messages.create server spans or their anthropic.model_request children, never both.

The other trap is the cache. In our test, the mock API returned input_tokens: 12 and cache_read_input_tokens: 100, and the SDK span reported gen_ai.usage.input_tokens: 112. That matches the convention, but it does not match the raw usage.input_tokens in the response body. If your cost code reads the response and your dashboard reads the span, they will disagree by exactly the cached tokens. Our prompt caching guide has the per-provider field names.

OpenAI and Gemini: a small span wrapper#

The OpenTelemetry project publishes @opentelemetry/instrumentation-openai, but its README lists support for "openai versions >=4.19.0 <7". The current SDK is 7.x, so on a fresh install it will not patch anything. Gemini's @google/genai SDK does not emit spans either. Until that changes, a 40-line wrapper gives you spans in the same shape as the Claude ones:

// llm-span.js
import { trace, SpanKind, SpanStatusCode } from '@opentelemetry/api';

const tracer = trace.getTracer('ai-backend');

export async function withLlmSpan(meta, call, readResponse) {
  const { provider, operation, model, serverAddress, attributes = {} } = meta;
  return tracer.startActiveSpan(
    `${operation} ${model}`,
    {
      kind: SpanKind.CLIENT,
      attributes: {
        'gen_ai.operation.name': operation,
        'gen_ai.provider.name': provider,
        'gen_ai.request.model': model,
        'server.address': serverAddress,
        ...attributes,
      },
    },
    async (span) => {
      try {
        const response = await call();
        span.setAttributes(readResponse(response));
        return response;
      } catch (err) {
        span.recordException(err);
        span.setStatus({ code: SpanStatusCode.ERROR, message: String(err?.message ?? err) });
        span.setAttribute('error.type', err?.status ? String(err.status) : (err?.name ?? 'Error'));
        throw err;
      } finally {
        span.end();
      }
    },
  );
}

Each provider then needs a reader that maps its usage fields onto the convention:

// OpenAI Chat Completions: prompt_tokens already includes cached tokens.
export function readOpenAIChat(res) {
  return {
    'gen_ai.response.id': res.id,
    'gen_ai.response.model': res.model,
    'gen_ai.response.finish_reasons': res.choices.map((c) => c.finish_reason),
    'gen_ai.usage.input_tokens': res.usage?.prompt_tokens ?? 0,
    'gen_ai.usage.output_tokens': res.usage?.completion_tokens ?? 0,
    'gen_ai.usage.cache_read.input_tokens': res.usage?.prompt_tokens_details?.cached_tokens ?? 0,
    'gen_ai.usage.reasoning.output_tokens': res.usage?.completion_tokens_details?.reasoning_tokens ?? 0,
  };
}

// Gemini generateContent: thoughts are reported apart from candidates.
export function readGemini(res) {
  const u = res.usageMetadata ?? {};
  return {
    'gen_ai.response.id': res.responseId ?? '',
    'gen_ai.response.model': res.modelVersion ?? '',
    'gen_ai.response.finish_reasons': (res.candidates ?? []).map((c) => c.finishReason ?? 'unknown'),
    'gen_ai.usage.input_tokens': u.promptTokenCount ?? 0,
    'gen_ai.usage.output_tokens': (u.candidatesTokenCount ?? 0) + (u.thoughtsTokenCount ?? 0),
    'gen_ai.usage.cache_read.input_tokens': u.cachedContentTokenCount ?? 0,
    'gen_ai.usage.reasoning.output_tokens': u.thoughtsTokenCount ?? 0,
  };
}

Gemini's usage metadata counts thoughtsTokenCount separately from candidatesTokenCount, so the reader adds them to follow the convention's rule that reasoning belongs in output tokens. promptTokenCount already includes cached content.

Using it looks like this:

import OpenAI from 'openai';
import { withLlmSpan, readOpenAIChat } from './llm-span.js';

const openai = new OpenAI();

const res = await withLlmSpan(
  {
    provider: 'openai',
    operation: 'chat',
    model: 'gpt-5.6-terra',
    serverAddress: 'api.openai.com',
    attributes: { 'app.feature': 'review_summary' },
  },
  () => openai.chat.completions.create({ model: 'gpt-5.6-terra', messages }),
  readOpenAIChat,
);

Swap the model string for whichever model you use. In our run against a mock server, this produced a chat {model} span with the usage attributes filled in, and a thrown error with status: 429 produced an error span with error.type: "429". If you use the Responses API instead of Chat Completions, write a third reader: its usage object has input_tokens, output_tokens, input_tokens_details.cached_tokens and output_tokens_details.reasoning_tokens.

Two details from testing are worth knowing. First, the wrapper does not add a traceparent header to the provider request, while the Claude SDK does. That only matters for providers that build server-side traces from it. Second, for streaming calls, end the span when the stream finishes, not when the first chunk arrives, and read usage from the final event. Our guide to streaming AI responses in a Chrome extension shows where that final event appears for each provider.

Add your own attributes, carefully#

The GenAI attributes describe the model call. Your own attributes describe the product, and they are what make traces useful in a dashboard:

• app.feature: review_summary, reply_draft, tag_reviews. One value per AI feature.
• app.plan: trial, pro, business. Cost per plan is a pricing input.
• app.user_id: a pseudonymous internal ID, never an email address.
• app.prompt_version: bump it when you change a prompt, so you can compare before and after. The conventions also define gen_ai.prompt.name and gen_ai.prompt.version for this.

Keep values low-cardinality where you plan to group by them. A user ID is fine as an attribute you filter on, but do not put it in the span name.

Keep personal data out of your traces#

Content capture is the setting to think hardest about. Store reviews, support tickets and emails contain names, addresses and sometimes order numbers. Once a prompt is in a span, it is in your observability vendor's storage under their retention rules, and Anthropic's docs make the same point: the platform "stores this content under its own access and retention settings."

A sensible default:

1. Leave content mode at metadata only in production. Tokens, latency, model and errors are enough for most debugging.
2. Turn on content for a sampled share of internal or test traffic when you need to debug prompt quality, and send it to a project with short retention.
3. Never put secrets in attributes. API keys belong in your backend, as our guide to hiding an API key in a Chrome extension explains, and they should not appear in spans either.
4. Mention tracing in your privacy policy if you capture content, because user text now goes to one more processor.

Turn traces into dashboards#

Once spans arrive, five charts cover most needs:

Set one alert on the error rate and one on the daily token total per feature. Everything else can stay a chart.

A rollout checklist#

1. Install @opentelemetry/sdk-trace-node and an OTLP exporter, and load the tracing file first.
2. Upgrade @anthropic-ai/sdk to 0.134.0 or later and pin it.
3. Wrap OpenAI and Gemini calls with withLlmSpan and one reader per API.
4. Add app.feature and app.prompt_version to every call.
5. Keep content mode at metadata only.
6. Optionally add a Claude Console destination for API spans, and count tokens from one source.
7. Build the five charts, then add two alerts.

Where store data fits#

Traces tell you how your AI feature behaves. They do not tell you how users react to it in the store. When you ship an AI feature in an extension, the reaction shows up as reviews and rating changes on the Chrome Web Store, Edge Add-ons and Firefox Add-ons. Extenify is a chrome extension analytics dashboard that tracks users, ratings, reviews and versions for your listings and your competitors' every day, and sends extension review monitoring alerts by email, Slack, Discord, Telegram or Teams. Pair a release trace with the reviews that arrive the following week, as in our checklist of what to watch after an extension release. Plans start with a free trial, and the pricing page has the details. If you turn those reviews into embeddings to find recurring complaints, our guide to pgvector in Node.js shows the database side, and the embedding calls can go through the same span wrapper with operation: 'embeddings'.

Frequently asked questions#

What is LLM observability?#

It is the practice of recording every model call your application makes, with its latency, token usage, model, result and error, in the context of the request that triggered it. Traces are the usual format, and OpenTelemetry is the standard most tools accept.

Does the Claude SDK support OpenTelemetry?#

Yes. The TypeScript SDK added native OpenTelemetry tracing in version 0.134.0 on October 14, 2026, and the Python, C#, Go, Java, PHP and Ruby SDKs emit spans too. It uses your registered tracer provider. Separately, the Claude API can export its own spans to an OTLP endpoint that an admin configures in the Console. Both are in beta.

Is there OpenTelemetry instrumentation for the OpenAI Node.js SDK?#

There is an official contrib package, @opentelemetry/instrumentation-openai, but its README lists support for openai versions from 4.19.0 up to, but not including, 7. On the 7.x SDK, add spans yourself as shown above, or check the package's README for newer support before you rely on it.

Should I log prompts and responses in traces?#

Not by default. Metadata is enough for latency, cost and error debugging, and prompts often contain personal data. Capture content only for sampled or internal traffic, in a project with short retention.

Are the OpenTelemetry GenAI conventions stable?#

No. They are marked Development, so attribute names can still change. Anthropic also warns that its span schema can change during the beta. Pin library versions and review dashboards after upgrades.

Summary#

LLM observability with OpenTelemetry means one CLIENT span per model call with gen_ai.* attributes, nested in your own request spans and exported over OTLP to any backend. Since October 14, 2026 the Claude TypeScript SDK does this by itself once you register a tracer provider, and the Claude API can send server-side spans too. For OpenAI on the 7.x SDK and for Gemini, a small wrapper produces the same shape. Count tokens from one source, remember that convention input tokens include the cache, keep content out of spans by default, and build a handful of charts for latency, tokens, cache hits, errors and truncation.

Image credits#

• Cover photo: Lovell Telescope at Jodrell Bank Observatory by iantresman on the WordPress Photo Directory, CC0, cropped from a panorama.