Tracing and cost tracking for LLM apps

OpenTelemetry GenAI spans from AI SDK into self-hosted Langfuse: per-request traces, tokens, cost, latency.

Run it as a prompt

Paste this into any AI agent, or fetch it: curl -s https://uplift.page/api/v1/prompts/llm-tracing-cost-tracking/raw

prompt.md

# Tracing and cost tracking for LLM apps

Every LLM request produces one trace with model, tokens, cost, tool calls, and user id, visible within 10 s; dashboards answer cost per user per day and p95 latency per feature; tracing adds under 5 ms p95 per request.

## Recommended stack

- **OpenTelemetry JS (@opentelemetry/sdk-node)** - Span pipeline and OTLP/HTTP export. Vendor-neutral, so the backend can change without touching app code.
- **AI SDK 7 telemetry (OpenTelemetry integration)** - Emits invoke_agent, chat, and execute_tool spans with gen_ai.* attributes. Follows the OpenTelemetry GenAI semantic conventions natively, with recordInputs/recordOutputs switches.
- **Langfuse v4, self-hosted** - Trace storage, cost calculation, dashboards. Accepts OTLP over HTTP at /api/public/otel and runs on your own Postgres, ClickHouse, Redis, and S3; open source and self-hostable after joining ClickHouse in Jan 2026.
- **Your existing alerting (Grafana, Prometheus, or similar)** - Cost and error alerts. Keeps paging in one place instead of a second alert system.

## Build steps

1. Deploy Langfuse with its Docker Compose or Helm setup (web, worker, Postgres, ClickHouse, Redis/Valkey, S3-compatible storage) and create project keys.
2. In instrumentation.ts start NodeSDK with new OTLPTraceExporter({ url: `<langfuse-host>/api/public/otel/v1/traces`, headers: { Authorization: 'Basic ' + base64(publicKey:secretKey), 'x-langfuse-ingestion-version': '4' } }) in a batch span processor (or set OTEL_EXPORTER_OTLP_ENDPOINT=<langfuse-host>/api/public/otel).
3. `npm i @ai-sdk/otel`, call registerTelemetry(new OpenTelemetry()) from 'ai' once at startup, and pass telemetry: { functionId: '<feature>' } per call (it becomes gen_ai.agent.name).
4. Set user.id, session.id, feature and langfuse.release on the root span (Langfuse also reads langfuse.user.id / langfuse.session.id); pass traceparent through queues and background jobs.
5. Decide content capture per feature: set recordInputs and recordOutputs to false where prompts contain personal data, or redact in a span processor.
6. Configure model prices in Langfuse and build dashboards: cost per user per day, p95 latency by feature, tool error rate.
7. Alert when daily cost exceeds 1.5x the 7-day average or tool error rate passes 5%.

## Watch out for

- Without the ingestion-version header, OTLP traces can show up to 10 minutes late.
- Inputs and outputs are recorded by default, so personal data lands in traces unless you turn it off or redact.
- Spans from queue workers become orphans when trace context is not propagated.
- Serverless functions freeze before export; call forceFlush at the end of the request or use waitUntil. Langfuse accepts HTTP only, not gRPC.

## Definition of done

- One chat request shows as one trace with nested chat and execute_tool spans and token counts.
- The dashboard shows cost per user for the last 24 hours.
- A feature with recordInputs false shows no prompt text in its traces.
- A load test shows under 5 ms p95 added latency with tracing on.

The full prompt

Recommended stack

OpenTelemetry JS (@opentelemetry/sdk-node) - Span pipeline and OTLP/HTTP export. Vendor-neutral, so the backend can change without touching app code.
AI SDK 7 telemetry (OpenTelemetry integration) - Emits invoke_agent, chat, and execute_tool spans with gen_ai.* attributes. Follows the OpenTelemetry GenAI semantic conventions natively, with recordInputs/recordOutputs switches.
Langfuse v4, self-hosted - Trace storage, cost calculation, dashboards. Accepts OTLP over HTTP at /api/public/otel and runs on your own Postgres, ClickHouse, Redis, and S3; open source and self-hostable after joining ClickHouse in Jan 2026.
Your existing alerting (Grafana, Prometheus, or similar) - Cost and error alerts. Keeps paging in one place instead of a second alert system.

Build steps

Deploy Langfuse with its Docker Compose or Helm setup (web, worker, Postgres, ClickHouse, Redis/Valkey, S3-compatible storage) and create project keys.
In instrumentation.ts start NodeSDK with new OTLPTraceExporter({ url: <langfuse-host>/api/public/otel/v1/traces, headers: { Authorization: 'Basic ' + base64(publicKey:secretKey), 'x-langfuse-ingestion-version': '4' } }) in a batch span processor (or set OTEL_EXPORTER_OTLP_ENDPOINT=/api/public/otel).
npm i @ai-sdk/otel, call registerTelemetry(new OpenTelemetry()) from 'ai' once at startup, and pass telemetry: { functionId: '' } per call (it becomes gen_ai.agent.name).
Set user.id, session.id, feature and langfuse.release on the root span (Langfuse also reads langfuse.user.id / langfuse.session.id); pass traceparent through queues and background jobs.
Decide content capture per feature: set recordInputs and recordOutputs to false where prompts contain personal data, or redact in a span processor.
Configure model prices in Langfuse and build dashboards: cost per user per day, p95 latency by feature, tool error rate.
Alert when daily cost exceeds 1.5x the 7-day average or tool error rate passes 5%.

Watch out for

Without the ingestion-version header, OTLP traces can show up to 10 minutes late.
Inputs and outputs are recorded by default, so personal data lands in traces unless you turn it off or redact.
Spans from queue workers become orphans when trace context is not propagated.
Serverless functions freeze before export; call forceFlush at the end of the request or use waitUntil. Langfuse accepts HTTP only, not gRPC.

Definition of done

One chat request shows as one trace with nested chat and execute_tool spans and token counts.
The dashboard shows cost per user for the last 24 hours.
A feature with recordInputs false shows no prompt text in its traces.
A load test shows under 5 ms p95 added latency with tracing on.

Served from the uplift.page library and refreshed within 5 minutes of every update.