Catch prompt regressions before you merge.
The telemetry.dev GitHub Action runs your evals on each pull request and fails the check when a score drops below your pass mark.
In the video, deleting the warranty line from the prompt takes Correctness from 100% to 75%.
The telemetry.dev GitHub Action runs your evals on each pull request and fails the check when a score drops below your pass mark.
In the video, deleting the warranty line from the prompt takes Correctness from 100% to 75%.
2
2
7
Introducing the new telemetry.dev
LLM observability, now easy on the eyes. We redesigned every page: traces, sessions, logs, alerts, MCP analytics, and setup.
Light and dark, and it works on any device.
LLM observability, now easy on the eyes. We redesigned every page: traces, sessions, logs, alerts, MCP analytics, and setup.
Light and dark, and it works on any device.
1
15
46
Ask Claude Code why your agent failed.
telemetry.dev has a read-only MCP server. Claude Code, Cursor, or ChatGPT can read your traces, errors, and costs, and work out what went wrong.
In this run, it found a refund that was converted to cents twice.
telemetry.dev has a read-only MCP server. Claude Code, Cursor, or ChatGPT can read your traces, errors, and costs, and work out what went wrong.
In this run, it found a refund that was converted to cents twice.
5
0
7
When an agent gets stuck in a loop, telemetry.dev emails you.
Turn on the Agent loop alert and it checks your traces every 5 minutes. If one costs more than $2 or runs more than 30 spans, the email names the agent and the tool it kept calling, and Investigate opens that trace.
Turn on the Agent loop alert and it checks your traces every 5 minutes. If one costs more than $2 or runs more than 30 spans, the email names the agent and the tool it kept calling, and Investigate opens that trace.
1
5
8
See what each Claude Code prompt costs on telemetry.dev.
With Claude Code's tracing beta on, each prompt arrives as a trace with every model request and tool call. This one fixed two failing tests for $0.13 at API prices.
Setup guide: telemetry.dev/docs/integrati…
With Claude Code's tracing beta on, each prompt arrives as a trace with every model request and tool call. This one fixed two failing tests for $0.13 at API prices.
Setup guide: telemetry.dev/docs/integrati…
0
2
3
It's live. Add telemetry.dev as an OpenTelemetry destination in the Claude Console, no code changes.
Setup guide: telemetry.dev/docs/integrati…
Setup guide: telemetry.dev/docs/integrati…
Claude traces are coming to telemetry.dev, from the SDKs, the Messages API and Managed Agents.
You'll see tokens, cost and latency for every Claude call, counted once. Setup guide soon.
0
2
4
If you want to send these to telemetry.dev, here are the destination settings. Full guide: telemetry.dev/docs/integrati…
0
0
0
When an agent run goes wrong, open its trace on telemetry.dev.
Every model call and tool step is there, in order, with how long each took. The failed step is red, and you can read what went in and what came out.
Connect with our TypeScript or Python SDK, or point any at it.
Every model call and tool step is there, in order, with how long each took. The failed step is red, and you can read what went in and what came out.
Connect with our TypeScript or Python SDK, or point any at it.
0
5
9
Building with Jev from TypeSafe @typesafeai? Its evaluate() calls through the AI SDK are traced like any other model call: the input state, the questions Jev answered, its answers, tokens and cost.
1
2
5
Evaluation runs are traced like any other call. For AI SDK 7 evaluate(), you get the state, questions and answers the judge saw, plus its tokens and cost. TanStack AI evaluations are traced too.
Available now in the TypeScript and Python SDKs: telemetry.dev/integrations
Available now in the TypeScript and Python SDKs: telemetry.dev/integrations
1
1
3
Image, speech, transcription and video calls now show up as spans in our OpenAI, Google GenAI and AI SDK integrations.
Rerank calls do too, in the Bedrock, LiteLLM and AI SDK integrations.
Rerank calls do too, in the Bedrock, LiteLLM and AI SDK integrations.
1
2
6
One trace covers a whole voice support call: transcription, a chat turn with a photo, then speech.
Each step shows its model, latency, tokens by modality and cost, so you can see which part of the call costs the most.
Each step shows its model, latency, tokens by modality and cost, so you can see which part of the call costs the most.
1
2
5
New in telemetry.dev: token usage split by text, image and audio.
You see the split on the dashboard, on every trace and on every span. Cost uses each model's separate text, image and audio prices where the provider publishes them.
You see the split on the dashboard, on every trace and on every span. Cost uses each model's separate text, image and audio prices where the provider publishes them.
1
5
8
MCP Analytics is live on telemetry.dev.
Instrument the client and the server with one line each, and see how many calls pair up, where the server span went missing, and which methods and tools carry the load.
Instrument the client and the server with one line each, and see how many calls pair up, where the server span went missing, and which methods and tools carry the load.
0
8
13
Introducing Sessions in telemetry.dev
Follow an agent run turn by turn, with tool calls and cost in one view.
See where it slowed down or hit an error, then click the timeline to jump to that turn instead of piecing together separate traces.
Follow an agent run turn by turn, with tool calls and cost in one view.
See where it slowed down or hit an error, then click the timeline to jump to that turn instead of piecing together separate traces.
1
8
21
Introducing MCP Analytics for Telemetry.dev
Most MCP servers run blind once agents start calling them.
One line instruments yours. You see:
- calls in, calls failed
- p95 per method
- sessions
- the trace behind every failure
npm i @telemetry-dev/mcp
Most MCP servers run blind once agents start calling them.
One line instruments yours. You see:
- calls in, calls failed
- p95 per method
- sessions
- the trace behind every failure
npm i @telemetry-dev/mcp
4
16
49
telemetry.dev accepts standard OpenTelemetry traces, logs, and metrics.
If your app already exports OTLP, point it at telemetry.dev with four environment variables. You do not need our SDK or any code changes.
If your app already exports OTLP, point it at telemetry.dev with four environment variables. You do not need our SDK or any code changes.
0
2
15
Introducing Telemetry - observability for AI apps.
Every model call, tool step, and retrieval in one trace: tokens, cost, latency, errors.
OpenTelemetry-native, no lock-in.
telemetry.dev
Every model call, tool step, and retrieval in one trace: tokens, cost, latency, errors.
OpenTelemetry-native, no lock-in.
telemetry.dev
11
49
580