You might be overpaying 5.3x+ for Claude Opus 4.7!
Our CEO @gabrielbianconi found out that on tool-heavy workloads, you're paying 5.3x more for Claude Opus 4.7 than GPT 5.4.
The common metric is to compare cost per million tokens. But different providers use different d the same input produces wildly different token counts.
Read our blog below to find out how you are being charged and how to actually know what you're paying for. 👇
Our CEO @gabrielbianconi found out that on tool-heavy workloads, you're paying 5.3x more for Claude Opus 4.7 than GPT 5.4.
The common metric is to compare cost per million tokens. But different providers use different d the same input produces wildly different token counts.
Read our blog below to find out how you are being charged and how to actually know what you're paying for. 👇
1
1
4
LLM evaluators are often noisy and weakly correlated with real-world outcomes.
Noisy evaluators have limited value for production decisions that hinge on judging a single output (e.g. guardrails).
However, even (very) noisy evaluators can reliably tell you which agent is better y can still help you pick the best variant to deploy and improve it over time.
Learn how ↓
Noisy evaluators have limited value for production decisions that hinge on judging a single output (e.g. guardrails).
However, even (very) noisy evaluators can reliably tell you which agent is better y can still help you pick the best variant to deploy and improve it over time.
Learn how ↓
2
1
4
Michelle Hui is joining us with a focus on developer relations. She recently graduated from Cornell with BS & MS degrees in computer science, during which she organized large tech events, conducted ML research, and held product internships (Alphabet / Wing, UN).
Welcome to the team, @michellehui!
Welcome to the team, @michellehui!
1
1
18
"If your security relies on your code being obfuscated, you're telling yourself a fake story."
Our CTO Viraj Mehta (@thebigmehtaphor) chats about AI scaling, open source, and being a technical founder following his PhD in Reinforcement Learning at CMU.
In our in-depth s:
🔸What to look for in engineering hires when coding is largely solved
🔸How AI has increased the hiring bar and what you can do to meet it
🔸Why you should build in Rust to make your code more secure and error-free
🔸Why open source is still an important business model
🔸 Why AI is still scaling... and there's continued "juice to squeeze" in AI
Full conversation in the thread ↓
Our CTO Viraj Mehta (@thebigmehtaphor) chats about AI scaling, open source, and being a technical founder following his PhD in Reinforcement Learning at CMU.
In our in-depth s:
🔸What to look for in engineering hires when coding is largely solved
🔸How AI has increased the hiring bar and what you can do to meet it
🔸Why you should build in Rust to make your code more secure and error-free
🔸Why open source is still an important business model
🔸 Why AI is still scaling... and there's continued "juice to squeeze" in AI
Full conversation in the thread ↓
2
0
4
TensorZero is committed to open source. We sit down with our CTO Viraj @thebigmehtaphor. Takeaways:
1. Thinking closed-source code == security is a LIE. You still need to build from first principles and fundamentally secure code.
2. Open-source keeps more eyes on your code, bugs and vulnerabilities.
3. Open-source code allows developers and their agents to understand your codebase and integrate with it better. Close-source hamstrings users to look only at documentation.
@mhui1109
1. Thinking closed-source code == security is a LIE. You still need to build from first principles and fundamentally secure code.
2. Open-source keeps more eyes on your code, bugs and vulnerabilities.
3. Open-source code allows developers and their agents to understand your codebase and integrate with it better. Close-source hamstrings users to look only at documentation.
@mhui1109
1
0
3
Can an automated AI engineer autonomously debug and optimize an LLM pipeline in 5 minutes?
Last night, ours did: it cut errors in ~half during its first live demo.
TensorZero Autopilot (our automated AI engineer) analyzed hundreds of historical LLM traces to identify failure , and verified improvements with an LLM judge — autonomously, in <5 minutes.
With more time, it can do much more: from model selection to fine-tuning to adaptive experimentation, TensorZero Autopilot dramatically improves the performance of LLM agents across diverse tasks.
Learn more below ↓
Last night, ours did: it cut errors in ~half during its first live demo.
TensorZero Autopilot (our automated AI engineer) analyzed hundreds of historical LLM traces to identify failure , and verified improvements with an LLM judge — autonomously, in <5 minutes.
With more time, it can do much more: from model selection to fine-tuning to adaptive experimentation, TensorZero Autopilot dramatically improves the performance of LLM agents across diverse tasks.
Learn more below ↓
1
3
7
For each task, we collected 100 rollouts with a GPT-5 mini baseline using TensorZero. We then ran TensorZero Autopilot with 5 independent seeds and evaluated each seed with 100 fresh rollouts on held-out tasks.
1
0
3
We’re building TensorZero Autopilot, an automated AI engineer that analyzes LLM observability data, optimizes prompts and models, sets up evals, and runs A/B tests.
It dramatically improves the performance of LLM agents on every single benchmark we’ve tried.
Read more below.
It dramatically improves the performance of LLM agents on every single benchmark we’ve tried.
Read more below.
1
6
34
Antoine Toussaint previously was a staff software engineer at companies like Shopify and Illumio. Before becoming an engineer, he was a quant and a math professor at Stanford. He holds a PhD in Financial Engineering from Princeton.
Welcome to the team, Antoine!
Welcome to the team, Antoine!
0
2
13
Shuyang Li previously was a staff software engineer at Google focused on next-generation search infrastructure, LLM-based search, and many other specialized search products (local, travel, maps, etc.). Before that, he worked on ML/analytics products at Palantir and graduated tre Dame.
Welcome to the team, @_shuyang_!
Welcome to the team, @_shuyang_!
1
1
7
Thank you for 10,000 stars and 100 contributors — on the same day!
We're excited to continue building a next-generation open-source stack for LLM applications.
Find us on GitHub ↓
We're excited to continue building a next-generation open-source stack for LLM applications.
Find us on GitHub ↓
3
3
20
Al Chen (@BigAL123) from Grammarly recently built an agent with a continuous improvement loop using TensorZero + LangGraph.
Watch his presentation at the NYC LangChain Meetup last week!
Watch his presentation at the NYC LangChain Meetup last week!
2
2
11
Announcing our $7.3M seed round!
TensorZero enables a data and learning flywheel for optimizing LLM applications: a feedback loop that turns production metrics and human feedback into smarter, faster, and cheaper models and agents.
Today, we provide an open-source stack for de LLM applications that unifies an LLM gateway, observability, optimization, evaluation, and experimentation. You can take what you need, adopt incrementally, and complement with other tools. Over time, these components enable you to set up a principled feedback loop for your LLM application. The data you collect is tied to your KPIs, ports across model providers, and compounds into a competitive advantage for your business.
Our vision is to automate much of LLM engineering. We're laying the foundation for that with open-source TensorZero. For example, with our data model and end-to-end workflow, we will be able to proactively suggest new variants (e.g. a new fine-tuned model), backtest it on historical data (e.g. using diverse techniques from reinforcement learning), enable a gradual, live A/B test, and repeat the process. With a tool like this, engineers can focus on higher-level workflows — deciding what data goes in and out of these models, how to measure success, which behaviors to incentivize and disincentivize, and so on — and leave the low-level implementation details to an automated system. This is the future we see for LLM engineering as a discipline.
Recently, TensorZero reached #1 trending repository of the week globally on GitHub (& we're about to cross 10k stars). We're fortunate to have received contributions from dozens of developers worldwide, and it's exciting to see TensorZero already powering cutting-edge LLM products at frontier AI startups and large organizations, including one of Europe's largest banks.
We're excited to share that we've raised $7.3M to accelerate TensorZero's efforts to build best-in-class open-source infrastructure for LLM engineers (we're hiring!). The round was led by @FirstMarkCap, with participation from @BessemerVP, @bedrock, @DRWTrading, @coalitionvc, and dozens of strategic angels.
TensorZero enables a data and learning flywheel for optimizing LLM applications: a feedback loop that turns production metrics and human feedback into smarter, faster, and cheaper models and agents.
Today, we provide an open-source stack for de LLM applications that unifies an LLM gateway, observability, optimization, evaluation, and experimentation. You can take what you need, adopt incrementally, and complement with other tools. Over time, these components enable you to set up a principled feedback loop for your LLM application. The data you collect is tied to your KPIs, ports across model providers, and compounds into a competitive advantage for your business.
Our vision is to automate much of LLM engineering. We're laying the foundation for that with open-source TensorZero. For example, with our data model and end-to-end workflow, we will be able to proactively suggest new variants (e.g. a new fine-tuned model), backtest it on historical data (e.g. using diverse techniques from reinforcement learning), enable a gradual, live A/B test, and repeat the process. With a tool like this, engineers can focus on higher-level workflows — deciding what data goes in and out of these models, how to measure success, which behaviors to incentivize and disincentivize, and so on — and leave the low-level implementation details to an automated system. This is the future we see for LLM engineering as a discipline.
Recently, TensorZero reached #1 trending repository of the week globally on GitHub (& we're about to cross 10k stars). We're fortunate to have received contributions from dozens of developers worldwide, and it's exciting to see TensorZero already powering cutting-edge LLM products at frontier AI startups and large organizations, including one of Europe's largest banks.
We're excited to share that we've raised $7.3M to accelerate TensorZero's efforts to build best-in-class open-source infrastructure for LLM engineers (we're hiring!). The round was led by @FirstMarkCap, with participation from @BessemerVP, @bedrock, @DRWTrading, @coalitionvc, and dozens of strategic angels.
9
14
107
Alan Mishler is an ML researcher with a background in causal inference, sequential decision making, uncertainty quantification, and algorithmic fairness (1.2k+ citations). Previously, he was an AI Research Lead at JPMorgan AI Research and received a PhD in Statistics from CMU, d at Google and Box.
Welcome to the team, Alan!
Welcome to the team, Alan!
1
2
16
Congratulations @YoungseogC for successfully defending his ML PhD at CMU!
(& thank you for the TensorZero shoutout!)
(& thank you for the TensorZero shoutout!)
0
2
9
WANTED: Founding MTS with a background that combines product and engineering
We can't keep up with TensorZero's open-source traction. You can help.
Join our founding team in a cross-functional role that combines product, engineering, and GTM.
Team: 3 ML researchers (Stanford, 6k+ citations), Rust compiler maintainer, staff SWE on LLM infra for Google Search, decacorn CPO, _____ you?
We have years of runway (large unannounced seed round — shh).
$200-300k salary + 0-1% equity + benefits
Onsite in NYC
Apply below ↓
We can't keep up with TensorZero's open-source traction. You can help.
Join our founding team in a cross-functional role that combines product, engineering, and GTM.
Team: 3 ML researchers (Stanford, 6k+ citations), Rust compiler maintainer, staff SWE on LLM infra for Google Search, decacorn CPO, _____ you?
We have years of runway (large unannounced seed round — shh).
$200-300k salary + 0-1% equity + benefits
Onsite in NYC
Apply below ↓
1
2
4
The latest release (2025.6.1) includes more granular inference timeouts.
You can define timeouts for non-streaming and streaming requests separately: `timeouts.non_streaming.total_ms` corresponds to the total request duration and `timeouts.streaming.ttft_ms` corresponds to the FT). This setting applies to individual requests to the model provider. If you’ve defined retries and fallbacks, the timeout will apply to each retry and fallback separately. This setting is particularly useful if you’d like to retry or fallback on a request that’s taking too long.
You can define timeouts for non-streaming and streaming requests separately: `timeouts.non_streaming.total_ms` corresponds to the total request duration and `timeouts.streaming.ttft_ms` corresponds to the FT). This setting applies to individual requests to the model provider. If you’ve defined retries and fallbacks, the timeout will apply to each retry and fallback separately. This setting is particularly useful if you’d like to retry or fallback on a request that’s taking too long.
0
0
2
Andrew is an ML researcher with deep expertise in Bayesian ML, causal inference, RL, and LLMs. He’s finishing his postdoc at Columbia and previously received a PhD from Oxford, during which he interned at Meta. He has 3.3k+ citations and several first-author papers at NeurIPS and elcome to the team @anndvision!
0
0
7
What happens under the hood of a $9.9B coding assistant?
We reverse-engineered Cursor's LLM client to observe the requests being made (including prompts!), A/B test different models, and eventually optimize our own prompts + models.
↓ Link in thread with code to reproduce + ts
We reverse-engineered Cursor's LLM client to observe the requests being made (including prompts!), A/B test different models, and eventually optimize our own prompts + models.
↓ Link in thread with code to reproduce + ts
We've raised $900 million in Series C funding from Thrive, Accel, Andreessen Horowitz, and DST.
We're happy to share that Cursor has grown to over $500 million in ARR and is used by more than half of the Fortune 500, including NVIDIA, Uber, and Adobe.
This scale will help us push the frontier of AI coding research.
6
8
23
TensorZero now supports exporting traces via OpenTelemetry!
Exporting traces via OpenTelemetry (OTLP) allows you to monitor the TensorZero Gateway in external observability platforms such as Jaeger, Datadog, or Grafana. This integration enables you to correlate gateway activity nfrastructure, providing deeper insights and unified monitoring across your systems.
Guide + GitHub example below ↓
Exporting traces via OpenTelemetry (OTLP) allows you to monitor the TensorZero Gateway in external observability platforms such as Jaeger, Datadog, or Grafana. This integration enables you to correlate gateway activity nfrastructure, providing deeper insights and unified monitoring across your systems.
Guide + GitHub example below ↓
1
0
3
📄 New Example: TensorZero + MCP (Model Context Protocol)
Learn how to use an MCP server with TensorZero. We'll use `mcp-clickhouse` to build a chatbot that can answer questions about the contents of your ClickHouse database.
GitHub link in thread ↓
Learn how to use an MCP server with TensorZero. We'll use `mcp-clickhouse` to build a chatbot that can answer questions about the contents of your ClickHouse database.
GitHub link in thread ↓
2
1
8
📄 Example: OpenAI Codex and TensorZero Integration
Why?
• Use every major model provider with Codex
• Gain comprehensive observability into your Codex usage, including detailed logs for every LLM call.
• Set up advanced inference features like retries, fallbacks, load me optimizations, experimentation (A/B testing), and more.
• Collect data for fine-tuning and other optimization techniques, and use TensorZero recipes to create custom models for your Codex usage.
Link below ↓
Why?
• Use every major model provider with Codex
• Gain comprehensive observability into your Codex usage, including detailed logs for every LLM call.
• Set up advanced inference features like retries, fallbacks, load me optimizations, experimentation (A/B testing), and more.
• Collect data for fine-tuning and other optimization techniques, and use TensorZero recipes to create custom models for your Codex usage.
Link below ↓
1
0
2
📄 New Example: TensorZero and LangGraph Integration
You can use TensorZero with agent orchestration frameworks like LangGraph. This example shows a minimal example for a chatbot with tool use that combines TensorZero + LangGraph.
See the full example on GitHub ↓
You can use TensorZero with agent orchestration frameworks like LangGraph. This example shows a minimal example for a chatbot with tool use that combines TensorZero + LangGraph.
See the full example on GitHub ↓
1
0
5
We wanted to know… how well does automated prompt engineering hold up as task complexity increases?
We put MIPRO, an automated prompt engineering algorithm, to the test across a range of tasks — from simple named entity recognition (CoNLL++), to multi-hop retrieval (HoVer), to ion (BabyAI), to customer support with agentic tool use (τ-bench).
Here's what we learned:
• Automated prompt engineering with MIPRO can significantly improve performance in simpler tasks, but the benefits start to diminish as task complexity grows.
•Larger models seem to benefit more from MIPRO optimization in complex settings. We hypothesize this difference is due to a better ability to handle long multi-turn demonstrations.
• Unsurprisingly, the quality of the feedback materially affects the quality of the MIPRO optimization process. But at the same time, we still see meaningful improvements from noisy feedback, including AI-generated feedback.
More details below ↓
We put MIPRO, an automated prompt engineering algorithm, to the test across a range of tasks — from simple named entity recognition (CoNLL++), to multi-hop retrieval (HoVer), to ion (BabyAI), to customer support with agentic tool use (τ-bench).
Here's what we learned:
• Automated prompt engineering with MIPRO can significantly improve performance in simpler tasks, but the benefits start to diminish as task complexity grows.
•Larger models seem to benefit more from MIPRO optimization in complex settings. We hypothesize this difference is due to a better ability to handle long multi-turn demonstrations.
• Unsurprisingly, the quality of the feedback materially affects the quality of the MIPRO optimization process. But at the same time, we still see meaningful improvements from noisy feedback, including AI-generated feedback.
More details below ↓
1
0
9
📄 New Example: Agentic RAG — Multi-Hop Question Answering with LLMs
Learn how to build a multi-hop retrieval agent using TensorZero. The agent iteratively searches Wikipedia to gather information and decides when it has enough context to answer a complex question.
GitHub link below ↓
Learn how to build a multi-hop retrieval agent using TensorZero. The agent iteratively searches Wikipedia to gather information and decides when it has enough context to answer a complex question.
GitHub link below ↓
1
1
6
TensorZero supports 🦙 Llama 4 out of the box!
Try it today on providers like @FireworksAI_HQ and @togethercompute with a one-line change.
Try it today on providers like @FireworksAI_HQ and @togethercompute with a one-line change.
0
0
3
📄 Case Study: Automating Code Changelogs at a Large Bank with LLMs
One of the largest European banks used TensorZero to build an LLM-powered changelog generator for GitLab merge requests. They optimized model quality, deployed fully on-premise, and iterated quickly — without gineers.
Link in thread ↓
One of the largest European banks used TensorZero to build an LLM-powered changelog generator for GitLab merge requests. They optimized model quality, deployed fully on-premise, and iterated quickly — without gineers.
Link in thread ↓
1
1
9
"I Found The Missing Intelligence Layer in Every LLM Stack (And It's Game-Changing)" — @markobilal
Watch Marko integrate TensorZero into his project and explain some of the benefits you get out of the box: provider interoperability, higher availability, observability, prompt In his own words, TensorZero is "the missing intelligence layer in every LLM stack that nobody's talking about - and it's about to change how you build. [...] This open-source tool cut my codebase by a third and let me ship twice as fast."
Watch Marko integrate TensorZero into his project and explain some of the benefits you get out of the box: provider interoperability, higher availability, observability, prompt In his own words, TensorZero is "the missing intelligence layer in every LLM stack that nobody's talking about - and it's about to change how you build. [...] This open-source tool cut my codebase by a third and let me ship twice as fast."
1
0
9
💡 TensorZero + Node (JavaScript / TypeScript)
Building in JS/TS? Combine TensorZero with the OpenAI Node client to get an LLM gateway, observability, optimization, experimentation, and more... in just two lines of code!
Building in JS/TS? Combine TensorZero with the OpenAI Node client to get an LLM gateway, observability, optimization, experimentation, and more... in just two lines of code!
1
0
4
Nik Caryotakis shares how TensorZero helps SuperDial orchestrate their healthcare phone agent with structured, typed LLM endpoints (& optimize against noisy business metrics from production).
3
0
7
TensorZero in the wild at @aiDotEngineer!
Nik Caryotakis shared how SuperDial is automating millions of phone calls to streamline healthcare back-office — using TensorZero under the hood to orchestrate and optimize their LLM systems!
Nik Caryotakis shared how SuperDial is automating millions of phone calls to streamline healthcare back-office — using TensorZero under the hood to orchestrate and optimize their LLM systems!
0
1
7