Pinned Tweet
3️⃣ 2️⃣ 1️⃣ Our free course on LLM evaluations for AI product teams starts today!
🎥 7 days of byte-sized videos into your inbox
⭐️ Certificate upon completion
👩💻 No coding skills required
👩🎓500+ students have signed up
You can still join the course👇
evidentlyai.com/llm-evaluation…
🎥 7 days of byte-sized videos into your inbox
⭐️ Certificate upon completion
👩💻 No coding skills required
👩🎓500+ students have signed up
You can still join the course👇
evidentlyai.com/llm-evaluation…
3
1
7
1,950
GitHub shares their learnings on evals across LLM-powered systems – for code analysis, developer tools, security, data analysis, and other production workflows.
github.blog/ai-and-ml/llms…
github.blog/ai-and-ml/llms…
1
1
8
937
How DoorDash runs evals for support chatbots using simulated conversations - at scale: careersatdoordash.com/blog/doordashs…
1
0
2
230
How Netflix evaluates the quality of show synopsis using LLM-as-a-judge approach:
netflixtechblog.com/evaluating-net…
netflixtechblog.com/evaluating-net…
1
0
4
1,096
How Zalando builds a search quality assurance framework with LLM-as-a-judge:
engineering.zalando.com/posts/2026/03/…
engineering.zalando.com/posts/2026/03/…
0
0
0
158
📌 In case you missed it
How to evaluate an AI agent?
Follow the tutorial as we:
1️⃣ Build an AI agent,
2️⃣ Create a test dataset,
3️⃣ Assess responses and tool choice,
4️⃣ Track the agent’s behaviour.
Follow the tutorial from our LLM evals course: youtube.com/watch?v=9KMmad…
How to evaluate an AI agent?
Follow the tutorial as we:
1️⃣ Build an AI agent,
2️⃣ Create a test dataset,
3️⃣ Assess responses and tool choice,
4️⃣ Track the agent’s behaviour.
Follow the tutorial from our LLM evals course: youtube.com/watch?v=9KMmad…
0
0
2
233
A Friday ML use case 📕
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How Uber improves driver availability at airports: Estimated time-to-request model, Earnings-per-hour prediction, and Driver-deficit forecasting.
uber.com/en-GB/blog/for…
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How Uber improves driver availability at airports: Estimated time-to-request model, Earnings-per-hour prediction, and Driver-deficit forecasting.
uber.com/en-GB/blog/for…
0
0
0
166
🦾 More AI agents aren’t always better.
Google evaluated 180 agent setups and found multi-agent systems help with parallel tasks but can hurt sequential ones.
The work also proposes a model to predict optimal agentic designs.
research.google/blog/towards-a…
Google evaluated 180 agent setups and found multi-agent systems help with parallel tasks but can hurt sequential ones.
The work also proposes a model to predict optimal agentic designs.
research.google/blog/towards-a…
0
0
1
115
📌 In case you missed it
Let’s test your RAG system!
Follow the tutorial as we:
1️⃣ Build a RAG system,
2️⃣ Generate test data,
3️⃣ Evaluate answers for correctness and faithfulness.
Watch the tutorial from our LLM evals course: youtube.com/watch?v=jckp5R…
Let’s test your RAG system!
Follow the tutorial as we:
1️⃣ Build a RAG system,
2️⃣ Generate test data,
3️⃣ Evaluate answers for correctness and faithfulness.
Watch the tutorial from our LLM evals course: youtube.com/watch?v=jckp5R…
0
0
0
145
A Friday ML use case 📕
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How GoDaddy built Lighthouse, an internal AI analytics platform: prompt engineering framework, model orchestration, solution architecture, and use cases.
godaddy.com/resources/news…
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How GoDaddy built Lighthouse, an internal AI analytics platform: prompt engineering framework, model orchestration, solution architecture, and use cases.
godaddy.com/resources/news…
0
0
0
89
Evidently AI retweeted
(policyNIM oss tool)
preflight command is working. when I provide a coding task, it kicks off a search through indexed policies to determine which rules are relevant for implementation.
@nvidia for embedding w/ @OpenAI + @lancedb for vector storage.
eval command is also lyAI for running eval suite.
preflight command is working. when I provide a coding task, it kicks off a search through indexed policies to determine which rules are relevant for implementation.
@nvidia for embedding w/ @OpenAI + @lancedb for vector storage.
eval command is also lyAI for running eval suite.
1
2
4
449
🚦 Meta’s “Agents Rule of Two”
According to Meta, AI agents should satisfy at most two of these conditions per session to reduce prompt-injection risk:
- Handle untrusted inputs
- Access sensitive data
- Change state / act externally
ai.meta.com/blog/practical…
According to Meta, AI agents should satisfy at most two of these conditions per session to reduce prompt-injection risk:
- Handle untrusted inputs
- Access sensitive data
- Change state / act externally
ai.meta.com/blog/practical…
0
0
0
60
📌 In case you missed it
How do you know if your RAG works?
You need to check:
✅ Can it find the right information?
✅ Is the final answer complete, relevant, and free of hallucinations?
Watch the intro to RAG evaluation from our LLM evals course: youtube.com/watch?v=qI2qQf…
How do you know if your RAG works?
You need to check:
✅ Can it find the right information?
✅ Is the final answer complete, relevant, and free of hallucinations?
Watch the intro to RAG evaluation from our LLM evals course: youtube.com/watch?v=qI2qQf…
0
0
1
180
A Friday ML use case 📕
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How DoorDash improves its RecSys using LLMs to bridge behavioral silos in multi-vertical recommendations.
careersatdoordash.com/blog/doordash-…
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How DoorDash improves its RecSys using LLMs to bridge behavioral silos in multi-vertical recommendations.
careersatdoordash.com/blog/doordash-…
0
0
0
70
💭 Can AI systems introspect?
Anthropic’s new research suggests Claude models can sometimes identify and describe their own internal states.
It’s still unreliable, but marks a step toward more transparent AI reasoning.
anthropic.com/research/intro…
Anthropic’s new research suggests Claude models can sometimes identify and describe their own internal states.
It’s still unreliable, but marks a step toward more transparent AI reasoning.
anthropic.com/research/intro…
0
0
0
55
📌 In case you missed it
Can LLMs write engaging tech tweets?
Follow the tutorial as we:
1️⃣ Build a tweet generator,
2️⃣ Score its outputs with custom LLM judges,
3️⃣ Improve the results with prompt iteration.
Watch the tutorial from our LLM evals course: youtube.com/watch?v=KhkiM9…
Can LLMs write engaging tech tweets?
Follow the tutorial as we:
1️⃣ Build a tweet generator,
2️⃣ Score its outputs with custom LLM judges,
3️⃣ Improve the results with prompt iteration.
Watch the tutorial from our LLM evals course: youtube.com/watch?v=KhkiM9…
0
0
2
189
A Friday ML use case 📕
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How Shopify transformed its product classification system from basic categorization to an AI-driven framework using Vision Language Models.
shopify.engineering/evolution-prod…
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How Shopify transformed its product classification system from basic categorization to an AI-driven framework using Vision Language Models.
shopify.engineering/evolution-prod…
0
1
2
77
📚 Context is everything.
OpenAI shares how it built an in-house data agent that answers complex questions in minutes.
It uses 6 layers of context:
- Table metadata
- Human annotations
- Codex enrichment
- Company knowledge
- Memory
- Runtime context
openai.com/index/inside-o…
OpenAI shares how it built an in-house data agent that answers complex questions in minutes.
It uses 6 layers of context:
- Table metadata
- Human annotations
- Codex enrichment
- Company knowledge
- Memory
- Runtime context
openai.com/index/inside-o…
0
0
1
119
📌 In case you missed it
Are LLMs good for classification tasks?
We built an LLM-based classifier for a travel support chatbot and compared its performance to a classic ML model.
Watch the tutorial from our LLM evals course: youtube.com/watch?v=Gl2X_o…
Are LLMs good for classification tasks?
We built an LLM-based classifier for a travel support chatbot and compared its performance to a classic ML model.
Watch the tutorial from our LLM evals course: youtube.com/watch?v=Gl2X_o…
0
1
2
164
A Friday ML use case 📕
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How Wayfair built Wilma, a customer service agent copilot: workflow, prompt templates, and the copilot’s evolution.
aboutwayfair.com/careers/tech-b…
📚 From the database of 800 ML & LLM systems: cutt.ly/SwrZWL0g
How Wayfair built Wilma, a customer service agent copilot: workflow, prompt templates, and the copilot’s evolution.
aboutwayfair.com/careers/tech-b…
0
0
0
88
🤖 How to develop and deploy chatbots at scale?
DoorDash shares how they created a simulation platform and evaluation flywheel, allowing them to test chatbots with fast feedback loops and without production risk.
careersatdoordash.com/blog/doordash-…
DoorDash shares how they created a simulation platform and evaluation flywheel, allowing them to test chatbots with fast feedback loops and without production risk.
careersatdoordash.com/blog/doordash-…
0
1
1
65