Pinned Tweet
This Fall at CMU we're teaching a new course on AI Agents!
The goal is that you learn how to create a scaffold, build evals, and train an agentic LLM using RL.
We'll try to balance theory and practice, and introduce modern frameworks and best practices.
The goal is that you learn how to create a scaffold, build evals, and train an agentic LLM using RL.
We'll try to balance theory and practice, and introduce modern frameworks and best practices.
56
283
2,364
356,335
I bought a B300 machine for my lab at CMU.
I was thinking of using it to do research on LLMs, but now I'm thinking of whether I should just use it to serve deepseek-v4.1-flash tokens with the new deepseek inference code and fund the entire rest of my lab that way 😅
I was thinking of using it to do research on LLMs, but now I'm thinking of whether I should just use it to serve deepseek-v4.1-flash tokens with the new deepseek inference code and fund the entire rest of my lab that way 😅
Heard from a neolab friend:
“Even if we fail to train the best model, we locked in GPU capacity early. H100/B200/B300 rental prices have since gone up 2–3x. We can just resell the compute and print money.”
OK, so that’s the business model.
27
24
685
27,904
Don't sleep on single agent systems
openhands.dev/blog/dont-slee…
openhands.dev/blog/dont-slee…
Anthropic is now recommending that companies REMOVE unnecessary subagents from their AI systems.
This is a pretty significant shift in agent architecture, with their team finding that a single agent equipped with skills can frequently outperform complex multi-agent systems, and one customer seeing roughly 2x faster execution.
The emerging rules of thumb:
→ Skills: Teach the agent HOW to do something, including procedures, expertise, and instructions.
→ Tools / MCP: Give the agent the ability to DO things through APIs, actions, and live data.
→ Subagents: Use for parallel work, isolated context, or tasks a single agent struggles with.
The interesting part is progressive disclosure. Instead of spawning specialized agents or stuffing everything into a giant prompt, one agent discovers and loads skills and tools as needed.
Their advice is to fold existing subagents into the main loop, turn their instructions into skills, expose their actions as tools, and run evals to see if performance holds up.
As models have gotten smarter, improving agent architecture is becoming less about what you can add and more about what you can delete.
claude.com/resources/arti…
9
5
63
6,269
Graham Neubig retweeted
“What movie is this scene from?”
You upload a screenshot. Your AI gives you a title. But is it the right film?
Even a leading model that scores 81% overall at identifying entities from text and images, only 41% accuracy on rare-entity subsets!
In Think Before You Link , @gneubig, and I measure these rare-entity gaps across 15 definitions of rarity and propose a simple training-free approach:
📝 Paper: arxiv.org/abs/2609.10745
🌐 Project: neulab.github.io/think-before-y…
🧑💻 Code: github.com/neulab/think-b…
📚 Data: huggingface.co/datasets/neula…
With no additional training, accuracy on that same subset rises from 41% to 60%! 🧵
You upload a screenshot. Your AI gives you a title. But is it the right film?
Even a leading model that scores 81% overall at identifying entities from text and images, only 41% accuracy on rare-entity subsets!
In Think Before You Link , @gneubig, and I measure these rare-entity gaps across 15 definitions of rarity and propose a simple training-free approach:
📝 Paper: arxiv.org/abs/2609.10745
🌐 Project: neulab.github.io/think-before-y…
🧑💻 Code: github.com/neulab/think-b…
📚 Data: huggingface.co/datasets/neula…
With no additional training, accuracy on that same subset rises from 41% to 60%! 🧵
4
2
31
1,799
I'll be doing a live stream on "how to build an open source software factory with open models" at Hacktoberfest 2026 in 15 minutes.
Tune in and give me best wishes that my hot-off-the-press demo doesn't fall over :)
hacktoberfest.com/schedule/
Tune in and give me best wishes that my hot-off-the-press demo doesn't fall over :)
hacktoberfest.com/schedule/
3
3
51
2,634
Today we announce 1Password for Agents platform, building with companies @Muse, SpaceX Super Intelligence Grok @bot, @AnthropicAI’s @claudeai, @OpenAI, @TownAI, @browserbase, @browser_use, @heydana_ai, Instinct, @usekernel, @perplexity_ai, @tryramp, @tomo, and @wajo_ai, with e way. 🎉
We've heard from so many of our customers who want to use these new personal AI agents to help them get more done, but want more control over what the agent can access with the same trust and ease of use they get when using 1Password directly.
1Password for Agents gives your agents access without giving up your passwords. When an agent reaches a sign-in form, it requests a credential from your vault. If you approve the request, 1Password directly auto-fills the credential in the destination site so the agent can’t see or take possession of the credentials.
The agent gets the task done, but it doesn’t get your passwords..
You stay in control. 1Password for Agents free and optional for personal and family accounts. Unless you opt-in to an agent connector, agents cannot touch your vault.
➡️Learn more: bit.ly/487da6J
2
2
13
2,596
Graham Neubig retweeted
A million hours of what?
If we’re scaling human demonstrations for robot learning, what should each hour contain?
We built S1, our skill capture glove, alongside our robot hand to record joint movements that map to robot targets, with touch and video on the same timeline.
If we’re scaling human demonstrations for robot learning, what should each hour contain?
We built S1, our skill capture glove, alongside our robot hand to record joint movements that map to robot targets, with touch and video on the same timeline.
38
66
324
49,064
Strong new model in the 40B class, exciting!
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from hs
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
3
0
41
3,152
Graham Neubig retweeted
I recently gave an invited lecture on deep research agents for CMU’s AI Agents course, covering evaluation beyond simple factoid QA, RL for open-ended tasks, and a new retrieval paradigm for deep research agents!
youtu.be/nKUBrXFQBUM
youtu.be/nKUBrXFQBUM
2
19
201
18,194
Amazing. For those who have used DeepSeek before, you know that the inference that they have on the official provider is absurdly good+cheap. This is a big release.
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/De…
1/4
2
2
125
8,320
Graham Neubig retweeted
The UT comp ling faculty (@jessyjli, @kanishkamisra, @kmahowald & myself) are looking for PhD students to join us! sites.utexas.edu/compling/ 🤘
We are an interdisciplinary group with interests across linguistics, cogsci and NLP. Apply by 12/1: liberalarts.utexas.edu/linguistics/gr…
We are an interdisciplinary group with interests across linguistics, cogsci and NLP. Apply by 12/1: liberalarts.utexas.edu/linguistics/gr…
1
33
136
18,921
In an ICLR paper I'm reviewing, the only text that pangram detects as not AI generated is the sentence that says the authors "Used LLMs to polish the writing."
The writing is, well, very polished I guess.
The writing is, well, very polished I guess.
24
8
572
59,279
Very ambitious work by Seungone on trying to figure out if LLMs can actually write strong follow-up papers across the natural sciences.
Based on our previous work on automated paper review, we apply rigorous standards to the generated papers and find that even with frontier models, none are hallucination-free and meet the bar of the scientists working in each domain.
We believe that this poses a challenge for upcoming LMs, can they actually do an end-to-end research work and generate an artifact that could move science forward?
Based on our previous work on automated paper review, we apply rigorous standards to the generated papers and find that even with frontier models, none are hallucination-free and meet the bar of the scientists working in each domain.
We believe that this poses a challenge for upcoming LMs, can they actually do an end-to-end research work and generate an artifact that could move science forward?
🧪Can AI agents take a scientific paper and write a meaningful follow-up to it?
We introduce Into the Unknown, an AI agent benchmark of 15 open scientific research problems. Each problem comes from a scientist currently or formerly at US DOE National Labs:
@LosAlamosNatLab b @AmesNatLab @PPPLab @Livermore_Lab
3
8
104
6,937
Graham Neubig retweeted
This new benchmark evaluates nine frontier agents on 15 open scientific follow-up problems developed with domain scientists, producing 135 research papers.
Most of the papers fail integrity checks, including discrepancies between claimed work and supporting execution evidence.
Most of the papers fail integrity checks, including discrepancies between claimed work and supporting execution evidence.
18
35
162
11,865
This model will be 💥
AI for experimental physics is a super-exciting area right now.
AI for experimental physics is a super-exciting area right now.
What can large-scale AI models bring to the growing complexity of particle physics?
@CERN and Mistral join forces to explore this question.
The ambition: a single Large Physics Model, trained on billions of simulated collisions.
4
0
39
3,127
Graham Neubig retweeted
We’re planning to expand EurekaBench, an expert-verified benchmark that tests whether AI agents can go beyond simply finding solutions to discover novel scientific insights.
We’re looking to collaborate with domain experts on two types of contributions:
1. Additional insight questions for existing problems: Different approaches to the same science problem can yield different scientific insights. If you've worked on a problem already included in EurekaBench but approached it differently, we’d love to hear what additional insights you think an AI agent could discover.
2. New scientific problems: We’d like to include more open scientific problems across our six existing domains, as well as in new domains. These may be recently published science problems that remain only partially solved, or unpublished problems.
If you’re interested in contributing, please check out our contribution guidelines for more details: eurekabench.github.io/contribute.html.
We’re looking to collaborate with domain experts on two types of contributions:
1. Additional insight questions for existing problems: Different approaches to the same science problem can yield different scientific insights. If you've worked on a problem already included in EurekaBench but approached it differently, we’d love to hear what additional insights you think an AI agent could discover.
2. New scientific problems: We’d like to include more open scientific problems across our six existing domains, as well as in new domains. These may be recently published science problems that remain only partially solved, or unpublished problems.
If you’re interested in contributing, please check out our contribution guidelines for more details: eurekabench.github.io/contribute.html.
AI scientist agents are great at optimizing and finding solutions through endless trial and error. But is that really what science is about?
As Terrance Tao eloquently put, the role of math and science should be more than that – they are "lighthouses" that guide and inspire erstanding and insights. Can AI agents discover novel insights through experimentation?
To study these gaps, we worked with domain experts to introduce EurekaBench, a benchmark spanning 6 science domains that evaluates an agent’s ability to discover genuinely new scientific insights.
(1/n)
4
15
90
5,016
I now use digital ocean VMs to host my personal cloud agents that automate a bunch of my daily tasks, so really excited to have OpenHands participate in this event!
Who gets to shape the future of AI?
Today we're bringing together some of the biggest voices in open models, data, and agents to answer that question, including @rbren_dev of @OpenHandsDev, @zlxi02 of @inferact, @GerryFPatia of @boltdotnew, @vinaygo of @Amplitude_HQ and @latkins en Intelligence Summit, co-hosted with @nvidia.
Watch live, starting at 1 PM PT:
Keynote: youtube.com/live/IhWSgZlVi…
Build: youtube.com/live/eN0TqmeBN…
Serve: youtube.com/live/fkT-MhcAm…
Run: youtube.com/live/QMrY1xZ-x…
2
1
11
3,025
Here's an announcement of tighter integration w/ DigitalOcean sandboxes! x.com/OpenHandsDev/s…
OpenHands now integrates with @digitalocean Managed Agents! 🌊
Run AI coding agents in isolated cloud environments, pause and resume sessions, and manage everything from Agent Canvas.
Use DigitalOcean for agent execution, model inference, or both.
More flexibility in where agents run and which models power them.
Read more about the integration and how to get started: hubs.la/Q04B3Hbl0
0
0
1
1,079
Recently, @_emliu et al.'s paper on "The Implicit Curriculum Hypothesis" won an outstanding paper award at COLM: arxiv.org/abs/2604.08510
This is of course due to the ingenuity and hard work of the authors, but it's also thanks to an @NSF ACCESS grant of compute resources, for which we are very grateful.
Just wanted to explicitly point this out as a positive story of public support for computational resources for open research!
This is of course due to the ingenuity and hard work of the authors, but it's also thanks to an @NSF ACCESS grant of compute resources, for which we are very grateful.
Just wanted to explicitly point this out as a positive story of public support for computational resources for open research!
7
36
328
16,377