Pinned Tweet
Can an AI model predict perfectly and still have a terrible world model?
What would that even mean?
Our new ICML paper formalizes these questions
One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
What would that even mean?
Our new ICML paper formalizes these questions
One result tells the story: A transformer trained on 10M solar systems nails planetary orbits. But it botches gravitational laws 🧵
210
983
6,618
1,397,374
Keyon Vafa retweeted
Deep learning is often taught as a sanitized collection of facts and math. But work at the frontier of AI is messy and empirical.
Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch.
on by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens?
deep-learning-alchemy.github.io
My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments).
We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up.
The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue
Try some quizzes below!
Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch.
on by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens?
deep-learning-alchemy.github.io
My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments).
We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up.
The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue
Try some quizzes below!
43
313
2,456
177,865
Keyon Vafa retweeted
UC Berkeley Department of Statistics is hiring! We’re seeking applicants for a tenure-track Assistant Professor position in Statistics, Probability, and Data Science.
Details & apply: aprecruit.berkeley.edu/JPF05567
#BerkeleyStats #Probability #DataScience #FacultyJobs #UCBerkeley
Details & apply: aprecruit.berkeley.edu/JPF05567
#BerkeleyStats #Probability #DataScience #FacultyJobs #UCBerkeley
0
12
35
8,985
Keyon Vafa retweeted
Proud to be part of this adventure with such a fantastic team—@kchonyc, @keunwoochoi, @HenriDwyer, and @elmanmansimov. So excited for what’s ahead. Come join us!
We’re excited to announce Ortet, a frontier AI lab for health.
A fundamental gap in care today is that the industry and its scientific tools focus on discovering what works at the population level, while it is individual patients we must focus on to deliver the best care. Individualized care requires a broad view of the patient, the care they receive, and the system around them. Yet the information needed for such a view remains fragmented.
At Ortet, our goal is to create continuously improving intelligence for every patient: the Ortet Health Model, bringing together biological, clinical, and operational dimensions to better predict what happens next and which actions are most likely to lead to better outcomes. To deliver it, we are creating frontier AI from the ground up—across compute, data infrastructure, and models—and building and operating at every layer of AI.
We are scientists and engineers behind foundational AI advances, including the attention mechanism, GRU, and neural text-to-image generation. Across the team, our work spans frontier AI research, health, life sciences, and large-scale data and compute infrastructure—from building AI for lab-in-the-loop molecular design and co-founding Prescient Design to establishing Genentech’s frontier research team and building the largest GPU cluster in life sciences.
We imagine a future where everyone has access to affordable, precise, and better care.
If you want to help us build that future, join us: ortet.ai
7
4
53
5,218
Keyon Vafa retweeted
i was diagnosed with thyroid cancer many years ago purely by chance, because i had to get blood work for my US green card application (thank you, USCIS!). i was in my mid-30’s and was reasonably healthy (or so i believed.) this was shocking and totally unpredictable news to me. thanks to the amazing team of clinicians at NYU Langone, i successfully went through a total thyroidectomy followed by radioactive iodine therapy. but, it was truly the most uncertain and confusing period of my life.
this is when i first realized that no one can opt out of healthcare. being sick isn’t what you choose to or choose not to. it is something that unpredictably happens to all of us at some point for one reason or another. this is why it is a healthcare system that we collectively build and use to support each other at the society level. because it must support everyone, the complexity of creating, maintaining and improving this system is a Herculean effort that calls for all the help in the world.
today, i am announcing @OrtetAI together with my dear co-founders, @keunwoochoi, @HenriDwyer, @elmanmansimov, @causalclaudia and Jeff, to lend our support to improving the health of each and every one of us. Ortet’s goal is to create a technological foundation that will enable health systems to deliver predictably better care to each and every patient. this will only be possible by working closely with all stakeholders in the broader ecosystem, which is enabled in part by Ortet’s close partnership with Thoreau.
Ortet is taking the very first step of a long journey ahead, and at the end of this journey, i strongly believe we would be able to say that Ortet had contributed to building better, more predictable healthcare and thus to improving health of each and every one of us.
see ortet.ai for more!
this is when i first realized that no one can opt out of healthcare. being sick isn’t what you choose to or choose not to. it is something that unpredictably happens to all of us at some point for one reason or another. this is why it is a healthcare system that we collectively build and use to support each other at the society level. because it must support everyone, the complexity of creating, maintaining and improving this system is a Herculean effort that calls for all the help in the world.
today, i am announcing @OrtetAI together with my dear co-founders, @keunwoochoi, @HenriDwyer, @elmanmansimov, @causalclaudia and Jeff, to lend our support to improving the health of each and every one of us. Ortet’s goal is to create a technological foundation that will enable health systems to deliver predictably better care to each and every patient. this will only be possible by working closely with all stakeholders in the broader ecosystem, which is enabled in part by Ortet’s close partnership with Thoreau.
Ortet is taking the very first step of a long journey ahead, and at the end of this journey, i strongly believe we would be able to say that Ortet had contributed to building better, more predictable healthcare and thus to improving health of each and every one of us.
see ortet.ai for more!
66
110
1,012
79,638
Keyon Vafa retweeted
Excited to join the new NYU Courant Institute School of Mathematics, Computing, and Data Science (@NYU_Courant) as an assistant professor of CS ~2 weeks ago!
WOW its already been v exciting for ML and math at Courant in particular! While fun I hope not ever week is like this!!!
WOW its already been v exciting for ML and math at Courant in particular! While fun I hope not ever week is like this!!!
20
12
548
17,872
Keyon Vafa retweeted
Scientists are probing whether language models trained on historical data can reproduce creative breakthroughs
go.nature.com/3Tld1sG
go.nature.com/3Tld1sG
20
136
549
182,135
Keyon Vafa retweeted
Lean Verified Transformers (srush.github.io/lean-transform…)
In which we prove a bunch of Transformer invariants from scratch in Lean, and speculate about how hard it would be to do that for the rest of the world's code.
In which we prove a bunch of Transformer invariants from scratch in Lean, and speculate about how hard it would be to do that for the rest of the world's code.
27
126
1,033
66,684
Keyon Vafa retweeted
The Institute for Responsible Superintelligence (RESI): a nonprofit founded by Shafi Goldwasser, Adam Kalai, Vinod Vaikuntanathan. @ShafiGoldwasser @Vinod_MIT @adamfungi
resi.org
Our goal: build scientific foundations needed to make superintelligence safe by design
resi.org
Our goal: build scientific foundations needed to make superintelligence safe by design
9
44
288
96,056
Keyon Vafa retweeted
Last day at Cursor. An incredible place. Stopped by for coffee, they gave me a keyboard that afternoon, and started helping out on Composer the next week. Feel lucky to have worked with such a fearless and ambitious team.
67
12
1,254
67,321
Thrilled to share I'll be joining UC Berkeley next year as an assistant professor in @UCBStatistics and affiliated with @Berkeley_EECS.
My research will build methods to test/improve the implicit world models of AI systems, so that they reflect reality and human understanding.
My research will build methods to test/improve the implicit world models of AI systems, so that they reflect reality and human understanding.
67
21
548
71,407
I'll start at Berkeley in Summer 2027. Until then, I'll be in the Boston area as a postdoctoral fellow at @harvard_data and @bicyclesftmind at MIT.
I'll be recruiting PhD students for Fall 2027. If these problems excite you, reach out anytime.
I'll be recruiting PhD students for Fall 2027. If these problems excite you, reach out anytime.
1
0
18
2,559
I owe this to many mentors, too many to name here.
Special thanks to my PhD advisor David Blei (@blei_lab), my postdoc mentors Sendhil Mullainathan (@m_sendhil) and David Parkes, and my longtime mentors/collaborators @Susan_Athey and Sasha Rush (@srushnlp).
Special thanks to my PhD advisor David Blei (@blei_lab), my postdoc mentors Sendhil Mullainathan (@m_sendhil) and David Parkes, and my longtime mentors/collaborators @Susan_Athey and Sasha Rush (@srushnlp).
1
0
14
1,863
Had a great time chatting with @TaliamSherman on her podcast Tomayto Tomahto.
Check out her full series -- so many great guests and conversations, especially for someone who just graduated from college!
open.spotify.com/episode/46tWNC….
Check out her full series -- so many great guests and conversations, especially for someone who just graduated from college!
open.spotify.com/episode/46tWNC….
0
1
3
1,449
Keyon Vafa retweeted
New paper in @PNASNews: we find that AI agents are sensitive to nudges, often more than humans.
Defaults, suggestions, and highlighted information can alter model behavior substantially—for both better and worse.
We believe this has important implications for agent design and delegation.
Work w/ @nikhilsinghmus @PattieMaes
pnas.org/doi/full/10.10…
Defaults, suggestions, and highlighted information can alter model behavior substantially—for both better and worse.
We believe this has important implications for agent design and delegation.
Work w/ @nikhilsinghmus @PattieMaes
pnas.org/doi/full/10.10…
3
11
25
2,405
RT @2plus2make5: New piece in @TheAtlantic!
We always hear that AI will cure cancer, and I would immediately benefit if it did. But I arg…
We always hear that AI will cure cancer, and I would immediately benefit if it did. But I arg…
0
29
0
18
Keyon Vafa retweeted
Next-token prediction is myopic. What if transformers learn to predict their own next latent state?
🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning nlocks up to 3.3x faster inference via self-speculative decoding! 🚀
🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning nlocks up to 3.3x faster inference via self-speculative decoding! 🚀
54
333
2,321
442,285
Keyon Vafa retweeted
Cool stuff, building on ideas around “world models” by Econs and CS people @keyonV, @asheshrambachan and @m_sendhil.
Question: If you train on next turns for Manhattan taxi drivers, will the model learn a map of Manhattan?
They showed GPTs can be almost perfect at next-token but paradoxically stupid regarding what the right map is.
New paper (NextLat) shows that predicting “next latent state” (jointly with next token) appears to much better at world modeling
It’d be great to see economists get into this game… help build architectures for world models of the global economy
Question: If you train on next turns for Manhattan taxi drivers, will the model learn a map of Manhattan?
They showed GPTs can be almost perfect at next-token but paradoxically stupid regarding what the right map is.
New paper (NextLat) shows that predicting “next latent state” (jointly with next token) appears to much better at world modeling
It’d be great to see economists get into this game… help build architectures for world models of the global economy
Next-token prediction is myopic. What if transformers learn to predict their own next latent state?
🌠 We present 𝗡𝗲𝘅𝘁-𝗟𝗮𝘁𝗲𝗻𝘁 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 (𝗡𝗲𝘅𝘁𝗟𝗮𝘁): a self-supervised learning method that teaches transformers to form compact world models for reasoning nlocks up to 3.3x faster inference via self-speculative decoding! 🚀
2
8
59
13,220
Keyon Vafa retweeted
Recently met @srush_nlp and he started giving me an impromptu lecture on how targeted on-policy self-distillation works.
I asked him if I could record it on my iPhone.
The basic idea is this: if the model made a mistake at some point in the rollout (for example, calling a tool want to discourage this specific error, but we don't want to just learn from the final reward, because it's a very noisy signal spread out over the whole trajectory.
So we have another model read this trajectory and figure where the error was made. It simply inserts some hint tokens to the part of the trajectory right above where the mistake was made.
Now with these injected hint tokens, have the model run a forward pass. You're not having to regenerate a new rollout - aka no new decode required.
The hint causes the model to assign lower probabilities to the error tokens. You then trains the original model to match these new probabilities, teaching it to downweight that specific mistake.
I asked him if I could record it on my iPhone.
The basic idea is this: if the model made a mistake at some point in the rollout (for example, calling a tool want to discourage this specific error, but we don't want to just learn from the final reward, because it's a very noisy signal spread out over the whole trajectory.
So we have another model read this trajectory and figure where the error was made. It simply inserts some hint tokens to the part of the trajectory right above where the mistake was made.
Now with these injected hint tokens, have the model run a forward pass. You're not having to regenerate a new rollout - aka no new decode required.
The hint causes the model to assign lower probabilities to the error tokens. You then trains the original model to match these new probabilities, teaching it to downweight that specific mistake.
42
175
2,558
431,675
Keyon Vafa retweeted
Really looking forward to this panel on AI + Science at Oxford. May 19 at 5pm. Please come!
How is AI reshaping scientific discovery?
Join Prof. Sendhil Mullainathan (MIT & Oxford), Sir Nigel Shadbolt (University of Oxford) & Dr Raia Hadsell (Google DeepMind) for a panel at the University of Oxford.
Free Registration: tinyurl.com/ahfhj3x6
0
4
18
5,310
Keyon Vafa retweeted
Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below!
with @AlecRad and @status_effects 🧵
with @AlecRad and @status_effects 🧵
197
451
3,629
1,468,551