Pinned Tweet
Does GenAI create new knowledge? arxiv.org/abs/2602.06021 gives
* 1st explicit characterization of diffusion model's generalization
* more precise than offered by classical stat. learning theory
* systematic integration of various inductive biases (training+architecture+inference)
* 1st explicit characterization of diffusion model's generalization
* more precise than offered by classical stat. learning theory
* systematic integration of various inductive biases (training+architecture+inference)
3
28
178
18,307
Molei Tao retweeted
I've posted some slides on my website outlining OpenAI's approach to disproving the exponential time hypothesis: msellke.com/Files/ETH-Fals…
The outline skips some steps, so it gains a smaller power of log(n) in the exponent than the fully optimized algorithm.
The outline skips some steps, so it gains a smaller power of log(n) in the exponent than the fully optimized algorithm.
6
36
189
18,313
Molei Tao retweeted
Tired of waiting for your diffusion models?
🚢 We just shipped a major FastGen update:
- new models: H3, LTX-2.5, LingBot-World, Cosmos 3, FLUX.2
- new methods: PDD, AYF, AnyFlow, Causal Forcing
- new features: multimodal distillation, image editing, camera conditioning, KV UI
🔗 Try it out here: github.com/NVlabs/fastgen
🚢 We just shipped a major FastGen update:
- new models: H3, LTX-2.5, LingBot-World, Cosmos 3, FLUX.2
- new methods: PDD, AYF, AnyFlow, Causal Forcing
- new features: multimodal distillation, image editing, camera conditioning, KV UI
🔗 Try it out here: github.com/NVlabs/fastgen
13
49
422
33,950
Molei Tao retweeted
Generating proteins -> ??? -> Testing them in the lab
The missing piece is SELECTBENCH – evaluating your generations before sending them to the lab
- We design metrics to evaluate your model on the selectivity task
- We collect three datasets of ON/OFF targets from different sources and of different difficulty levels
- We study how modern generative models perform on these tasks (far from perfect)
- With SELECTBENCH, the question of generating proteins becomes the question of the throughput of your model!
- Check it out at #neurips2026 and benchmark your model using the code from github.com/lovetaro/selec…
This is one of the links in our long chain of works on generating selective proteins (see my slides for the synthesis docs.google.com/presentation/d…)
The missing piece is SELECTBENCH – evaluating your generations before sending them to the lab
- We design metrics to evaluate your model on the selectivity task
- We collect three datasets of ON/OFF targets from different sources and of different difficulty levels
- We study how modern generative models perform on these tasks (far from perfect)
- With SELECTBENCH, the question of generating proteins becomes the question of the throughput of your model!
- Check it out at #neurips2026 and benchmark your model using the code from github.com/lovetaro/selec…
This is one of the links in our long chain of works on generating selective proteins (see my slides for the synthesis docs.google.com/presentation/d…)
A molecule can bind the right protein and still be the wrong drug. Why? Because it might also bind proteins we don't want it to, potentially causing serious side effects.
📄 Paper: arxiv.org/abs/2610.24124
2
19
133
6,172
Molei Tao retweeted
Please apply. No teaching, complete freedom! You can work with any faculty!
UC Berkeley Department of Statistics is hiring! We’re seeking applications for our Postdoctoral Neyman Fellow position.
Details & apply: aprecruit.berkeley.edu/JPF05588
#Statistics #Probability #DataScience #FacultyJobs #UCBerkeley
2
5
80
11,406
Molei Tao retweeted
Parallel Decoding Distillation (PDD) is taking off 🛫
We’ve only released PDD end of July and it’s already adopted in:
- Alibaba’s MiniMax H3: huggingface.co/alibaba-pai/Mi…
- WorldPlay2’s causal distillation: worldplay2.github.io
- FastVideo’s FastH3 Omni: haoailab.com/blogs/fastvide…
- NVIDIA’s large language continuous diffusion model: arxiv.org/pdf/2610.02665
- Amazon’s generative forecasting model: arxiv.org/abs/2610.22358
- NVIDIA’s & MSR’s extensions to parametric student trajectories: alan-lanfeng.github.io/PTD & arxiv.org/pdf/2610.18472
We’ll be open-sourcing our implementation soon - stay tuned!
We’ve only released PDD end of July and it’s already adopted in:
- Alibaba’s MiniMax H3: huggingface.co/alibaba-pai/Mi…
- WorldPlay2’s causal distillation: worldplay2.github.io
- FastVideo’s FastH3 Omni: haoailab.com/blogs/fastvide…
- NVIDIA’s large language continuous diffusion model: arxiv.org/pdf/2610.02665
- Amazon’s generative forecasting model: arxiv.org/abs/2610.22358
- NVIDIA’s & MSR’s extensions to parametric student trajectories: alan-lanfeng.github.io/PTD & arxiv.org/pdf/2610.18472
We’ll be open-sourcing our implementation soon - stay tuned!
Having worked with many distillation methods for accelerating diffusion & flow models, I’m still blown away by the simplicity of Parallel Decoding Distillation (PDD).
1️⃣ State-of-the-art quality and diversity with variable NFE
2️⃣ First to scale trajectory-distillation to few-PDD is about as cheap and simple as it gets:
🎯single, regression-based MSE loss using 1 student & 1 detached teacher forward
❌NO data required
❌NO adversarial loss or alternating training
❌NO JVP or finite-difference approx.
❌NO additional time-embedding or trainable models (e.g. fake score, discriminator)
❌NO stabilization tricks (e.g. normalization, clipping, warm-up)
2
11
107
5,695
Molei Tao retweeted
Join @FlatironInst @SimonsFdn as a Research Fellow in NYC 🚀 Lead cutting-edge work in computational science & ML with top collaborators. Apply by Nov 15: apply.interfolio.com/193799
More on ML@CCM 👉 users.flatironinstitute.org/~lsaul/ml_ccm.…
Keen to work with folks in #AIforScience & #GenerativeAI
More on ML@CCM 👉 users.flatironinstitute.org/~lsaul/ml_ccm.…
Keen to work with folks in #AIforScience & #GenerativeAI
1
14
51
3,984
Molei Tao retweeted
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org
Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
27
203
661
64,504
Molei Tao retweeted
RFDiffusion or Proteina? You don't want to miss this one 🔥🧬
Avengers assemble: 1 year BakerLab + NVIDIA GenAIR = 🔥 rfd4-proteina.org
Presenting RFD4-Proteina, our latest protein design model, significantly better and faster than everything we compared against, both in silico and in the lab! 1/30
0
5
15
706
Molei Tao retweeted
🚀 Today at GTC Berlin we introduced RFD4-Proteina! A close one-year collaboration between GenAIR and David Baker's group at @UWproteindesign !
🧬 Design proteins faster and send fewer, better candidates to the lab.
Introducing RFD4-Proteina, a new NVIDIA BioNeMo open model co-developed with @UWproteindesign.
▫️Fast: 13× faster generation than RFdiffusion3 at 200 residues
▫️ Smart: state-of-the-art across 12+ protein mable: set your design inputs and use test-time compute on harder problems
Start with our binder-design tutorial nvda.ws/3VK9Uvr
3
9
35
1,440
Molei Tao retweeted
Check out RVM: simple, effective RL fine-tuning for diffusion models ⚡
Roll out (no grads) → compute rewards → re-noise → reward-weighted velocity regression + anchoring
Unifies previous approaches and scales to video models with SOTA performance!
Roll out (no grads) → compute rewards → re-noise → reward-weighted velocity regression + anchoring
Unifies previous approaches and scales to video models with SOTA performance!
Simple, efficient, scalable diffusion RL without likelihood ratios?
Meet RVM: Reward-based Velocity Matching
🎯 Rewards directly in velocity space
⚡ Cheap video fine-tuning
🧩 Unifies recent objectives
🎬 Fixes the static-video problem by dynamic motion reward
How it works 🧵
1
5
25
1,447
Molei Tao retweeted
Exciting to see the community building on FastGen and pushing distillation techniques forward! 🚀
This cool new work builds on ideas from our PDD and PiFlow, exploring novel parameterizations of student heads. Great to see these ideas evolving!
This cool new work builds on ideas from our PDD and PiFlow, exploring novel parameterizations of student heads. Great to see these ideas evolving!
Excited to introduce Simplified Matching Distillation (SMD) for few-step diffusion🧵
No auxiliary score model. No adversarial training. No generator modification.
You will get:
• Photorealistic appearance
• High-quality dynamics
• Memory-efficient training
• Mode covering MD works well across images, video, joint audio–video, and 3D gen, validated across model scales up to 33B parameters.
Besides, SMD supports:
- Multi-teacher capability merging (like OPD in LLM)
- Cross-model capability transfer
- Bidirectional teacher to causal distillation
Full release. Paper, code, model and data.
Project Page: zjwfufu.github.io/SMD-Page/
Paper: arxiv.org/abs/2610.18472
Code: github.com/zjwfufu/SMD
Model Card & Data: huggingface.co/zjwfufu/SMD
Demo Video 👇
0
8
61
4,118
Molei Tao retweeted
1/4 Today, we’re releasing Mercury 3. Mercury 3 is by far our most powerful model and the most intelligent diffusion LLM ever released. It outperforms GPT-6 Luna and Haiku 5.5 on quality while running at 1000+ tok/sec.
289
456
5,273
485,518
Molei Tao retweeted
A nice presentation by @paveltyletski on
Diffusion Gemma: 4x faster text generation.
I think that text diffusion will not only be 4x faster and 10x more energy efficient, but eventually it will also be of higher quality, and supersede autoregressive LLMs.
youtu.be/3n5RN7i8h2Y?si… via @YouTube
Diffusion Gemma: 4x faster text generation.
I think that text diffusion will not only be 4x faster and 10x more energy efficient, but eventually it will also be of higher quality, and supersede autoregressive LLMs.
youtu.be/3n5RN7i8h2Y?si… via @YouTube
13
41
220
16,547
Molei Tao retweeted
I recently moved to TU Darmstadt (Germany) and am currently hiring PhD student through ELISS.
ellis.eu/news/ellis-phd…
If you interested, please apply and choose me as a potential advisor. Topics include Continual/Distributed/Federated/Active/Transparent AI.
ellis.eu/news/ellis-phd…
If you interested, please apply and choose me as a potential advisor. Topics include Continual/Distributed/Federated/Active/Transparent AI.
8
35
217
10,290
Molei Tao retweeted
1/13. Rufus-Air (arxiv.org/abs/2609.29421) 🧵
Sep 2025–Apr 2026, minus a 2-month production pause: a small research team at Amazon Search built an open post-training recipe from scratch on GLM-4.5-Air-Base, using public data and open-source components, rarely over 400 H200s.
Sep 2025–Apr 2026, minus a 2-month production pause: a small research team at Amazon Search built an open post-training recipe from scratch on GLM-4.5-Air-Base, using public data and open-source components, rarely over 400 H200s.
3
27
116
12,182
Molei Tao retweeted
We are hiring research interns in the broad area of machine learning and statistics @MSRNE @MSFTResearch! Come work with us!
Personally I have very broad interests and I’d love to learn about your favorite problems!
Personally I have very broad interests and I’d love to learn about your favorite problems!
5
16
138
7,545
Molei Tao retweeted
Deep learning is often taught as a sanitized collection of facts and math. But work at the frontier of AI is messy and empirical.
Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch.
on by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens?
deep-learning-alchemy.github.io
My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments).
We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up.
The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue
Try some quizzes below!
Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch.
on by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens?
deep-learning-alchemy.github.io
My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments).
We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up.
The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue
Try some quizzes below!
44
312
2,460
178,522
Molei Tao retweeted
Excited to share our latest RL method for diffusion models!
The cord idea behind this simple and effective object: collect final samples from PFODE rollout and their rewards, then compute an advantage-weighted pretraining loss to reinforce/forget good/bad generated samples
The cord idea behind this simple and effective object: collect final samples from PFODE rollout and their rewards, then compute an advantage-weighted pretraining loss to reinforce/forget good/bad generated samples
Simple, efficient, scalable diffusion RL without likelihood ratios?
Meet RVM: Reward-based Velocity Matching
🎯 Rewards directly in velocity space
⚡ Cheap video fine-tuning
🧩 Unifies recent objectives
🎬 Fixes the static-video problem by dynamic motion reward
How it works 🧵
0
6
51
2,777
Latest result in fine tuning of diffusion model, general method, but demonstrated highly effective even for video generations.
Simple, efficient, scalable diffusion RL without likelihood ratios?
Meet RVM: Reward-based Velocity Matching
🎯 Rewards directly in velocity space
⚡ Cheap video fine-tuning
🧩 Unifies recent objectives
🎬 Fixes the static-video problem by dynamic motion reward
How it works 🧵
0
2
32
2,195