Pinned Tweet
๐ค What if robots could reason not just in words, but through the dynamics of the physical world?
๐ Introducing ๐ง๐ต๐ถ๐ป๐ธ๐ช๐ผ๐ฟ๐น๐ฑ from @NVIDIAAI !
๐ก By reasoning through shared physical dynamics, robots could learn from human experience and transfer motion intelligence ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐๐ต๐ฎ๐ถ๐ป-๐ผ๐ณ-๐๐๐ป๐ฎ๐บ๐ถ๐ฐ๐ (๐๐ผ๐): A shared motion reasoning space across human and robot embodiments.
โ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Imagines future world representations conditioned on CoD.
โ ๐ฅ๐ ๐ณ๐ฟ๐ผ๐บ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Reinforces reasoning through alignment between imagined and observed futures, enabling learning from human and robot videos.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Robust real-world manipulation under unseen objects and environments.
โ Human-guided skill composition for novel multi-step robot tasks.
โ Robust generalization under task and environment shifts in simulation.
๐ ๐ข๐๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป: Scale generalizable motion intelligence through human videos toward a unified System-1 reasoning layer for Physical AI.
Paper, robot demos & more below! ๐
๐ Introducing ๐ง๐ต๐ถ๐ป๐ธ๐ช๐ผ๐ฟ๐น๐ฑ from @NVIDIAAI !
๐ก By reasoning through shared physical dynamics, robots could learn from human experience and transfer motion intelligence ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐๐ต๐ฎ๐ถ๐ป-๐ผ๐ณ-๐๐๐ป๐ฎ๐บ๐ถ๐ฐ๐ (๐๐ผ๐): A shared motion reasoning space across human and robot embodiments.
โ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Imagines future world representations conditioned on CoD.
โ ๐ฅ๐ ๐ณ๐ฟ๐ผ๐บ ๐ช๐ผ๐ฟ๐น๐ฑ ๐๐บ๐ฎ๐ด๐ถ๐ป๐ฎ๐๐ถ๐ผ๐ป: Reinforces reasoning through alignment between imagined and observed futures, enabling learning from human and robot videos.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Robust real-world manipulation under unseen objects and environments.
โ Human-guided skill composition for novel multi-step robot tasks.
โ Robust generalization under task and environment shifts in simulation.
๐ ๐ข๐๐ฟ ๐๐ถ๐๐ถ๐ผ๐ป: Scale generalizable motion intelligence through human videos toward a unified System-1 reasoning layer for Physical AI.
Paper, robot demos & more below! ๐
11
17
92
5,298
Fu-En (Fred) Yang retweeted
Meet GR00T 2, our new generation of world-action models (WAMs): it imagines how the physical world will unfold and generates the actions that get it there. One model runs on parallel grippers, dexterous hands, and humanoids.
research.nvidia.com/labs/gear/gr00โฆ
research.nvidia.com/labs/gear/gr00โฆ
55
180
1,271
89,213
Fu-En (Fred) Yang retweeted
A finer touch for humanoid robots. ๐ค
At #CoRL2026, NVIDIA is introducing GR00T 2, an open world action model for humanoids, alongside new open industrial benchmarks from @RLWRLD_ai and @UCBerkeley built on Isaac Lab-Arena.
Explore how NVIDIAโs open world action models, ls and agentic workflows help researchers train, test and refine robot skills, from connecting cables to assembling small components.
Discover more โก๏ธ nvda.ws/4ixvdZV
At #CoRL2026, NVIDIA is introducing GR00T 2, an open world action model for humanoids, alongside new open industrial benchmarks from @RLWRLD_ai and @UCBerkeley built on Isaac Lab-Arena.
Explore how NVIDIAโs open world action models, ls and agentic workflows help researchers train, test and refine robot skills, from connecting cables to assembling small components.
Discover more โก๏ธ nvda.ws/4ixvdZV
54
77
528
53,214
Fu-En (Fred) Yang retweeted
we are presenting pi0.7 tomorrow at CoRL (Oral)!
talk: Monday ~10:50am
poster: Monday 5pm, poster 63
talk: Monday ~10:50am
poster: Monday 5pm, poster 63
Our newest model, ฯ0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!
10
24
429
23,139
Fu-En (Fred) Yang retweeted
DreamX: one latent action space for humans and robots
It repurposes a pretrained video diffusion model to learn compact latent actions (300x compression) that capture motion alone, enabling any-to-any video transfer and zero-shot human-to-robot action transfer.
It repurposes a pretrained video diffusion model to learn compact latent actions (300x compression) that capture motion alone, enabling any-to-any video transfer and zero-shot human-to-robot action transfer.
1
8
27
2,186
Fu-En (Fred) Yang retweeted
Embodied models don't just think in text, they can think in visual plans, and fast.
Introducing VFlash, System-1 VLM for Physical AI.
Joint work by @YuCheng_Chou_, Shihao Wang and Jin Huang, LPR Team (@jankautz), GEAR Lab (@DrJimFan), and many other collaborators.
๐ Website: research.nvidia.com/labs/lpr/vflasโฆ
๐งต[1/8] More details in the thread๐
Introducing VFlash, System-1 VLM for Physical AI.
Joint work by @YuCheng_Chou_, Shihao Wang and Jin Huang, LPR Team (@jankautz), GEAR Lab (@DrJimFan), and many other collaborators.
๐ Website: research.nvidia.com/labs/lpr/vflasโฆ
๐งต[1/8] More details in the thread๐
5
28
138
22,167
Fu-En (Fred) Yang retweeted
Weโre introducing ToyBlocks, an open simulated tabletop for controlled experiments in robot learning.
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. ๐งต
github.com/allenai/ToyBloโฆ
It enables robotics researchers to experimentally control training data & models, run targeted evals, & trace failures to their causes. ๐งต
github.com/allenai/ToyBloโฆ
44
43
246
31,618
Fu-En (Fred) Yang retweeted
๐๐ฒ๐ฅ๐ผ๐ฏ๐ผ๐ ๐๐ฌ.๐ณ.๐ฌ ๐ถ๐ ๐๐๐ฉ๐ ๐๐ค
From terabytes of data to prompting robots! Check out all what's new in the video below and join the 50,000+ developers & 300+ organizations in the LeRobot ecosystem ๐ค
From terabytes of data to prompting robots! Check out all what's new in the video below and join the 50,000+ developers & 300+ organizations in the LeRobot ecosystem ๐ค
19
43
280
30,162
Fu-En (Fred) Yang retweeted
Back in ~Dec 2025, I really wanted to work on mobile manipulation, but robots were either too expensive or not performant enough.
So we built our own!
Excited to finally share something Iโve had so much fun working on:
1. Pixel: a co-designed full stack system for learning Pixel-UMI: environment aware whole body manipulation without robot data.
w/ @JasonJZLiu
(1/N)
So we built our own!
Excited to finally share something Iโve had so much fun working on:
1. Pixel: a co-designed full stack system for learning Pixel-UMI: environment aware whole body manipulation without robot data.
w/ @JasonJZLiu
(1/N)
26
55
342
39,917
Fu-En (Fred) Yang retweeted
Nobody trained DreamX to move human motion onto a robot, and it does it anyway. KAIST and NVIDIA turned a pretrained video diffusion model into a latent action learner. Show it a human video and a robotโs first frame, and it generates the robot repeating the motion.
1
3
20
926
Fu-En (Fred) Yang retweeted
Excited to share one of the most ambitious projects from my CMU lab, led by @_tonytao_ and @JasonJZLiu.
This work shows that SOTA robot capabilities don't necessarily require massive resources. By carefully co-designing the entire robotics stack, one can achieve s at low cost:
(1) Pixel: A humanoid robot under $15K.
(2) Pixel-UMI: A training framework enabling environment-aware, whole-body manipulation for dexterous and mobile tasks.
TL;DR: For under $15K, anyone can build this robot and train it with our framework to reproduce mobile dexterity results at the level of most robotics frontier labs.
We hope it is a meaningful step toward democratizing general-purpose robots capable of operating in complex, real-world environments.
This work shows that SOTA robot capabilities don't necessarily require massive resources. By carefully co-designing the entire robotics stack, one can achieve s at low cost:
(1) Pixel: A humanoid robot under $15K.
(2) Pixel-UMI: A training framework enabling environment-aware, whole-body manipulation for dexterous and mobile tasks.
TL;DR: For under $15K, anyone can build this robot and train it with our framework to reproduce mobile dexterity results at the level of most robotics frontier labs.
We hope it is a meaningful step toward democratizing general-purpose robots capable of operating in complex, real-world environments.
Excited to finally share Pixel: a full-stack robot learning project @_tonytao_ and I worked on earlier this year!
Weโre releasing two papers:
1. Pixel: A low-cost mobile manipulator, co-designed with its UMI system and learning stack. pixel-robot.com/pixel/
2. Pixel-UMI: or learning environment-aware, whole-body manipulation
pixel-robot.com/pixel-umi/
(1/N)
35
28
457
39,441
Fu-En (Fred) Yang retweeted
Think evaluating your model in the real world is too much work?
If youโre in academia, weโve got you covered! Weโre offering 10,000 hours of FREE real-world, in-the-wild robot evaluations, conducted by our dedicated full-time team across our fleet of 100+ robots.
Yes, completely free! ๐ค
Sign up today: app.youform.com/forms/riubycrq
If youโre in academia, weโve got you covered! Weโre offering 10,000 hours of FREE real-world, in-the-wild robot evaluations, conducted by our dedicated full-time team across our fleet of 100+ robots.
Yes, completely free! ๐ค
Sign up today: app.youform.com/forms/riubycrq
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread โ
0
4
17
1,334
Fu-En (Fred) Yang retweeted
Robotics needs its own Hubble Space Telescope or CERN, a scientific megaproject to cut through the noise in physical AI.
Another paper, another launch, another โbestโ model. Do those gains hold up on real robots, at scale, in the wild?
We built RoboColosseum to find out.
๐๐งต
Another paper, another launch, another โbestโ model. Do those gains hold up on real robots, at scale, in the wild?
We built RoboColosseum to find out.
๐๐งต
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread โ
15
14
85
4,120
Fu-En (Fred) Yang retweeted
Introducing RoboColosseum
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread โ
An open robot eval initiative by @huggingface @BitRobotNetwork @eaic_sg @DJiafei
Access:
- 100+ robots (@UnitreeRobotics G1s, @LeRobotHF SO-101s, YAMs, Franka and more)
- 140+ in-the-wild scenes
- Open rollout data
- 10,000 FREE eval hrs
Thread โ
55
74
454
128,744
Fu-En (Fred) Yang retweeted
Happy to share our work DreamX, on universal latent action learning! ๐ฅ
DreamX supports cross-embodiment video and action transfer, maximizing the use of video modelโs rich generative prior.
Check out more video and results on our project page: cvlab-kaist.github.io/DreamX/
DreamX supports cross-embodiment video and action transfer, maximizing the use of video modelโs rich generative prior.
Check out more video and results on our project page: cvlab-kaist.github.io/DreamX/
๐ฅ What if video diffusion models could learn a universal language of actions across humans and robots?
๐ Introducing ๐๐ฟ๐ฒ๐ฎ๐บ๐ซ: Video Diffusion Model Is a Universal Latent Action Learner!
DreamX extracts compact, transferable latent actions directly from pretrained video ating motion from visual appearance.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐ฉ๐ถ๐ฑ๐ฒ๐ผ ๐ฑ๐ถ๐ณ๐ณ๐๐๐ถ๐ผ๐ป ๐ฝ๐ฟ๐ถ๐ผ๐ฟ๐: Built on Cosmos-Predict2.5, DreamX extracts action-relevant dynamics directly from pretrained spatiotemporal features.
โ ๐ ๐ผ๐๐ถ๐ผ๐ป ๐ฑ๐ถ๐๐ฒ๐ป๐๐ฎ๐ป๐ด๐น๐ฒ๐บ๐ฒ๐ป๐: Uses generative priors and future-video reconstruction to separate motion from appearance.
โ ๐๐ผ๐บ๐ฝ๐ฎ๐ฐ๐ ๐น๐ฎ๐๐ฒ๐ป๐ ๐ฝ๐ผ๐น๐ถ๐ฐ๐: Predicts compact latent action codes instead of future videos for efficient robot control.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Any-to-any video transfer across scenes and embodiments.
โ Zero-shot human-to-robot video and executable action transfer without paired humanโrobot training data.
โ High-quality latent action representations, achieving state-of-the-art performance on LARYBench.
Paper, videos & demos below! ๐
0
2
40
1,902
๐ฅ What if video diffusion models could learn a universal language of actions across humans and robots?
๐ Introducing ๐๐ฟ๐ฒ๐ฎ๐บ๐ซ: Video Diffusion Model Is a Universal Latent Action Learner!
DreamX extracts compact, transferable latent actions directly from pretrained video ating motion from visual appearance.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐ฉ๐ถ๐ฑ๐ฒ๐ผ ๐ฑ๐ถ๐ณ๐ณ๐๐๐ถ๐ผ๐ป ๐ฝ๐ฟ๐ถ๐ผ๐ฟ๐: Built on Cosmos-Predict2.5, DreamX extracts action-relevant dynamics directly from pretrained spatiotemporal features.
โ ๐ ๐ผ๐๐ถ๐ผ๐ป ๐ฑ๐ถ๐๐ฒ๐ป๐๐ฎ๐ป๐ด๐น๐ฒ๐บ๐ฒ๐ป๐: Uses generative priors and future-video reconstruction to separate motion from appearance.
โ ๐๐ผ๐บ๐ฝ๐ฎ๐ฐ๐ ๐น๐ฎ๐๐ฒ๐ป๐ ๐ฝ๐ผ๐น๐ถ๐ฐ๐: Predicts compact latent action codes instead of future videos for efficient robot control.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Any-to-any video transfer across scenes and embodiments.
โ Zero-shot human-to-robot video and executable action transfer without paired humanโrobot training data.
โ High-quality latent action representations, achieving state-of-the-art performance on LARYBench.
Paper, videos & demos below! ๐
๐ Introducing ๐๐ฟ๐ฒ๐ฎ๐บ๐ซ: Video Diffusion Model Is a Universal Latent Action Learner!
DreamX extracts compact, transferable latent actions directly from pretrained video ating motion from visual appearance.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
โ ๐ฉ๐ถ๐ฑ๐ฒ๐ผ ๐ฑ๐ถ๐ณ๐ณ๐๐๐ถ๐ผ๐ป ๐ฝ๐ฟ๐ถ๐ผ๐ฟ๐: Built on Cosmos-Predict2.5, DreamX extracts action-relevant dynamics directly from pretrained spatiotemporal features.
โ ๐ ๐ผ๐๐ถ๐ผ๐ป ๐ฑ๐ถ๐๐ฒ๐ป๐๐ฎ๐ป๐ด๐น๐ฒ๐บ๐ฒ๐ป๐: Uses generative priors and future-video reconstruction to separate motion from appearance.
โ ๐๐ผ๐บ๐ฝ๐ฎ๐ฐ๐ ๐น๐ฎ๐๐ฒ๐ป๐ ๐ฝ๐ผ๐น๐ถ๐ฐ๐: Predicts compact latent action codes instead of future videos for efficient robot control.
๐ช๐ต๐ฎ๐ ๐ถ๐ ๐ฒ๐ป๐ฎ๐ฏ๐น๐ฒ๐
โ Any-to-any video transfer across scenes and embodiments.
โ Zero-shot human-to-robot video and executable action transfer without paired humanโrobot training data.
โ High-quality latent action representations, achieving state-of-the-art performance on LARYBench.
Paper, videos & demos below! ๐
3
15
82
5,191
๐ Explore DreamX!
๐ Paper (arXiv): arxiv.org/abs/2611.05377
๐ค Hugging Face Paper: huggingface.co/papers/2611.05โฆ
๐ Project Page: cvlab-kaist.github.io/DreamX/
Check out our any-to-any video transfer, zero-shot human-to-robot action transfer, and real-robot policy demos! ๐ค
๐ Paper (arXiv): arxiv.org/abs/2611.05377
๐ค Hugging Face Paper: huggingface.co/papers/2611.05โฆ
๐ Project Page: cvlab-kaist.github.io/DreamX/
Check out our any-to-any video transfer, zero-shot human-to-robot action transfer, and real-robot policy demos! ๐ค
1
1
5
443
Huge thanks to our amazing DreamX team for all the contributions and collaboration! ๐
Jisoo Kim* (@zixxuu ), Chaehyun Kim* (@ChaehyunK87776 ), Junsu Kim (@jjunos0926 ), Jiho Park, Seongchan Kim, Eunbeen Hong, Soowon Son, Fu-En Yang (@FuEnYang1 ), and Prof. Seungryong Kim.
Grateful for this collaboration between @KAIST_AI and @NVIDIAAI !
๐ DreamX has also been accepted to the #CoRL2026 Scaling H2R workshop!
Excited to explore how video foundation models can unlock more generalizable robot intelligence! ๐ค
Jisoo Kim* (@zixxuu ), Chaehyun Kim* (@ChaehyunK87776 ), Junsu Kim (@jjunos0926 ), Jiho Park, Seongchan Kim, Eunbeen Hong, Soowon Son, Fu-En Yang (@FuEnYang1 ), and Prof. Seungryong Kim.
Grateful for this collaboration between @KAIST_AI and @NVIDIAAI !
๐ DreamX has also been accepted to the #CoRL2026 Scaling H2R workshop!
Excited to explore how video foundation models can unlock more generalizable robot intelligence! ๐ค
0
0
6
499
Fu-En (Fred) Yang retweeted
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
โข Novel view appearance
โข Multi-view consistent RGB-D
โข 3D geometry
โข Spatially aligned tactile world model now at jwang771.github.io/3dworldsimulatโฆ!
Our model predicts in a unified 3D representation space, capable of generating:
โข Novel view appearance
โข Multi-view consistent RGB-D
โข 3D geometry
โข Spatially aligned tactile world model now at jwang771.github.io/3dworldsimulatโฆ!
57
164
1,758
166,918