sneak peak there's even a section about sim data in the waitlist questionnaire 👀
one of the growing capabilities of our models is the ability to generalize to sim. We are actively thinking about how to best leverage real and sim data to improve models and deployments!
one of the growing capabilities of our models is the ability to generalize to sim. We are actively thinking about how to best leverage real and sim data to improve models and deployments!
0
0
7
Met some absolute legends of the simulation community, an incredible conference hosted by nvidia and google in london!
4
1
153
@AjdDavison and @VilleKuosmanen for gathering robotics folks in london on such short notice to get dinner with me
1
0
10
There’s the feeling that Astra could just zero shot all the robotics foundation model companies.
Then again one could just leverage astra together with a good robotics model to get even better results
Then again one could just leverage astra together with a good robotics model to get even better results
GPT-6 Astra is the most significant leap in robotics I’ve seen in the past few years. It cracked RoboLab with a near-perfect score. Solid infrastructure + scaling ultimately outperformed the heuristics explored in small-scale studies. We’re definitely on the brink of physical .co/JvQcTZAy5i
18
17
397
real factorio inserter
The AutoFAB system has been tested over 1000 times across 3 deployment sites in the SF Bay Area
autofab.net
2
0
39
easy to miss detail and pretty neat: you can just modify the UMI style data collection devices to use different tools
Why create robot intelligence for just one hand, when we could have it learn from many?
GEN-1, our latest embodied foundation model, now supports a broad range of end effectors from 5-finger hands, to specialized tools, and everything in between.
7
10
103
today in “things I didn’t expect people to run in my sim”: sheep getting herded by XLerobot robots @VectorWang2
3
1
70
@GoDrift_ai a little bit of false advertising here lol?
none of the scenes shown, especially the first one, are generated via prompt. They were hand crafted
none of the scenes shown, especially the first one, are generated via prompt. They were hand crafted
1
0
14
just arrived in SF and excited to begin my research internship at physical intelligence @physical_int working on simulation!
35
3
461
palatial wins again. this time the collision mesh might actually be usable. Very few parts, figured out the table top should be flat and is largely modeled by a few shapes. rigyd generated way too many triangles + oblong ones
although tables are easy and mostly in distribution
although tables are easy and mostly in distribution
1
0
10
palatial's visual wireframe much better, but still overly complicated for a table. not a good look for rigyd
next collision mesh:
next collision mesh:
3
0
11
both palatial and rigyd fixed their apps. Palatial left, rigyd right. Palatial got the textures mostly right.
wireframes+collision meshes in thread
wireframes+collision meshes in thread
input vs output
we are still pretty far away lol (red are generated collision meshes, I presume the 90 degree flip is just a bug but even then it's not great). I wish the best of luck to all the companies in this incredibly difficult (but valuable) part of robotics sim scaling
2
1
18
I reverted my decision to hand off ManiSkill to someone else. A ManiSkill 4 is now in progress with lots of improvements (focusing on robot manipulation)
If you planned to not use ManiSkill because it might not be maintained once I graduate, I hope this convinces you otherwise!
If you planned to not use ManiSkill because it might not be maintained once I graduate, I hope this convinces you otherwise!
6
3
88
input vs output
we are still pretty far away lol (red are generated collision meshes, I presume the 90 degree flip is just a bug but even then it's not great). I wish the best of luck to all the companies in this incredibly difficult (but valuable) part of robotics sim scaling
we are still pretty far away lol (red are generated collision meshes, I presume the 90 degree flip is just a bug but even then it's not great). I wish the best of luck to all the companies in this incredibly difficult (but valuable) part of robotics sim scaling
The only way to scale robotics is to scale simulation. And sim is bottlenecked by high quality assets.
Today we're releasing Palatial V1.0, the first automated pipeline to generate Sim-Ready assets at scale.
Agent-guided and multimodal, it supports soft-body, articulation, and utomated asset generator for Nvidia's Newton simulator. @NVIDIARobotics
Start generating: dashboard.palatial.cloud
7
1
55
in Seattle for 2 weeks finishing up my work at @allen_ai (at their extremely beautiful office). lmk if you want to meet!
3
0
55
some other guessing:
- the shadows in the back are remarkably consistent = background is just gsplat, not objects. Otherwise someone would have to spent a lot of time tuning the room lighting / env map
- robot has wires behind it in real, not sim = they gsplat the office ahead of time, then put tggests also the robot materials are made separately and they use standard rendering + tuned lighting to make the sim look close to real (and not like rendering gsplat parts that some papers have done)
- the shadows in the back are remarkably consistent = background is just gsplat, not objects. Otherwise someone would have to spent a lot of time tuning the room lighting / env map
- robot has wires behind it in real, not sim = they gsplat the office ahead of time, then put tggests also the robot materials are made separately and they use standard rendering + tuned lighting to make the sim look close to real (and not like rendering gsplat parts that some papers have done)
1
0
2
ok I kind of blindly retweeted this last time (I see a neat project using maniskill I usually retweet), but spent a little time looking through this project more
Incredibly well documented, gifs of every task, RLDS/open-x dataset format support (which I never got around to doing some nice benchmarking ideas around memory with categorization, and somehow dense rewards and/or motion planning scripts for 90 something tasks
Incredibly well documented, gifs of every task, RLDS/open-x dataset format support (which I never got around to doing some nice benchmarking ideas around memory with categorization, and somehow dense rewards and/or motion planning scripts for 90 something tasks
🎉 We released MIKASA-Robo-VLA v1.0.0 — a benchmark suite for studying memory in Vision-Language-Action (VLA) policies for tabletop robotic manipulation.
mikasarobo.github.io
🧠 The goal is simple:
make memory evaluation in robotic manipulation more systematic.
👇
2
11
81
First blog post up on Robotics Simulation Infrastructure! I give a high-level overview, followed by an elementary example of better infrastructure for pose management. Link in thread below
would people be interested in a blog series on "simulation infrastructure" where I look at popular robotics sim tools (e.g. mjlab, isaac...) and explain the pros and cons of their design decisions. The series would cover what I believe to be key pillars of good, usable, sim code
7
22
214
I'd like to believe I started the trend for running "shaking" tests to test sim stability 😀 (although I'm sure this existed way before)
I do find it surprising a lot of sims fail this? Very curious to try out the sim when it releases, it claims impressive rendering performance
I do find it surprising a lot of sims fail this? Very curious to try out the sim when it releases, it claims impressive rendering performance
GS-Playground is a pretty wild step for robot learning
photorealistic sim + fast physics finally working together.
Runs ~104 FPS with 3D Gaussian Splatting and handles vision + contact-rich tasks in one system.
2
3
29
Wasn't at GTC but a sneak peak of some of my internship work at NVIDIA was! (sim2real cable routing presented by my manager Yashraj)
youtube.com/watch?v=d5wm5h…
youtube.com/watch?v=d5wm5h…
2
3
49
just found out this company uses maniskill as part of their robotics educational materials :)
youtu.be/1gfD27mCEB8?si…
youtu.be/1gfD27mCEB8?si…
vizuara is so underrated man. if you guys wanna get into learning ml from basics to advanced, vizuara is the way to go.
1
5
41
10 minute coherence is impressive! Was funny to see that their model also controls both the human and robot in the video. There are still plenty of errors though. Once models scale up to reduce quality/dynamics errors, can we even still maintain practical inference speeds?
1/ World models are getting popular in robotics 🤖✨
But there’s a big problem: most are slow and break physical consistency over long horizons.
2/ Today we’re releasing Interactive World Simulator:
An action-conditioned world model that supports stable long-horizon interaction.
inutes of interactive prediction
✅ 15 FPS
✅ on a single RTX 4090🔥
4/ Why this matters: it unlocks two critical robotics applications:
🚀 Scalable data generation for policy training
🧪 Faithful policy evaluation
5/ You can play with our world model NOW at yixuanwang.me/interactive_wo…. NO git clone, NO pip install, NO python. Just click and play!
NOTE ⚠️
ALL videos here are generated purely by our model in pixel space! They are **NOT** from a real camera
More details coming 👇 (1/9)
#Robotics #AI #MachineLearning #WorldModels #RobotLearning #ImitationLearning
3
5
44
I will be giving a talk at UPenn @GRASPlab tomorrow 3-4PM EST on my research in sim/robotics. I’ll be discussing how sim integrated robot learning can drive and accelerate robotics progress. If you are in the area let’s meet up! Link with more details in the thread
2
2
32
I will never understand why isaac lab recommends a full git clone of the project to start using it, but this is a very nice feature to smooth some my headaches
0
0
20
mile long train delaying AGI at neurips 😂, been stuck here blocking the conference venue for a good 20 minutes now
7
2
70
presenting now!
GPU parallelized envs have accelerated RL, but most implementations exhibit critical instability when running on-policy RL with short rollouts. We present Staggered Environment Resets. A few lines of code are all you need!
Presenting today, 4:30PM poster 310 #NeurIPS2025
🧵(1/8)
3
6
195
Scaling: Does throwing more GPUs at the problem help?
With naive resets, performance often saturates as you add envs due to correlated data.
With Staggered Resets, wall-clock time continues to drop as parallelism increases, unlocking the full potential of your hardware (7/8)
With naive resets, performance often saturates as you add envs due to correlated data.
With Staggered Resets, wall-clock time continues to drop as parallelism increases, unlocking the full potential of your hardware (7/8)
1
0
6
We tested on hard ManiSkill3+Isaac Lab tasks with PPO and SAPG.
Staggered Resets (Red) achieves:
🚀Faster wall-clock convergence
📈Higher success rates
🧱Better stability
vs. naive synchronous resets (Blue)
(and can you guess why it doesn't improve for locomotion envs 🤔)
(6/8)
Staggered Resets (Red) achieves:
🚀Faster wall-clock convergence
📈Higher success rates
🧱Better stability
vs. naive synchronous resets (Blue)
(and can you guess why it doesn't improve for locomotion envs 🤔)
(6/8)
2
1
9
The solution is simple and algo-agnostic. Instead of starting every env at t=0, we desynchronize them. Envs are initialized at varied offsets across the task horizon. This ensures every rollout contains a temporally diverse mix of experiences, stabilizing RL training.
(5/8)
(5/8)
1
1
6
For complex robotics tasks, we visualized the states in rollouts. With naive resets (top), the data distribution shifts wildly between updates, difficult to learn from! With Staggered Resets (bottom), the distribution is stable and mimics long trajectories, easy to learn! (4/8)
1
0
4
Via toy environments, we identified when this instability is fatal. Staggered Resets are essential when
1️⃣ Horizon >> Rollout length
2️⃣ Homogeneous Resets: (Envs always starting at the same state)
3️⃣ Loose Gating: Agents exploring later states without mastering earlier ones
(3/8)
1️⃣ Horizon >> Rollout length
2️⃣ Homogeneous Resets: (Envs always starting at the same state)
3️⃣ Loose Gating: Agents exploring later states without mastering earlier ones
(3/8)
1
0
4
To maximize env throughput, we often use short rollouts much smaller than the task horizon.
If envs reset synchronously, we get cyclical nonstationarity:
Batch 1: All early states🟢
...
Batch X: All late states🔴
Reset 🔄
which biases gradients + catastrophic forgetting! (2/8)
If envs reset synchronously, we get cyclical nonstationarity:
Batch 1: All early states🟢
...
Batch X: All late states🔴
Reset 🔄
which biases gradients + catastrophic forgetting! (2/8)
1
0
4
GPU parallelized envs have accelerated RL, but most implementations exhibit critical instability when running on-policy RL with short rollouts. We present Staggered Environment Resets. A few lines of code are all you need!
Presenting today, 4:30PM poster 310 #NeurIPS2025
🧵(1/8)
Presenting today, 4:30PM poster 310 #NeurIPS2025
🧵(1/8)
7
8
152
the 60 paper author's company (Algoverse) has a website claiming ~70% acceptance rates at workshops (which have very low bars for acceptance). I don't think any of their papers are even at the main conference. Some papers look great, many seem kind of just random/slop (although se days...)
its great to encourage more talented people to do research, but this whole website sounds grifty, targeting the parents willing to pay money to get research on their kids' resume + go to dream universities
its great to encourage more talented people to do research, but this whole website sounds grifty, targeting the parents willing to pay money to get research on their kids' resume + go to dream universities
A colleague brought this miraculous situation to my attention. @NeurIPSConf is there an upper limit on the number of workshop papers that a single person can author? Having 60 (sixty) papers in one conference? How is this humanly possible? openreview.net/search?term=~K… #neurips2025
5
3
50
@_aaanh_ @KyumLee0215 reminds me of this guy from pantheon that. Would be interesting if hardware design became incredibly easy and you can just plug and play a foundation model that learns asap
1
0
5
Every fund raise and we get people posting “generational wealth loss” again. Can I claim a record for being one of the earliest people they reached out to 😂
(If I joined I would have been maybe employee #5-7?)
Cursor has changed the world, congratulations on the series D!
(If I joined I would have been maybe employee #5-7?)
Cursor has changed the world, congratulations on the series D!
happy 'missed another opportunity to create generational wealth' day to those who celebrate :)
11
6
369
sim2real experiment died because a quaternion flipped into a subspace my rl policy has never seen before…
that’s a new one for me 🥲
that’s a new one for me 🥲
17
7
235
wait they actually used our keyboard+mouse click and drag teleop system
I thought people didn't like to use this feature after some muted feedback about it 😆. Glad its useful here
I thought people didn't like to use this feature after some muted feedback about it 😆. Glad its useful here
Ever want to enjoy all the privileged information in sim while seamlessly transferring to the real world? How can we correct policy mistakes after deployment?
👉Introducing GSWorld, a real2sim2real photo-realistic simulator with interaction physics with fully open-sourced code.
0
0
12