Pinned Tweet
This one has been a long time coming: today we’re introducing MEM, an approach for giving VLAs short-term and long-term memory.
Memory is such an obvious capability, but adding it isn’t easy (most VLAs today are memory-less). A short thread on challenges, solutions, and the new s for us.
Memory is such an obvious capability, but adding it isn’t easy (most VLAs today are memory-less). A short thread on challenges, solutions, and the new s for us.
8
11
115
10,412
Karl Pertsch retweeted
π is looking to opportunistically hire for a very special role : )
we're growing our contributions to the robotics ecosystem and hiring a scrappy generalist with a deep understanding of robotics & hardware to own this push. it'll be an open mandate to roll up your sleeves to do fun projects and build an incredible community (e.g. sponsoring grants, giving away HW, hackathon ecosystems).
please dm / tag exceptional people who have worked in public!
we're growing our contributions to the robotics ecosystem and hiring a scrappy generalist with a deep understanding of robotics & hardware to own this push. it'll be an open mandate to roll up your sleeves to do fun projects and build an incredible community (e.g. sponsoring grants, giving away HW, hackathon ecosystems).
please dm / tag exceptional people who have worked in public!
14
15
367
65,421
Over the last few weeks we have seen some evidence of benchmark manipulation on RoboArena, in part flagged by members of the robotics community. We take this very seriously and have taken steps to protect the integrity of the benchmark and ensure fair, unbiased evaluations.
Most importantly, going forward only organizations without an active policy on the leaderboard will be permitted to submit evaluations. We have applied these changes retroactively across the period of suspicious benchmark activity, to make sure all evals on the leaderboard can be trusted.
A big thank you to all the RoboArena evaluators who volunteer their time to provide unbiased robot evaluations, and to the community members who helped audit the evaluation results and flagged potential wrong-doing.
Most importantly, going forward only organizations without an active policy on the leaderboard will be permitted to submit evaluations. We have applied these changes retroactively across the period of suspicious benchmark activity, to make sure all evals on the leaderboard can be trusted.
A big thank you to all the RoboArena evaluators who volunteer their time to provide unbiased robot evaluations, and to the community members who helped audit the evaluation results and flagged potential wrong-doing.
We observed evidence of benchmark hacking on RoboArena since April. We have taken steps to prevent this in the future, and we have rolled back evals in accordance with these steps to retain the integrity of the benchmark. Read more about our changes here: robo-arena.github.io.
3
1
34
8,074
Karl Pertsch retweeted
Excited to be in Rio this week to present RETAIN (w/ @zhiyuan_zhou_ , @ajwagenmaker, @KarlPertsch, and @svlevine) at #ICLR2026! 🇧🇷
Saturday 10:30 AM – 1:00 PM at P3-#1208.
Project Page: retain.yajatyadav.com
x.com/zhiyuan_zhou_/….
Saturday 10:30 AM – 1:00 PM at P3-#1208.
Project Page: retain.yajatyadav.com
x.com/zhiyuan_zhou_/….
Do you ever find finetuning VLA overfits to the target task, to the point where generalist ability is lost and even minor deviations beyond the SFT data break the policy?
We found an extremely simple solution: directly merge the base and finetuned policy in weight space 🤯
👇🧵
2
12
61
19,211
Happy to share some new results! π0.7 comes with memory, and algorithmic advances to pull out more performance and generalization from diverse training data! Check it out!
Our newest model, π0.7, has some interesting emergent capabilities: it can control a new robot to fold shirts for which we had no shirt folding data, figure out how to use an appliance with language-based coaching, and perform a wide range of dexterous tasks all in one model!
1
1
57
5,050
Karl Pertsch retweeted
We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
40
292
2,189
447,532
Karl Pertsch retweeted
late but: RETAIN will be presented at #ICLR2026 in Rio! The code is also out at github.com/yajatyadav/RET…, though all you really need is this one line
Do you ever find finetuning VLA overfits to the target task, to the point where generalist ability is lost and even minor deviations beyond the SFT data break the policy?
We found an extremely simple solution: directly merge the base and finetuned policy in weight space 🤯
👇🧵
1
3
26
4,165
Karl Pertsch retweeted
Many real-world tasks require memory to be successful. Yet, most robots don’t have any form of memory. Today, we are going to change that. We developed a system called MEM that introduces memory into VLAs on multiple scales
We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory.
Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇
5
12
63
5,774
Karl Pertsch retweeted
We equipped PI policies with memory!
And taught our robots to do long-horizon real world tasks such as preparing the items for a recipe, cooking a grilled cheese and cleaning the kitchen!
And taught our robots to do long-horizon real world tasks such as preparing the items for a recipe, cooking a grilled cheese and cleaning the kitchen!
We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory.
Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇
8
15
89
24,122
Karl Pertsch retweeted
We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory.
Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇
Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇
48
256
2,135
462,921
Finally, many prior works have reported that policies get *worse* on dexterous tasks when adding memory (because of spurious correlations, causal confusion etc). We find that by equipping pi06 with MEM and training it on our most diverse data mix, we can match pi06 performance on re memory (while clearly outperforming on memory tasks).
This is IMO one of the biggest results here: we have a recipe for adding memory to VLAs without significant tradeoffs, both in terms of latency and performance!
This is IMO one of the biggest results here: we have a recipe for adding memory to VLAs without significant tradeoffs, both in terms of latency and performance!
1
0
4
411
This was one of the longest-running research projects at pi — adding memory to your models stretches all parts of your infra and needs innovation on the whole stack.
The project started as @HomerWalke's internship project with @DannyDriess, but had lots of help from countless people at pi to get over the finish line. Special shoutout to @marceltornev who worked tirelessly to teach our models the long-horizon behaviors you saw in the videos above!
For more details, check out our blog & paper: pi.website/research/memory
The project started as @HomerWalke's internship project with @DannyDriess, but had lots of help from countless people at pi to get over the finish line. Special shoutout to @marceltornev who worked tirelessly to teach our models the long-horizon behaviors you saw in the videos above!
For more details, check out our blog & paper: pi.website/research/memory
0
0
6
420
Use DROID data and PolaRiS sim evals to test your ideas on strong generalist policies!
Congrats to the CoVer-VLA team!
Congrats to the CoVer-VLA team!
🧵(6) DROID Eval
CoVer-VLA achieves 14% gains in task progress and 9% in success rate on the challenging red-team PolaRiS benchmark. In the pan cleaning task, π₀.₅ shows incorrect intent, grasping the pan handle. In contrast, CoVer-VLA correctly uses sponge to scrub the pan.
1
0
27
3,110
Very exciting to see first steps of our models doing useful things in the world!
Thanks to Ultra and Weave for being great partners in these deployments!
Thanks to Ultra and Weave for being great partners in these deployments!
General-purpose AI models are behind some of the most exciting applications we now can't live without. We envision that an analogous “physical intelligence layer” built with models like π0.6 will similarly spur a new wave of applications for the physical world.
We’ve recently begun working with a handful of companies that have deployed their robots to do real-world, useful things.
pi.website/blog/partner/?…
0
0
31
1,943
Check out Will's new project! By increasing the "prompting surface" of a VLA (keypoints, language at multiple levels of abstraction) we can get VLMs to steer them much more effectively.
This allows us to "import" lots of useful VLM capabilities (reasoning, in-context learning) and push overall system performance!
This allows us to "import" lots of useful VLM capabilities (reasoning, in-context learning) and push overall system performance!
How can robot policies be trained to best leverage VLMs' CoT reasoning and in-context learning for generalization?
The key is Steerable Policies: vision-language-action models that can be flexibly controlled in many ways!
steerable-policies.github.io
1/9
1
7
76
7,643
Check out Tony's new work on training and evaluating robot reward models -- accurate, scalable rewards are key for learning from online robot experience!
In RoboReward we introduce a benchmark & new reward models that significantly outperform off-the-shelf VLMs in their ability to provide accurate rewards. Check out Tony's thread for all the details!
In RoboReward we introduce a benchmark & new reward models that significantly outperform off-the-shelf VLMs in their ability to provide accurate rewards. Check out Tony's thread for all the details!
Reliable rewards are a bottleneck for real-world RL for robotics: human labels are costly, and handcrafted rewards are brittle.
In RoboReward 🤖💰, we study VLMs as reward models and find they are unreliable across tasks, embodiments, and scenes.
Paper: arxiv.org/abs/2601.00675
2
1
13
2,617
Also shoutout to Andrew (@ajwagenmaker) for mentoring Tony throughout this project and providing his RL expertise! :)
x.com/ajwagenmaker/s…
x.com/ajwagenmaker/s…
Reliable rewards are critical for effective RL, yet in most robotic applications obtaining such rewards requires significant task-specific human effort.
Can we do better? Check out RoboReward, our new generalist, language-conditioned reward model for real-world robot RL!
0
1
4
1,149