Congratulations to @zheyuanzhang99 for receiving the Amazon AI fellowship!
I’m incredibly honored to be selected as an Amazon AI PhD Fellow! I’m deeply grateful to @amazon, @AmazonScience for this generous award, and to Johns Hopkins University and the Department of Computer Science for their support. I’m also sincerely thankful to my advisors for their invaluable guidance and encouragement!
1
1
23
1,738
Tianmin Shu retweeted
Congratulations to @BafnaNiyati, @stolenpyjak, and @zheyuanzhang99 on being named @amazon Doctoral Fellows! 🎉 They were selected based on their outstanding publication records, research proposals, and mentor support. cs.jhu.edu/news/three-cs-…
0
4
45
7,811
Robot teammates in the real world may not be perfect. How to infer and adapt the physical limits of a new robot teammate? See @suyu_ye’s new work on adaptive robot teaming.
What can you learn about a robot partner that still matters when the task changes?
Its physical limits.
We introduce the problem of inferring those limits by watching it work with another robot, then coordinating with it zero-shot on a new task.
Introducing Watch, Infer, e: watch-infer-coordinate.github.io
Preprint: arxiv.org/abs/2610.02170
0
1
14
932
How to adapt latent world models to new dynamics via persistent test-time training? Check out JEPA-TTT led by @zheyuanzhang99
A pretrained latent world model's predictions can become unreliable when test-time dynamics differ from training.
Introducing JEPA-TTT, which enables persistent test-time training of latent world models for planning under dynamics shifts.
Project Page: jepa-ttt.github.io
5pKK8yu
0
6
82
7,406
Tianmin Shu retweeted
A pretrained latent world model's predictions can become unreliable when test-time dynamics differ from training.
Introducing JEPA-TTT, which enables persistent test-time training of latent world models for planning under dynamics shifts.
Project Page: jepa-ttt.github.io
5pKK8yu
Introducing JEPA-TTT, which enables persistent test-time training of latent world models for planning under dynamics shifts.
Project Page: jepa-ttt.github.io
5pKK8yu
5
15
66
15,942
Tianmin Shu retweeted
Have we built machines that think like humans? 🧠🤖
In a new preprint, we propose CogGym, which compares AI and human judgments across 258 commonsense-reasoning experiments from 100 cognitive-science papers.
Paper: arxiv.org/abs/2609.21259
Platform: coggym.org
In a new preprint, we propose CogGym, which compares AI and human judgments across 258 commonsense-reasoning experiments from 100 cognitive-science papers.
Paper: arxiv.org/abs/2609.21259
Platform: coggym.org
6
38
136
17,067
Tianmin Shu retweeted
Continual Learning Mechanisms Compose for Long-Horizon Memorization
paper: huggingface.co/papers/2609.06…
paper: huggingface.co/papers/2609.06…
6
15
87
18,985
Tianmin Shu retweeted
28x better memory for LLMs
By composing continual learning mechanisms, a new paper lifts final retention from 1.2% to 34.9% across 100 sequential tasks. The dataset is now on Hugging Face.
By composing continual learning mechanisms, a new paper lifts final retention from 1.2% to 34.9% across 100 sequential tasks. The dataset is now on Hugging Face.
3
22
167
7,136
Our recent work found a simple recipe for long-horizon memorization: it turns out that compositions of continual learning mechanisms can greatly reduce catastrophic forgetting. See the thread by @zheyuanzhang99 below
Existing continual learning methods still suffer from catastrophic forgetting. Our recent work found a simple recipe to substantially improve 𝗹𝗼𝗻𝗴-𝗵𝗼𝗿𝗶𝘇𝗼𝗻 𝗺𝗲𝗺𝗼𝗿𝗶𝘇𝗮𝘁𝗶𝗼𝗻:
Composing multiple continual learning mechanisms extends memory lifetime and raises from 1.2% to 34.9% across 3 datasets: a 28-fold improvement over naive sequential SFT.
Project page: compose-cl.github.io
Paper: arxiv.org/abs/2609.06986
Code + datasets: github.com/cozheyuanzhang…
1
3
13
1,410
Tianmin Shu retweeted
📢 Excited to announce G2A: From Generative Models to Robot Actions workshop @ CoRL 2026!
📝 Submission deadline: October 8, 2026 AoE
📍 Austin, TX | 📅 Nov. 12, 2026
🔗 actiongapworkshop.github.io
How can knowledge learned from large-scale, action-unlabeled video, image, and 3D executable robot policies?
We will explore this central question at the intersection of generative models, robot learning, and embodied AI.
Come and join us to discuss how we can bridge the gap from generative knowledge to executable robot actions!
#CoRL2026 #RobotLearning #Robotics #EmbodiedAI #VLA #WorldModels #GenerativeAI
📝 Submission deadline: October 8, 2026 AoE
📍 Austin, TX | 📅 Nov. 12, 2026
🔗 actiongapworkshop.github.io
How can knowledge learned from large-scale, action-unlabeled video, image, and 3D executable robot policies?
We will explore this central question at the intersection of generative models, robot learning, and embodied AI.
Come and join us to discuss how we can bridge the gap from generative knowledge to executable robot actions!
#CoRL2026 #RobotLearning #Robotics #EmbodiedAI #VLA #WorldModels #GenerativeAI
1
3
11
1,284
Tianmin Shu retweeted
📢 Announcing Towards Test-Time Continual Learning Agents (TTCL) workshop at NeurIPS 2026
#NeurIPS #NeurIPS2026
How can AI agents keep learning after deployment without catastrophic forgetting?
🗓 Submission deadline: Aug 29, 2026 AoE
🔗 ttcl-agents.github.io
🧵1/n
#NeurIPS #NeurIPS2026
How can AI agents keep learning after deployment without catastrophic forgetting?
🗓 Submission deadline: Aug 29, 2026 AoE
🔗 ttcl-agents.github.io
🧵1/n
2
7
17
5,441
Tianmin Shu retweeted
Happy to share that ThoughtTrace has received the Best Paper Award from #ICML2026 RLxF workshop 🙏
We believe users' unspoken thoughts are a new data modality toward advanced social intelligence.
The oral presentation will take place 9:10-10:00 AM KST on July 10th at Grand Ballroom 101-102. Please check it out if you are around!
We believe users' unspoken thoughts are a new data modality toward advanced social intelligence.
The oral presentation will take place 9:10-10:00 AM KST on July 10th at Grand Ballroom 101-102. Please check it out if you are around!
What are users thinking during their interactions with LLMs?
We introduce ThoughtTrace — the first large-scale dataset that captures what users think during real-world human–AI conversations, not just what they type.
→ 10,174 thought annotations
→ 2,155 multi-turn urns
→ 1,058 users
→ 20 LLMs
These thoughts improve user behavior prediction (+41.7%) and model alignment (+25.6%).
This opens a new paradigm of user-centric LLM research. Full information in the thread 🧶
Read our paper: arxiv.org/abs/2605.20087
Check our project website: thoughttrace-project.github.io
1
4
27
4,406
I won't be at #ICML2026 this year, but my students/collaborators will present MindZero and ThoughtTrace. See 🧵 below for details:
1
2
22
2,087
MindZero: Learning Online Mental Reasoning With Zero Annotations
Morning poster session on the first day of the main conference: Tue, Jul 7, 2026, 10:30 AM – 12:15 PM KST, at Hall A #412
icml.cc/virtual/2026/p…
x.com/ShunchiZhang/s…
Morning poster session on the first day of the main conference: Tue, Jul 7, 2026, 10:30 AM – 12:15 PM KST, at Hall A #412
icml.cc/virtual/2026/p…
x.com/ShunchiZhang/s…
How to teach AI assistants to infer human minds and proactively provide online assistance?
Sharing our ICML 2026 paper 🧠 𝙈𝙞𝙣𝙙𝙕𝙚𝙧𝙤: Learning Online Mental Reasoning with Zero Annotations!
🤗 Code and weights are fully open source
🌐 scai.cs.jhu.edu/MindZero
1
1
9
990
ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions (Best paper award at the RLxF workshop!)
Oral session at the RLxF workshop: Fri, Jul 10, 9:10–10:00 AM KST, at Grand Ballroom 101-102
x.com/chuanyang_jin/…
Oral session at the RLxF workshop: Fri, Jul 10, 9:10–10:00 AM KST, at Grand Ballroom 101-102
x.com/chuanyang_jin/…
What are users thinking during their interactions with LLMs?
We introduce ThoughtTrace — the first large-scale dataset that captures what users think during real-world human–AI conversations, not just what they type.
→ 10,174 thought annotations
→ 2,155 multi-turn urns
→ 1,058 users
→ 20 LLMs
These thoughts improve user behavior prediction (+41.7%) and model alignment (+25.6%).
This opens a new paradigm of user-centric LLM research. Full information in the thread 🧶
Read our paper: arxiv.org/abs/2605.20087
Check our project website: thoughttrace-project.github.io
0
1
4
317
One of the challenges for training Theory of Mind models is the lack of ground-truth mental state annotations. How can we address this? It turns out that you don't need any annotations! Introducing MindZero, a self-supervised RL approach for training LLMs to internalize Bayesian inverse planning with ZERO annotations, accepted to #ICML2026.
Key idea: Bayesian inverse planning provides self-supervised reward for training an LLM; after training, LLM generates amortized inferences end-to-end. This is a significant step toward scaling Bayesian inverse planning style Theory of Mind reasoning. Check out @ShunchiZhang's thread for details 👇
Key idea: Bayesian inverse planning provides self-supervised reward for training an LLM; after training, LLM generates amortized inferences end-to-end. This is a significant step toward scaling Bayesian inverse planning style Theory of Mind reasoning. Check out @ShunchiZhang's thread for details 👇
How to teach AI assistants to infer human minds and proactively provide online assistance?
Sharing our ICML 2026 paper 🧠 𝙈𝙞𝙣𝙙𝙕𝙚𝙧𝙤: Learning Online Mental Reasoning with Zero Annotations!
🤗 Code and weights are fully open source
🌐 scai.cs.jhu.edu/MindZero
0
6
31
4,557
Tianmin Shu retweeted
ThoughtTrace just hit 5K+ downloads within two weeks of release!
TT is a simple, ready-to-use dataset with surprisingly diverse applications for understanding users and training AI models. Check it out!
TT is a simple, ready-to-use dataset with surprisingly diverse applications for understanding users and training AI models. Check it out!
What are users thinking during their interactions with LLMs?
We introduce ThoughtTrace — the first large-scale dataset that captures what users think during real-world human–AI conversations, not just what they type.
→ 10,174 thought annotations
→ 2,155 multi-turn urns
→ 1,058 users
→ 20 LLMs
These thoughts improve user behavior prediction (+41.7%) and model alignment (+25.6%).
This opens a new paradigm of user-centric LLM research. Full information in the thread 🧶
Read our paper: arxiv.org/abs/2605.20087
Check our project website: thoughttrace-project.github.io
0
3
14
1,979
Tianmin Shu retweeted
Nice work! World models meet active sensing and closed-loop planning.
We'll be showcasing this demo at the Lambda booth @chuanli11 during CVPR main conference at Denver. If you're attending, stop by and check it out! #cvpr @CVPR
We are also hosting a world-model-related workshop at CVPR : cvpr26wmas.github.io/cvpr26wmas.html
We'll be showcasing this demo at the Lambda booth @chuanli11 during CVPR main conference at Denver. If you're attending, stop by and check it out! #cvpr @CVPR
We are also hosting a world-model-related workshop at CVPR : cvpr26wmas.github.io/cvpr26wmas.html
What should a world model capture for embodied agents?
An agent acting under partial observability needs a belief over the 3D world: what it has seen, what may exist beyond view, and how that belief should update as it moves.
Introducing 3D-Belief, a 3D world model for embodied partial observability. 🧵[1/9]
0
2
11
1,737
To plan effectively in an unseen 3D world, embodied agents need to form and update beliefs of the 3D world. In our recent work, @yifanyin_11 proposed 3D-Belief, a generative 3D world model that continuously updates scene memory AND 3D imagination of unseen space as an agent explores a new environment. We show great results of improved imagination quality 🖼️, spatial reasoning 🌎, and closed-loop planning 🤖 with 3D-Belief. This work presents a new way to think about what kinds of world modeling are necessary for embodied reasoning and planning under partial observability, particularly in unseen environments.
See the thread below for more details 👇
See the thread below for more details 👇
What should a world model capture for embodied agents?
An agent acting under partial observability needs a belief over the 3D world: what it has seen, what may exist beyond view, and how that belief should update as it moves.
Introducing 3D-Belief, a 3D world model for embodied partial observability. 🧵[1/9]
1
2
19
2,730