Introducing Mundo:

A customized world model that turns photos of a given place into a world you can explore, edit, and interact with.

We're excited to introduce AW-0: Agentic World Action Models! 🤖 (0/8)

The first World Action Model that plans in code, predicts the future, and controls the robot—all in one generative model.

AW-0 jointly generates:

🧠 Code - explicit skill calls, object arguments, and video - predictive visual dynamics

🦾 Robot actions - continuous motor control

All three are generated together through joint denoising, letting semantic plans directly shape both predicted futures and robot behavior, with:

🤝 Hybrid execution — generated programs can also run directly through perception and control APIs

⚡ Just 40 ms of additional inference latency over the base WAM on an RTX 5090

📚 Pretrained on ~3,000 hours of robot + human demonstrations

One model for planning, prediction, and execution.

🌐 Project: agentic-wam.github.io

📄 Paper: arxiv.org/abs/2610.30136…

💻 Code: github.com/ZhuoyangLiu200…

🤗 Model: huggingface.co/miniFranka/age…

🧵 How it works—and what we found ↓

I'm at Budapest from 10/24-10/30 for #EMNLP2026 , and will present our recent work studying designs of visual encoders in adapting them to multi-image, interactive settings.

We believe it's an important problem as vision in current days is often contextualized in interactions, encodings should also be conditioned on interactions too.

Check out our poster ↓, and I'm happy to chat in-person around multimodal applications, physical ai and interactive decision-making.

FrontierCS 2 is our new benchmark for CS research & development.

Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.

New #NVIDIA paper: what if robots never had to wait for their own thoughts?

TacitVLA learns to reason in chain-of-thought or subtasks, but its actions never read them.

→ Keeps the gains of reasoning at up to 3.4× lower latency on 4 benchmarks, and tops RoboMME and RMBench 🧵👇

Generation 𝗖𝗔𝗡 improve understanding!

With the right recipe, we show that training on generation data can help understanding tasks.

Introducing OmniTaskonomy: our year-long study of when and how generation helps understanding.

Cannot agree more on this (on empirical AI papers)! Taking the convention the first author is the submitter, leading >2 papers a month or >24 papers a year makes me think of AI slops much more than credible work.

Let me try a new style of posting

Looks like we have over 50000 submissions to ICLR this year.

2016 -> 2026; 10 years but 100x 🥲

Open-source video generation is now faster than playback without compromising quality.

Introducing Video Delta Net (VDN): hybrid attention for live text-to-video with near-lossless quality.

VDN accelerates Minimax-H3 by 75 - 90 x, generating 14 seconds of 768p video in 11 00 GPUs.

Checkpoints + training/inference code + Technical Blog ⬇️
(1/6)

Ready to tackle the systems challenges of Agentic Reinforcement Learning? @YichuanM and @shuhuayu_ will share how to train agents that can runs hundreds of turns along with the infrastructure and training recipe to make it scalable and reproducible at PyTorch Conference North is deep dive into building an end-to-end agentic RL training loop. Register now: hubs.la/Q04v4SL60

Excited to share I’m joining @Stanford CS as an Assistant Professor! My lab will focus on AI-driven optimization of data systems. I’m recruiting PhD students for Fall 2027 (list me in your app if interested).
Before starting, I’ll spend the year @OpenAI as a visiting researcher.

For the other workshop, SVE has also been selected for an oral presentation 🥳

This is for the Visual Perception and Reasoning in the Interactable World workshop ;)

Happy to share two papers have been straightforwardly accepted into #EMNLP2026!

SVE: Stateful Visual Encoders for VLMs
statefulvisualencoders.github.io

AgentBait: Content Rewriting for Agentic Choice
agentbait.github.io

Check them out and see you in Budapest!

Happy to share that SVE has been selected for 𝗮𝗻 𝗼𝗿𝗮𝗹 𝗽𝗿𝗲𝘀𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 at the Medical Foundation Models and Benchmarks workshop in ECCV 2026! See you in Malmö :)

Zero Train–Inference Mismatch — now for linear attention, and under async RL 🎯

We got bitwise-exact trainer/generator parity for Gated DeltaNet (Qwen3.5-9B / 35B-A3B) on TorchTitan RL + vLLM, then asked the question nobody had actually tested in open source: does it help async as we know):
1️⃣ First open-source stack to hit train/inference logprob diff = 0 on a
linear-attention model
2️⃣ First to measure what zero mismatch is worth under async off-policy RL

How — smaller than it sounds:
• trainer and generator share one model definition (TorchTitan unified model). No more aligning trainer & generator op by op
• recurrent kernel everywhere on the forward (prefill + decode + trainer), chunked only for backward
• split-K off, batch-invariant GEMM, + a batch-invariant bmm for the MoE router
• we never touch FLA kernel internals — the recurrent kernel is intrinsically batch-invariant: one sequence's own state, fixed order, no cross-sequence reductions
• vLLM's linear-cache management + prefix caching keep working as-is

📊 Qwen3.5-9B + DAPO-Math, async at offpolicy = 4 / 12 / 32:
• logprob_diff = exactly 0 at step 0 ✅ every nonzero after that is staleness, not precision
• at offpolicy=32 the stock stack runs away past 0.065; ours holds ~0.035
• the unified model is free — the entire bill is the kernels
• also verified on MoE 35B-A3B and a 64-turn / 64K-context terminal agent

🤔 Now the honest part: it depends on the config. On math at offpolicy=12, BI leads on train reward and peaks highest on held-out AIME. On the terminal agent it's slightly ahead. On search, and on other math configs, the curves are indistinguishable. What is consistent is the numerics — BI keeps the gap from exploding as you widen the window. Cost: 2–3× trainer throughput (≈5× on the agent).

Our take: bitwise parity is a debugging tool, not a production default. Flip BI on for 20 on-policy steps — if the gap is 0 and your run is still broken, the infra is exonerated and the bug is in your data or your algorithm. That's worth 20 steps, not a whole run.

Next:
• make BI cheap (kill the 2–3×)
• more workloads, more seeds
• stay tuned for more complex agentic training recipes/tests 👀
Full writeup 🔗 yichuan-w.github.io/blog/GDN-train…

We collaborated closely with the TorchTitan team, and we sincerely encourage everyone to check out TorchTitan RL. With its clean, efficient unified model architecture and native async RL design, we believe it is a strong foundation for your next RL engine.

#LLM #RL #agentic #pytorch #torchtitan

Since when did almost every ML paper submission receive reviews that ask for statistical significance and paired tests 🤣

Today, we’re introducing [schema]: a harness reaching 99% RHAE with Opus 4.8 + Fable 5 and 95.35% with GPT-5.6 Sol on ARC-AGI-3 Public set.

[schema] makes an LLM think like a physicist. 🧵

G.O.A.T

I am starting a blog about deep learning theory and its value to practitioners! First post is about Adam, broken convergence proofs, and what theory can contribute when stuff just works anyways without it. Subscribe on Substack if you like it! undertheassumptions.substack.com/p/the-optimize…