Tzu-Mao Li retweeted
We are looking forward to seeing you at the Light Transport session at #SIGGRAPH 2026 on Tuesday!
Check out our work with my mentor @marcosalvi, @tzumaoli , @BartWronsk @winmad4869 our labmates @sneezygiraffe, Cheng and Ravi! : A generalizable light transport 3D embedding for ere we learn light transport in a generalizable, scalable manner. lnkd.in/gsBpmCmy ⚽
#siggraph2026
Check out our work with my mentor @marcosalvi, @tzumaoli , @BartWronsk @winmad4869 our labmates @sneezygiraffe, Cheng and Ravi! : A generalizable light transport 3D embedding for ere we learn light transport in a generalizable, scalable manner. lnkd.in/gsBpmCmy ⚽
#siggraph2026
0
3
12
1,923
Tzu-Mao Li retweeted
I am happy to introduce our Siggraph 2026 paper: YASPS, a Symbolic Framework for Extensible, High-performance IPC Simulation: github.com/txstc55/yasps
YASPS is designed to answer the question: how can we write this simulation (more in threads)
#siggraph2026
YASPS is designed to answer the question: how can we write this simulation (more in threads)
#siggraph2026
1
9
78
9,226
Tzu-Mao Li retweeted
🌲Introducing our paper A generalizable light transport 3D embedding for Global Illumination arxiv.org/pdf/2510.18189.
Just as Transformers learn long-range relationships between words or pixels, our new paper shows they can also learn how light interacts and bounces around a 3D analogy of light transport operator and Attention, our fully scalable model learns a generalizable light transport 3D embedding that captures global illumination.
It takes as input the 3D scene assets—geometry, materials, and lighting represented as a point cloud—and encodes the complex light interaction into a 3D latent space, with codes anchored at the sampled scene points.
Remarkably, the model is independent of both viewpoint and resolution, enabling view-consistent rendering across diverse scenes. (See teaser and attention figure.)
(Our team work: with my mentor Marco @marcosalvi, labmates Mukund @mukundvermar, Cheng, NV folks Bart @BartWronsk, Lifan @winmad4869, and Tzu-Mao @tzumaoli & Ravi.)
🧵
Just as Transformers learn long-range relationships between words or pixels, our new paper shows they can also learn how light interacts and bounces around a 3D analogy of light transport operator and Attention, our fully scalable model learns a generalizable light transport 3D embedding that captures global illumination.
It takes as input the 3D scene assets—geometry, materials, and lighting represented as a point cloud—and encodes the complex light interaction into a 3D latent space, with codes anchored at the sampled scene points.
Remarkably, the model is independent of both viewpoint and resolution, enabling view-consistent rendering across diverse scenes. (See teaser and attention figure.)
(Our team work: with my mentor Marco @marcosalvi, labmates Mukund @mukundvermar, Cheng, NV folks Bart @BartWronsk, Lifan @winmad4869, and Tzu-Mao @tzumaoli & Ravi.)
🧵
4
33
128
20,117
By the way, our recent paper tackled exactly this problem using ratio control variates: suikasibyl.github.io/vvmc/
Random related question: have people tried to address the color variance in EnvMap sampling? Sampling the (weighted) average over wavelengths will not give you zero variance even if we ignore BRDFs.
0
0
12
1,068
Tzu-Mao Li retweeted
In our #SIGGRAPH2025 work, we enable artist-level editing of hair reconstructed from video for the first time! We convert 3D strands into a procedural model controllable via guide strands and intuitive operators like curl and bend. 1/4
weschang.com/publications/i… (Code available!)
weschang.com/publications/i… (Code available!)
1
2
22
1,310
Tzu-Mao Li retweeted
Dear MAGA friends,
I have been worrying about STEM in the US a lot, because right now the Senate is writing new laws that cut 75% of the STEM budget in the US.
Sorry for the long post, but the issue is really important, and I want to share what I know about it.
The entire funding for the NSF and NIH together is only 0.82% of the federal budget, but it is hugely important for science, funding the entire science research and education pipeline in the US.
The Senate is now planning to cut more than half of that science funding permanently. For decades the USA has already underfunded science compared to how important it is for the future, but now with these cuts, the future of science in this country will be devastated.
Let me break down what's happening:
(1) First, how we got into this mess.
(2) Second, why it is such a disaster.
(1) Here is why we are making the mistake.
This year, more than half of all projects in the NSF and NIH have been cut because they mention DEI, but we have NOT redirected the money to non-DEI STEM education. Instead it's just cut permanently.
To understand why there is so much DEI to be cut, you need to look at how NSF has operated. For decades, NSF has required grant proposals to address "broader impacts" - showing how the research would benefit society beyond just scientific discovery. NSF specifically asked for activities focused on "full participation of women, persons with disabilities, and underrepresented minorities in STEM." This led professors to include plans for broadening participation, literally writing programs like this: "we will design educational activities for a robot club for girls."
Now remember, the professors were asked to propose these programs by the government. We are all trained in physics or AI or robots or biology - but the law says, you must make sure your programs encourage women to do STEM, so that is what we all did. So basically 100% of science educators followed this law. But now that DEI is no longer allowed, it has all been cut - DOGE scrubbed through all the proposals for any of these ideas and ended up cancelling 75% of STEM education programs.
But here is the big mistake we are making. Instead of redirecting the money to tell the professors to boost STEM for "ALL students and ALL people," we are CUTTING the funding PERMANENTLY.
The same scientists would enthusiastically make programs for ALL American students - boys and girls, rural and urban, from every background. We just need Congress to redirect the funding instead of eliminating it. But instead, the senators are planning to just cut the whole budget for everything that included that girls club - all the research, all the education, all the experts, everything - basically forcing our teaching scientists to leave for other careers.
For example 85% of all fundamental physics research has been cut. You cannot cut 85% of a budget without losing nearly everybody. And so the departures have already started in the NSF. They have already lost hundreds of expert STEM staff.
So that is how we got to making this mistake.
(2) The second big thing to understand is how BAD this mistake is.
By making the cuts permanent and losing everybody, instead of just changing the priorities for the professors, we end up permanently shrinking not just the NSF and NIH but all science in the US.
Every single scientist in this country learned from another scientist; they all went to school to learn. But by specifically losing professors, we are shutting down the education pipeline, which is the future of the field.
So even though there is still a lot of science funding in the budget for, example, engineers making Defense weapons systems - every single one of those engineers had to have decades of training. And so we need to have a scientific training and teaching pipeline, which is done by the NSF and NIH, but this is exactly what is getting zeroed out.
Every time I talk to a talented PhD student about career choices, I work hard to try to convince them to stay in low-paying science teaching and research instead of getting rich on Wall Street. Because teaching is about the future of ideas, the future of talent in the country. But that is a really hard conversation, because even though teaching and research is idealistic, Wall Street is literally dangling millions of dollars in front of the best PhD students. Being a professor doesn't pay by comparison.
Cutting the NSF and NIH will force our teaching scientists to leave for other careers, and those scientists will quickly move on to some other technical career that doesn't involve teaching. They will go to Wall Street, or they will go make and sell a product. We might think: why is that so bad? Maybe they can go do something more useful with their talent, something that makes more money.
The reason it is so bad is because in this country we really have a shortage of teachers for the next generation. Not only for the K-12 kids, but also top professors for teaching the top experts how to advance the frontiers of our scientific fields. Science and technology is advancing so quickly, for there to be opportunities in the US, we really need our best experts at the frontier in training positions, to help teach more scientists.
Think about it this way: every single American scientist - whether they work for SpaceX, design weapons for defense contractors, develop new drugs, or create AI systems - they ALL learned from professors funded by NSF and NIH. Without these teaching scientists, we won't have ANY scientists in 10 years. Not for defense, not for industry, not for medicine. We're not just cutting some programs - we're cutting off the entire pipeline that creates every technical expert in America.
The real thing that is keeping me from sleeping, is that this "little" budget decision will actually be the end of science teaching in this country. That half-a-percent investment is really about cutting the whole future for all future scientists in the US.
I ran a search for programs that have already been cut, and in Kansas and Nebraska, we've cut Grant #2409150 - a STEM Pathways Alliance providing scholarships, research opportunities, and tutoring. In Alaska, we've cut Grant #2308786 - a program providing research opportunities and research travel across the university system. In Kansas, we've also terminated Grant #2314275 - a program teaching tech skills from basic computer literacy to coding and cybersecurity. In West Virginia, we've cut Grant #2411642 - a program to strengthen STEM departments at WVU. In Nebraska, we've terminated Grant #2415667 - a program connecting youth to agricultural technology and environmental science.
Why cut these programs? They are being terminated simply because they mention serving women, minorities, indigenous students, or ex-cons. But the professors running them are scientists and engineers, not activists. They'd be thrilled to teach ANY student who wants to learn. Just redirect the scholarships to ALL deserving students. Open the tech training to ALL unemployed workers. Expand the rural programs to serve ALL rural kids. The infrastructure is already there - the labs, the mentors, the industry partnerships. All Congress needs to do is change "underrepresented minorities" to "ALL Americans." If anything, we need MORE funding for these programs, not less - there are plenty of Americans across the country who need these STEM programs.
If you or anybody you know lives in one of these states, you have a lot of influence. Here are some of the key senators debating the issue: @JerryMoran (KS) @SenCapito (WV) @lisamurkowski (AK) @SenKatieBritt (AL) @SenatorFischer (NE).
All these Senators have all supported STEM in the past, and we need to make sure that they know how important we all think it is to preserve (and redirect) that half a percent of the budget for STEM research and education instead of zeroing it out.
It's one thing to debate fairness in STEM. But it is a huge mistake to just cut it all.
Please ask these senators to:
(1) Keep the STEM funding but redirect it to serve ALL Americans
(2) Protect American jobs and innovation
Even a simple message saying "Don't cut STEM - redirect it to serve everyone" would help. These specific senators are making these decisions NOW, this week, this month.
I have been worrying about STEM in the US a lot, because right now the Senate is writing new laws that cut 75% of the STEM budget in the US.
Sorry for the long post, but the issue is really important, and I want to share what I know about it.
The entire funding for the NSF and NIH together is only 0.82% of the federal budget, but it is hugely important for science, funding the entire science research and education pipeline in the US.
The Senate is now planning to cut more than half of that science funding permanently. For decades the USA has already underfunded science compared to how important it is for the future, but now with these cuts, the future of science in this country will be devastated.
Let me break down what's happening:
(1) First, how we got into this mess.
(2) Second, why it is such a disaster.
(1) Here is why we are making the mistake.
This year, more than half of all projects in the NSF and NIH have been cut because they mention DEI, but we have NOT redirected the money to non-DEI STEM education. Instead it's just cut permanently.
To understand why there is so much DEI to be cut, you need to look at how NSF has operated. For decades, NSF has required grant proposals to address "broader impacts" - showing how the research would benefit society beyond just scientific discovery. NSF specifically asked for activities focused on "full participation of women, persons with disabilities, and underrepresented minorities in STEM." This led professors to include plans for broadening participation, literally writing programs like this: "we will design educational activities for a robot club for girls."
Now remember, the professors were asked to propose these programs by the government. We are all trained in physics or AI or robots or biology - but the law says, you must make sure your programs encourage women to do STEM, so that is what we all did. So basically 100% of science educators followed this law. But now that DEI is no longer allowed, it has all been cut - DOGE scrubbed through all the proposals for any of these ideas and ended up cancelling 75% of STEM education programs.
But here is the big mistake we are making. Instead of redirecting the money to tell the professors to boost STEM for "ALL students and ALL people," we are CUTTING the funding PERMANENTLY.
The same scientists would enthusiastically make programs for ALL American students - boys and girls, rural and urban, from every background. We just need Congress to redirect the funding instead of eliminating it. But instead, the senators are planning to just cut the whole budget for everything that included that girls club - all the research, all the education, all the experts, everything - basically forcing our teaching scientists to leave for other careers.
For example 85% of all fundamental physics research has been cut. You cannot cut 85% of a budget without losing nearly everybody. And so the departures have already started in the NSF. They have already lost hundreds of expert STEM staff.
So that is how we got to making this mistake.
(2) The second big thing to understand is how BAD this mistake is.
By making the cuts permanent and losing everybody, instead of just changing the priorities for the professors, we end up permanently shrinking not just the NSF and NIH but all science in the US.
Every single scientist in this country learned from another scientist; they all went to school to learn. But by specifically losing professors, we are shutting down the education pipeline, which is the future of the field.
So even though there is still a lot of science funding in the budget for, example, engineers making Defense weapons systems - every single one of those engineers had to have decades of training. And so we need to have a scientific training and teaching pipeline, which is done by the NSF and NIH, but this is exactly what is getting zeroed out.
Every time I talk to a talented PhD student about career choices, I work hard to try to convince them to stay in low-paying science teaching and research instead of getting rich on Wall Street. Because teaching is about the future of ideas, the future of talent in the country. But that is a really hard conversation, because even though teaching and research is idealistic, Wall Street is literally dangling millions of dollars in front of the best PhD students. Being a professor doesn't pay by comparison.
Cutting the NSF and NIH will force our teaching scientists to leave for other careers, and those scientists will quickly move on to some other technical career that doesn't involve teaching. They will go to Wall Street, or they will go make and sell a product. We might think: why is that so bad? Maybe they can go do something more useful with their talent, something that makes more money.
The reason it is so bad is because in this country we really have a shortage of teachers for the next generation. Not only for the K-12 kids, but also top professors for teaching the top experts how to advance the frontiers of our scientific fields. Science and technology is advancing so quickly, for there to be opportunities in the US, we really need our best experts at the frontier in training positions, to help teach more scientists.
Think about it this way: every single American scientist - whether they work for SpaceX, design weapons for defense contractors, develop new drugs, or create AI systems - they ALL learned from professors funded by NSF and NIH. Without these teaching scientists, we won't have ANY scientists in 10 years. Not for defense, not for industry, not for medicine. We're not just cutting some programs - we're cutting off the entire pipeline that creates every technical expert in America.
The real thing that is keeping me from sleeping, is that this "little" budget decision will actually be the end of science teaching in this country. That half-a-percent investment is really about cutting the whole future for all future scientists in the US.
I ran a search for programs that have already been cut, and in Kansas and Nebraska, we've cut Grant #2409150 - a STEM Pathways Alliance providing scholarships, research opportunities, and tutoring. In Alaska, we've cut Grant #2308786 - a program providing research opportunities and research travel across the university system. In Kansas, we've also terminated Grant #2314275 - a program teaching tech skills from basic computer literacy to coding and cybersecurity. In West Virginia, we've cut Grant #2411642 - a program to strengthen STEM departments at WVU. In Nebraska, we've terminated Grant #2415667 - a program connecting youth to agricultural technology and environmental science.
Why cut these programs? They are being terminated simply because they mention serving women, minorities, indigenous students, or ex-cons. But the professors running them are scientists and engineers, not activists. They'd be thrilled to teach ANY student who wants to learn. Just redirect the scholarships to ALL deserving students. Open the tech training to ALL unemployed workers. Expand the rural programs to serve ALL rural kids. The infrastructure is already there - the labs, the mentors, the industry partnerships. All Congress needs to do is change "underrepresented minorities" to "ALL Americans." If anything, we need MORE funding for these programs, not less - there are plenty of Americans across the country who need these STEM programs.
If you or anybody you know lives in one of these states, you have a lot of influence. Here are some of the key senators debating the issue: @JerryMoran (KS) @SenCapito (WV) @lisamurkowski (AK) @SenKatieBritt (AL) @SenatorFischer (NE).
All these Senators have all supported STEM in the past, and we need to make sure that they know how important we all think it is to preserve (and redirect) that half a percent of the budget for STEM research and education instead of zeroing it out.
It's one thing to debate fairness in STEM. But it is a huge mistake to just cut it all.
Please ask these senators to:
(1) Keep the STEM funding but redirect it to serve ALL Americans
(2) Protect American jobs and innovation
Even a simple message saying "Don't cut STEM - redirect it to serve everyone" would help. These specific senators are making these decisions NOW, this week, this month.
22
68
463
124,168
Tzu-Mao Li retweeted
I am looking for graduate students for my new lab at the University of Cambridge! Join me to understand and build models of visual perception.
9
87
629
88,241
Tzu-Mao Li retweeted
Our research group at Eyeline Studios - Powered by Netflix is hiring research interns for summer 2025.
We create transformative tools for visual storytelling, read more about the positions and apply here: scanlinevfx.com/careers/?job=2…
(1/2)
We create transformative tools for visual storytelling, read more about the positions and apply here: scanlinevfx.com/careers/?job=2…
(1/2)
2
3
17
2,240
Tzu-Mao Li retweeted
It's getting to that point where our research team is thinking about next year's interns. If interested, feel free to e-mail me, or (even better) find someone on our team whose research aligns with yours and e-mail them.
research.nvidia.com/labs/rtr/
research.nvidia.com/labs/rtr/
2
27
86
9,148
Tzu-Mao Li retweeted
I'll be giving a talk today at 4:20 in San Gabriel that will include introductions to differentiable rendering and distribution theory.
I'll be presenting a poster at OOPSLA starting very soon on Distributions for Compositionally Differentiating Parametric Discontinuities.
If you're at OOPSLA and interested in differentiable programming applied to physics, probabilistic programming, or rendering, stop by!
0
1
4
1,103
Tzu-Mao Li retweeted
The NVIDIA HiFi Physics team is looking for interns with interest/skills in
1⃣ Physics simulation (solids/fluids/differentiable etc)
2⃣ Digital Humans
3⃣ Neural Rendering/Geometry, Video Models
4⃣ Monte Carlo Methods
5⃣ Digital Fabrication
Apply: nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx…
1⃣ Physics simulation (solids/fluids/differentiable etc)
2⃣ Digital Humans
3⃣ Neural Rendering/Geometry, Video Models
4⃣ Monte Carlo Methods
5⃣ Digital Fabrication
Apply: nvidia.wd5.myworkdayjobs.com/en-US/NVIDIAEx…
1
23
128
14,272
weschang.com/publications/s…
Check out Wesley, Xuanda, and Yash's latest work on combining cross bilateral filtering and Adam to make optimization in graphics more robust and fast, through better preconditioning the gradient.
Check out Wesley, Xuanda, and Yash's latest work on combining cross bilateral filtering and Adam to make optimization in graphics more robust and fast, through better preconditioning the gradient.
0
15
88
7,840
fzy28.github.io/BSDF_diffusion…
Rendering nerds! Check out Ziyang Fu's latest work, where he shows that low-dimensional, deterministic diffusion models can be useful for importance sampling in Monte Carlo rendering applications.
Rendering nerds! Check out Ziyang Fu's latest work, where he shows that low-dimensional, deterministic diffusion models can be useful for importance sampling in Monte Carlo rendering applications.
0
12
99
6,371
Tzu-Mao Li retweeted
What a pleasant big surprise. Free to share a $1M secret to get a best paper award: to get Iliyan presenting the paper. @EGSympRendering @iliyang. Congrats to @tzumaoli, Trevor and Ravi.
7
14
92
10,340
Tzu-Mao Li retweeted
📢Thrilled to announce that the #HPG2024 student competition will have two incredible prizes up for grabs! 🎁A Meta Quest 3 and 🎁an NVIDIA GeForce 4080, first-place winner gets their choice! Don’t miss out – check the rules and submit your entries now! highperformancegraphics.org/2024/student-c…
0
3
12
2,833
iliyan.com/publications/R…
Rendering nerds! Check out @_bingxu's latest EGSR paper about importance sampling the difference between two frames in path space. The idea is to spawn paths that always go through the edited part of the scene and reduce variance.
Rendering nerds! Check out @_bingxu's latest EGSR paper about importance sampling the difference between two frames in path space. The idea is to spawn paths that always go through the edited part of the scene and reduce variance.
1
19
92
6,338
ri.cmu.edu/pub_files/pub4…
"Inverse Volume Rendering Approach to 3D
Reconstruction from Multiple Images", Yamazaki et al. (with Takeo Kanade as last author!) from 2006 proposed the idea to solve 3D reconstruction using inverse volume rendering (aka NeRF).
"Inverse Volume Rendering Approach to 3D
Reconstruction from Multiple Images", Yamazaki et al. (with Takeo Kanade as last author!) from 2006 proposed the idea to solve 3D reconstruction using inverse volume rendering (aka NeRF).
2
12
81
6,139
It drew connections to image matting and used EM to solve for the occupancy and colors, which is different from (but related to) modern gradient-based methods. There are also tricks to focus on the object boundaries (aka adaptive shells). This paper is cited 10 times currently.
0
0
10
709
Tzu-Mao Li retweeted
A Tour of Differentiable Rasterization (srush.github.io/DiffRast/). Tutorial about writing an SVG-lite in Jax.
This is only loosely about graphics, more about how I stil want a "Differentiable Everything" future 😃.
This is only loosely about graphics, more about how I stil want a "Differentiable Everything" future 😃.
8
47
381
59,896
Tzu-Mao Li retweeted
We finally made V1.0.0 of our book on physics-based simulation freely available at phys-sim-book.github.io with github.com/phys-sim-book/…!
This marks our commitment to make the theory and algorithms accessible to everyone.
You are welcome to contribute at github.com/phys-sim-book/…!
This marks our commitment to make the theory and algorithms accessible to everyone.
You are welcome to contribute at github.com/phys-sim-book/…!
5
123
446
63,683