Pinned Tweet
FlashOptim is our latest research on low precision optimizers. We came up with the right numeric tricks to halve model training memory without any quality degradation, and we are releasing the library and kernels that make all of this magic possible!
🚀 Today we’re releasing FlashOptim: better implementations of Adam, SGD, etc, that compute the same updates but save tons of memory. You can use it right now via `pip install flashoptim`. 🚀
arxiv.org/abs/2602.23349
A bunch of cool ideas make this possible: [1/n]
1
11
71
6,233
I've been doing this for months, and I can confirm it's great. I go one step further and hook the cleanup/clarification into the harness (github.com/JJGO/pi-rephra…), so that it can use the full session context to resolve ambiguities and provide better test instructions.
One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're trying to achieve, but you're too lazy to type them. In these cases I like to lean back, switch to /voice and just ramble for like 10 minutes, total mess, anything goes, full stream of consciousness. Sometimes I declare it up top, something like "switching to speech recognition sorry for any typos...". Sometimes I turn it into a small interview of a few turns. But I find that the LLMs are somehow very good at reconstructing long incoherent rambles and often their echo of your own tangle of thoughts comes out quite a bit cleaner than what you started with. The result is that you improve the mind meld and have to correct things less from that point on.
0
0
6
1,181
This completely matches my first hand experience, Pi is faster, more accurate, and cheaper than codex or claude code for the same task. The customization is the cherry on top. Thanks @badlogicgames for making a harness that is 3.14x better than the other ones
We benchmarked coding agents on our own internal tasks at Databricks and learned a lot!
There are many surprising opportunities to lower cost and increase quality, and many models including open source ones are truly competitive now. 🧵
2
1
13
2,826
Jose Javier Gonzalez retweeted
At 11k employees, our AI costs are going up. Which model & harness should we use to lower cost but also retain great quality?
We didn't want to blindly trust public benchmarks. So we ran a comprehensive evaluation on our tasks, code base, infra. It's been produced by more than 3,000 software engineers, spans 3 hyperscalar clouds and many languages and tasks.
The results are surprising. We find that for the SAME mdoel, the choice of harness can significantly save costs (~2x). We also find that GLM 5.2 performs extremely well. We run Omnigent in front of these and can easily multiplex different harnesses and models for different tasks.
Check it out:
databricks.com/blog/benchmark…
We didn't want to blindly trust public benchmarks. So we ran a comprehensive evaluation on our tasks, code base, infra. It's been produced by more than 3,000 software engineers, spans 3 hyperscalar clouds and many languages and tasks.
The results are surprising. We find that for the SAME mdoel, the choice of harness can significantly save costs (~2x). We also find that GLM 5.2 performs extremely well. We run Omnigent in front of these and can easily multiplex different harnesses and models for different tasks.
Check it out:
databricks.com/blog/benchmark…
106
208
1,580
430,942
I'm at #ICML2026 this week, and today I'll be presenting FlashOptim, our latest work on memory efficient optimizers! We are in Poster Session 8 at 5pm, so come to HALL A #410 to get a free explainer, grill me with questions, or to hear the war stories of doing optimizer numerics!
1
8
15
2,565
Our OSS code is at github.com/databricks/fla… and the latest version of the paper is live at openreview.net/forum?id=Wfe1i…
1
1
4
130
And here's a thread 🧵 explaining the core ideas: x.com/davisblalock/s…
🚀 Today we’re releasing FlashOptim: better implementations of Adam, SGD, etc, that compute the same updates but save tons of memory. You can use it right now via `pip install flashoptim`. 🚀
arxiv.org/abs/2602.23349
A bunch of cool ideas make this possible: [1/n]
0
1
4
141
Probably the best youtube channel in my subscriptions. Go watch Laurie talk about cool and interesting computer topics
Just hit 500,000 Subscribers!
I never could have imagined (what really is my weekend hobby!) growing this much.
Getting really close to overflowing a signed 20 bit word!
There’s still so, so much to explore in the field of Computer Science. My biggest advice is go make the inspired a younger you!
No, it’s not my dayjob. As much as I get a kick out of people thinking that I’m some giant private equity production (lol), it’s just me, my small bedroom, and some cameras.
Thanks everyone for the support, I’m excited to keep sharing this field with you all.
Until next time,
LaurieWired out.
0
0
3
187
Jose Javier Gonzalez retweeted
I implemented this in JAX: `uv pip install flashoptim-jax`!
Read below for the journey over 4 implementations to get speedups over the torch baseline while matching the ~2x memory improvements.
Read below for the journey over 4 implementations to get speedups over the torch baseline while matching the ~2x memory improvements.
🚀 Today we’re releasing FlashOptim: better implementations of Adam, SGD, etc, that compute the same updates but save tons of memory. You can use it right now via `pip install flashoptim`. 🚀
arxiv.org/abs/2602.23349
A bunch of cool ideas make this possible: [1/n]
1
12
72
7,375
Really happy to see that FlashOptim now has an optimized JAX implementation. Huge kudos to @Sam_Acqua for implementing it so quickly and testing it so thoroughly! I really enjoyed the write-up as well, recommended reading
2
3
22
3,657
Blogpost samacquaviva.com/projects/flash… and repo github.com/samacqua/flash…
0
0
7
191
@Clashluke There are a couple of issues here. First, for stochastic rounding you need to increase scaling from ULP/2 -> ULP to keep correctness. Second, the RNE monkeypatching is not working with torch.compile. With those fixed, both ECC settings converge identical to FP32.
why bias matters more than precision, and how you can use this to save 50% memory
convergentthinking.sh/posts/bias-com…
1
0
4
190
Submitted a PR to HeavyBall to fix the ECC (github.com/HomebrewML/Hea…), and revised script for the plot (github.com/ClashLuke/clas…)
0
0
2
55
This is one of my all-time favorite research projects. It was a joy to chase numerics bugs with @davisblalock, @gupta__abhay, and @ChrisRinard.
0
0
2
140
Six whole years later, @jonhoo @anishathalye, and I got together again at @MIT_CSAIL to teach another edition of the Missing Semester, with updated lectures and topics. Hope you all enjoy it!
In January, @jonhoo, @jjgort, and I returned to @MIT_CSAIL to teach Missing Semester, a class on topics missing from most CS programs—tools and techniques that everyone should know, like Bash, Git, CI/CD, and AI tools. Today, we’re releasing the course for free online!
0
5
24
2,295
Attending NeurIPS this week until 7th Dec. Excited to chat about training/inference efficiency, quantization, optimizers or in general just how to make things go fast!!
You can also find me at the @databricks booth tomorrow Friday 9am - 12pm
You can also find me at the @databricks booth tomorrow Friday 9am - 12pm
0
0
9
411
Jose Javier Gonzalez retweeted
@code_star CONVERT YOUR CODEBASES TO REAL NUMBERS, I REPEAT REAL NUMBERS
77
437
5,955
389,326
Jose Javier Gonzalez retweeted
Presenting MultiverSeg — a scalable in-context system for interactively segmenting new datasets — at #ICCV2025 today!
📍poster 110 (10:45 AM–12:45 PM)
📍poster 110 (10:45 AM–12:45 PM)
0
2
7
524