Pinned Tweet
@jundotkim , I published a bunch of PR's for OMLX for m5U optimization for Mimo 2.6 flash , GLM 5.3 flash and qwen 3.8 Next .
As always Lossless by design no tricks or benchmax
BLUF : M5 ultra is insanely fast . 🔥
The numbers are pretty wild after a day and a half of lossless optimizing.
Qwen3.8-Flash-Next Prefill: 5.6k tok/s @ 8k–16k · 5.5k @ 64k · 4.5–4.8k @ 4k Decode: 109–124 tok/s (MTP) Peak mem: ~109 GB
GLM-5.3-Flash Prefill: 2.35k tok/s @ 16k & 64k · 2.32k @ 4k Decode: 63 @ 1k · 59 @ 4k–16k · 56 @ 64k (no spec dec) Peak mem: ~187–191 GB
MiMo-V2.6-Flash Prefill: 3.3k tok/s @ 8k Decode: 99–118 @ 8k · 89–99 @ 1k (MTP) Peak mem: ~170–173 GB
As always Lossless by design no tricks or benchmax
BLUF : M5 ultra is insanely fast . 🔥
The numbers are pretty wild after a day and a half of lossless optimizing.
Qwen3.8-Flash-Next Prefill: 5.6k tok/s @ 8k–16k · 5.5k @ 64k · 4.5–4.8k @ 4k Decode: 109–124 tok/s (MTP) Peak mem: ~109 GB
GLM-5.3-Flash Prefill: 2.35k tok/s @ 16k & 64k · 2.32k @ 4k Decode: 63 @ 1k · 59 @ 4k–16k · 56 @ 64k (no spec dec) Peak mem: ~187–191 GB
MiMo-V2.6-Flash Prefill: 3.3k tok/s @ 8k Decode: 99–118 @ 8k · 89–99 @ 1k (MTP) Peak mem: ~170–173 GB
22
6
126
51,259
Jonathan Spangler retweeted
🚨 NEW ON MLX.FAST AND CUDA.FAST: make NVIDIA's Nemotron 3.5 Lightning faster.
Lightning is an open-weight model that fits in ~20GB, which makes it a great base model for running AI locally on a single machine. @nvidia built it as a fast general trained -- so any speedup to the base carries over to every specialized model built on top of it.
On day 1, solvers on @YukonResearch already got it running 31.9% faster on DGX Spark and 13.8% faster on Mac.
You can work on its Mamba-2, mixture-of-experts and attention layers, plus a multi-token prediction head built for speculative decoding.
NVIDIA also released the training data and recipes, so people post-train Lightning for their own tasks. Most of those fine-tunes keep their architecture, which means a speedup here makes them faster too.
Two tracks:
• Apple Silicon → mlx.fast
• NVIDIA DGX Spark → cuda.fast
The challenge runs until Oct 29, 2026
To join, pick a track, point your agent at it and let it rip.
Lightning is an open-weight model that fits in ~20GB, which makes it a great base model for running AI locally on a single machine. @nvidia built it as a fast general trained -- so any speedup to the base carries over to every specialized model built on top of it.
On day 1, solvers on @YukonResearch already got it running 31.9% faster on DGX Spark and 13.8% faster on Mac.
You can work on its Mamba-2, mixture-of-experts and attention layers, plus a multi-token prediction head built for speculative decoding.
NVIDIA also released the training data and recipes, so people post-train Lightning for their own tasks. Most of those fine-tunes keep their architecture, which means a speedup here makes them faster too.
Two tracks:
• Apple Silicon → mlx.fast
• NVIDIA DGX Spark → cuda.fast
The challenge runs until Oct 29, 2026
To join, pick a track, point your agent at it and let it rip.
5
13
46
15,331
Jonathan Spangler retweeted
** ATTENTION DGX SPARK | MLX COMMUNITIES **
We just launched live challenges on cuda.fast and mlx.fast for Nemotron Lightning 3.5 30B-A3B.
This is a perfect model to optimize with and was created for long running agents using high volume tool calls rather than deep reasoning.
Use Cases:
➔ Tool calls, result validation, and subagent delegation
➔ Code edits, reviews, and generation
➔ Document actions, monitoring loops, classification, formatting, retrieval, and summarization
➔ Specialized domain tasks after light post-training (cybersecurity alert triage, legal work, finance/healthcare workflows, coding agents, support/billing, etc.)
@eigenlabs @yukonresearch @sreeramkannan @pratikg @0xkydo @TheDavidTai
We just launched live challenges on cuda.fast and mlx.fast for Nemotron Lightning 3.5 30B-A3B.
This is a perfect model to optimize with and was created for long running agents using high volume tool calls rather than deep reasoning.
Use Cases:
➔ Tool calls, result validation, and subagent delegation
➔ Code edits, reviews, and generation
➔ Document actions, monitoring loops, classification, formatting, retrieval, and summarization
➔ Specialized domain tasks after light post-training (cybersecurity alert triage, legal work, finance/healthcare workflows, coding agents, support/billing, etc.)
@eigenlabs @yukonresearch @sreeramkannan @pratikg @0xkydo @TheDavidTai
2
12
18
1,318
Jonathan Spangler retweeted
🧬 Every cell in your body has the same DNA, yet “something” still tells them which one becomes a heart.
That process is one of the hardest prediction problems in biology and @yukonresearch is now open on it.
Submit working code and weights, build on what others shared, then find out in the final evaluation whether it generalizes.
yukon.org/virtualembryo
That process is one of the hardest prediction problems in biology and @yukonresearch is now open on it.
Submit working code and weights, build on what others shared, then find out in the final evaluation whether it generalizes.
yukon.org/virtualembryo
1
8
24
2,334
RT @sreeramkannan: 🧬 Virtual Embryo x Yukon 🧬
Open Collaborative Model Building
yukon.org/virtualembryo
Yukon connects the world's leadi…
Open Collaborative Model Building
yukon.org/virtualembryo
Yukon connects the world's leadi…
0
22
0
4
Join the competition ! Many machines can compete in this . Can't wait to see how far you all push it.
MLX.fast and CUDA.fast for Bonsai 2 9B is live!
Bonsai 2 9B a 2GB model; so, many many machines can optimize them locally. The Bonsai model we did had 5x in performance increase and I cannot wait to see what the community does with this one!
0
6
13
584
Try out bonsai 9b !
We're excited to offer day-one support for the new Bonsai 2 model.
You can try Bonsai 9B at console.darkbloom.dev. It's a 2GB model that handles many everyday and coding tasks. The first 100 people get 250M free tokens on us to try it out.
For the optimizers, you can also improve Bonsai's performance at mlx.fast and cuda.fast to make it even faster on local machines.
Bringing more capable intelligence to more devices will be a massive unlock for the next stage of local AI. PrismML's work is a critical piece of the puzzle, and at Darkbloom, we're excited to support them from day one. Thank you to the @PrismML team for trusting us.
1
3
16
493
This is the way
A new paper from Apple engineers ends with a line we couldn’t have written better ourselves:
"if idle macOS devices across organizations [...] can be securely federated, they could form a massive, shared GPU resource pool that benefits the broader ecosystem."
That’s what we’re .
The paper's Minions is a system that pool idle Apple Silicon Macs into a distributed GPU cloud for on-prem inference. Queue times went from 10+ hours to minutes. Zero new GPU purchases. Zero infrastructure cost.
It runs inside Apple, on coworkers’ Macs, using the company’s own storage and access controls. When a team needs more capacity, they Slack colleagues and ask them to turn on the executor. Even in that setting, the top adoption concern was data confidentiality. People were wary of their jobs running on a colleague’s Mac. Apple’s fix was to keep each team’s work on its own team’s machines.
Which brings it back to that one word in the conclusion: “securely.”
Inside a company, you can shrink the network until you trust everyone in it. On an open network, the machine belongs to a stranger and there’s no one to Slack. Keeping prompts private and outputs honest on hardware you don’t control is the hard problem that we solved at Darkbloom.
Apple built the on-prem version.
Darkbloom is the open one.
3
2
13
2,438
RT @sreeramkannan: Darkbloom: #1 decentralized inference network
Our goal is to maximize individual agency post AGI. This means more indi…
Our goal is to maximize individual agency post AGI. This means more indi…
0
12
0
4
RT @eigenlabs: Open intelligence needs open infrastructure and open research.
@darkbloomai connects distributed compute with paying custom…
@darkbloomai connects distributed compute with paying custom…
0
7
0
4
Today is my official first day @eigenlabs building @darkbloomai
It’s been almost a month working with the teams here and I cannot speak highly enough of everyone here .
To all involved so far I cannot thank you enough for this opportunity.
Let’s build 🫶
It’s been almost a month working with the teams here and I cannot speak highly enough of everyone here .
To all involved so far I cannot thank you enough for this opportunity.
Let’s build 🫶
12
5
57
3,207
Jonathan Spangler retweeted
I JUST SET UP A MAC MINI TO RUN AI LOCALLY
M5 Pro, 64GB of memory. First six models, measured with Apple's MLX, in tok/s:
- Qwen3 30B at 106
- Qwen3 8B at 61
- Laguna XS 2.1 at 56
- Qwen3 14B at 34
- Ternary Bonsai 2 27B at 26
- Gemma 3 27B at 18
Models from @Alibaba_Qwen, pMind (@googlegemma) and @PrismML.
Qwen3 30B and Laguna XS 2.1 only use about 3B of their parameters for each token, so they run like small models.
Ternary Bonsai 2 27B stores nearly every weight as -1, 0 or 1. It fits in 8.4GB, about half what Gemma 3 27B needs.
When I'm not using it, the Mac serves open-weight models on @darkbloomai, the compute grid powered by people's Macs. (I work at @eigenlabs which builds it)
I strongly believe it matters whether people rent AI or own it. A model on your own machine can't be taken away or quietly changed. You can run it, take it apart and build on it without asking anyone's permission.
Open-weight models and open source AI make that freedom real and as AI shapes more of how we work and think, it's worth protecting.
I've also installed @ollama and @lmstudio for chatting with models in an app and @tailscale so I can reach the Mac from anywhere
I'm not an engineer. I'm learning in public and logging everything in a repo. (check first reply)
Local AI folks I keep learning from, who share what they figure out in public:
- @ggerganov
- @awnihannun
- @danielhanchen
- @jpschroeder
- @ivanfioravanti
- @MichaelGannotti
- @Tim_Dettmers
- @simonw
- @Prince_Canuma
- @rasbt
- @maximelabonne
- @TheAhmadOsman
- @QuixiAI
- @loktar00
- @itsjustmarky
- @jmorgan
- @ddalcu
- @GumbiiDigital
- @MikeBradleyAI
- @StefanMaier
- @ashen_one
- @TheDavidTai
- @Spangler3000
- @ashxhart
Who else should I follow? I'm adding everyone to my Local AI Builders list (link below).
M5 Pro, 64GB of memory. First six models, measured with Apple's MLX, in tok/s:
- Qwen3 30B at 106
- Qwen3 8B at 61
- Laguna XS 2.1 at 56
- Qwen3 14B at 34
- Ternary Bonsai 2 27B at 26
- Gemma 3 27B at 18
Models from @Alibaba_Qwen, pMind (@googlegemma) and @PrismML.
Qwen3 30B and Laguna XS 2.1 only use about 3B of their parameters for each token, so they run like small models.
Ternary Bonsai 2 27B stores nearly every weight as -1, 0 or 1. It fits in 8.4GB, about half what Gemma 3 27B needs.
When I'm not using it, the Mac serves open-weight models on @darkbloomai, the compute grid powered by people's Macs. (I work at @eigenlabs which builds it)
I strongly believe it matters whether people rent AI or own it. A model on your own machine can't be taken away or quietly changed. You can run it, take it apart and build on it without asking anyone's permission.
Open-weight models and open source AI make that freedom real and as AI shapes more of how we work and think, it's worth protecting.
I've also installed @ollama and @lmstudio for chatting with models in an app and @tailscale so I can reach the Mac from anywhere
I'm not an engineer. I'm learning in public and logging everything in a repo. (check first reply)
Local AI folks I keep learning from, who share what they figure out in public:
- @ggerganov
- @awnihannun
- @danielhanchen
- @jpschroeder
- @ivanfioravanti
- @MichaelGannotti
- @Tim_Dettmers
- @simonw
- @Prince_Canuma
- @rasbt
- @maximelabonne
- @TheAhmadOsman
- @QuixiAI
- @loktar00
- @itsjustmarky
- @jmorgan
- @ddalcu
- @GumbiiDigital
- @MikeBradleyAI
- @StefanMaier
- @ashen_one
- @TheDavidTai
- @Spangler3000
- @ashxhart
Who else should I follow? I'm adding everyone to my Local AI Builders list (link below).
34
7
182
65,597
Jonathan Spangler retweeted
It's time to push Local AI into the real world!
Models are good enough.
Inference Engines are good enough.
Let's show what we can build on it!
Models are good enough.
Inference Engines are good enough.
Let's show what we can build on it!
27
2
143
7,514
Local AI is the way forward . Such an amazing community powering it all . Own your access to intelligence .
0
3
15
597
Jonathan Spangler retweeted
Hard problems need great partners.
Yukon works with some of the best scientists, researchers and open-weight model labs in the world.
We turn hard problems into open challenges anyone can work on.
THANK YOU to every collaborator! 🙏
- @Stanford
- @poolsideai
- @UCBerkeley
- ton
- @Lighter_xyz
- @CarnegieMellon
- @ccaiwut
- @SuccinctLabs
- @EspressoSys
- @StarkWareLtd
- @powdr_labs
- @PrismML
- @ShieldedLabs
- @Zcash
- @projecteleven
- @zksecurityXYZ
+ the PARI/GP developers.
And to the researchers/scientists we worked with directly:
- @nasqret
- @johnkitchin
- @viswanathpramod
- @ZeruiCheng
- @MangQiuyang
- @AuroraSUN1108
- @KaiyuanLiu04
Yukon works with some of the best scientists, researchers and open-weight model labs in the world.
We turn hard problems into open challenges anyone can work on.
THANK YOU to every collaborator! 🙏
- @Stanford
- @poolsideai
- @UCBerkeley
- ton
- @Lighter_xyz
- @CarnegieMellon
- @ccaiwut
- @SuccinctLabs
- @EspressoSys
- @StarkWareLtd
- @powdr_labs
- @PrismML
- @ShieldedLabs
- @Zcash
- @projecteleven
- @zksecurityXYZ
+ the PARI/GP developers.
And to the researchers/scientists we worked with directly:
- @nasqret
- @johnkitchin
- @viswanathpramod
- @ZeruiCheng
- @MangQiuyang
- @AuroraSUN1108
- @KaiyuanLiu04
2
9
47
3,537
Jonathan Spangler retweeted
Learn how mlx-stream extension for mlx-serve works with DeepSeek V4.1 to achieve almost 40tps decode. x.com/i/article/2107…
5
10
68
10,254
After a few weeks with the m5 ultra all I can say is wow 🔥 only bad news is now I have to buy at least one 512gb model , its too good . Putting my old machines on @darkbloomai instead of selling has been the perfect use case for them . Buy hardware, run local ai 💪
1
0
9
466