Karim Mattar retweeted
🚨 BREAKING: Bonsai 2 9B now runs 5x faster on both Mac and NVIDIA.
Apple Silicon (MLX) is at +442.3%. Decode went from 144.9 to 913.1 tokens/sec.
NVIDIA DGX Spark (CUDA) is at +401.1%. The two tracks have traded the lead all week. Mac holds it right now.
5 days into the open from @PrismML x @DarkbloomAI on @YukonResearch via mlx.fast.
13 solvers and 113 promoted runs, with Opus 5.5, GPT-6 Astra and DeepSeek-V4 agents all on the board.
My agents took the top spot twice overnight (now #3 and #5) and +41.9% on Saturday is still the biggest single jump on the Mac track.
A 2GB open-weight model, made 5x faster in 5 days by people on their own machines, with every new best open for anyone to build on.
Open frontier research accelerates open intelligence.
Apple Silicon (MLX) is at +442.3%. Decode went from 144.9 to 913.1 tokens/sec.
NVIDIA DGX Spark (CUDA) is at +401.1%. The two tracks have traded the lead all week. Mac holds it right now.
5 days into the open from @PrismML x @DarkbloomAI on @YukonResearch via mlx.fast.
13 solvers and 113 promoted runs, with Opus 5.5, GPT-6 Astra and DeepSeek-V4 agents all on the board.
My agents took the top spot twice overnight (now #3 and #5) and +41.9% on Saturday is still the biggest single jump on the Mac track.
A 2GB open-weight model, made 5x faster in 5 days by people on their own machines, with every new best open for anyone to build on.
Open frontier research accelerates open intelligence.
🚨 JUST LAUNCHED: Bonsai 2 from @PrismML.
Open-weight ternary models in 2B, 4B and 9B, small enough to run on a phone. The 2B hits 90 tokens/sec on iPhone 17 Pro Max, with reasoning, vision and tool calling all on-device.
You can try Bonsai 2 9B today on @darkbloomai, where the 0M free tokens.
And we’re making it faster on @YukonResearch.
Bonsai 2 9B runs at 170+ tokens/sec at launch.
There are two tracks, Apple Silicon on mlx.fast and NVIDIA DGX Spark on cuda.fast.
To join, pick a track, point your agent at it and submit a faster run.
Every new best is open for anyone to build on.
14
16
265
17,975
Karim Mattar retweeted
How much faster can a compact local model run when we optimize the inference engine?
We’ve opened two Bonsai 2 9B challenges on @yukonresearch with @PrismML :
- mlx.fast on Apple Silicon
- cuda.fast on NVIDIA DGX Spark.
Optimize kernels, caches g while keeping each track’s target weights fixed.
Pass the correctness gates, improve the paired benchmark score,climb the leaderboard and let others build on your best.
Full walkthrough on how to get started below
We’ve opened two Bonsai 2 9B challenges on @yukonresearch with @PrismML :
- mlx.fast on Apple Silicon
- cuda.fast on NVIDIA DGX Spark.
Optimize kernels, caches g while keeping each track’s target weights fixed.
Pass the correctness gates, improve the paired benchmark score,climb the leaderboard and let others build on your best.
Full walkthrough on how to get started below
3
8
54
1,449
Karim Mattar retweeted
More testing of Ternary Bonsai 2 - looking at 9B this time, on my RTX 3080 10GB.
With intelligence, this is close to the best of the 4-bit competition that I normally work with (NeoHorse and Ornith).
With speed, it's in another league - almost 30% faster with prompts and 75% ary models are here to stay. 👉
With intelligence, this is close to the best of the 4-bit competition that I normally work with (NeoHorse and Ornith).
With speed, it's in another league - almost 30% faster with prompts and 75% ary models are here to stay. 👉
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
11
10
81
4,568
Karim Mattar retweeted
1000 tok/s in the browser is nuts
Today we're excited to extend the Bonsai 2 family of models with ternary 2B, 4B, and 9B models based on Qwen 3.5. These may not solve IMO problems or the Riemann hypothesis but are incredible for high volume and high speed tasks on everyday devices.
The Bonsai 2 4B model runs 32 y Macbook Pro (M5 Pro) to process 250 emails and parse them into JSONs with summaries, action items, calendar events, and priority scoring. The peak memory use is less than 4GB and in total generate over 700 tokens per second.
1
3
18
1,475
Karim Mattar retweeted
Come try out the new Bonsai 2 model on @darkbloomai with 250M tokens.
If you're not familiar with Darkbloom, it's 1100+ Apple Silicon devices from around the world that serve secure inference.
Check it out and add your Apple Silicon to the mesh and earn from serving...Also as a provider, you can use an API key to run the models locally.
If you're not familiar with Darkbloom, it's 1100+ Apple Silicon devices from around the world that serve secure inference.
Check it out and add your Apple Silicon to the mesh and earn from serving...Also as a provider, you can use an API key to run the models locally.
We're excited to offer day-one support for the new Bonsai 2 model.
You can try Bonsai 9B at console.darkbloom.dev. It's a 2GB model that handles many everyday and coding tasks. The first 100 people get 250M free tokens on us to try it out.
For the optimizers, you can also improve Bonsai's performance at mlx.fast and cuda.fast to make it even faster on local machines.
Bringing more capable intelligence to more devices will be a massive unlock for the next stage of local AI. PrismML's work is a critical piece of the puzzle, and at Darkbloom, we're excited to support them from day one. Thank you to the @PrismML team for trusting us.
0
1
7
309
Karim Mattar retweeted
Small, but mighty 🌳
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
0
4
10
975
Karim Mattar retweeted
A 9B model with vision and tool calling in 2GB. That's smaller than most of the apps on your phone.
Bonsai 2 is ternary: every weight is just -1, 0 or 1. What you need to run it:
> 2B: 0.43GB
> 4B: 0.95GB
> 9B: 2.0GB
Built on Qwen3.5. Any phone or old laptop has the memory for the biggest one. Local AI just stopped needing a GPU budget.
Bonsai 2 is ternary: every weight is just -1, 0 or 1. What you need to run it:
> 2B: 0.43GB
> 4B: 0.95GB
> 9B: 2.0GB
Built on Qwen3.5. Any phone or old laptop has the memory for the biggest one. Local AI just stopped needing a GPU budget.
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
0
1
4
338
Karim Mattar retweeted
Getting a model onto a phone is one milestone. Making it useful enough to stay there is the next.
With Bonsai 2, we focused on the everyday stuff: following instructions, calling tools, and getting to the answer without rambling.
What would you want it to handle locally?
With Bonsai 2, we focused on the everyday stuff: following instructions, calling tools, and getting to the answer without rambling.
What would you want it to handle locally?
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
1
5
12
823
Karim Mattar retweeted
Join the competition ! Many machines can compete in this . Can't wait to see how far you all push it.
MLX.fast and CUDA.fast for Bonsai 2 9B is live!
Bonsai 2 9B a 2GB model; so, many many machines can optimize them locally. The Bonsai model we did had 5x in performance increase and I cannot wait to see what the community does with this one!
0
6
13
601
Karim Mattar retweeted
We’re also launching an optimization competition with @darkbloomai to make Bonsai 2 9B faster. Check it out on @yukonresearch!
4
12
74
6,608
Karim Mattar retweeted
Bonsai 2 9B is LIVE on Darkbloom from day one.
@PrismML’s new model fits reasoning, vision and tool calling into 2 GB and handles everyday and coding tasks.
It runs on a network of Macs owned by everyday people.
The first 100 people get 250M free tokens at darkbloom.ai.
Want it faster? Bonsai 2 9B is also an open challenge at mlx.fast and cuda.fast on @YukonResearch.
@PrismML’s new model fits reasoning, vision and tool calling into 2 GB and handles everyday and coding tasks.
It runs on a network of Macs owned by everyday people.
The first 100 people get 250M free tokens at darkbloom.ai.
Want it faster? Bonsai 2 9B is also an open challenge at mlx.fast and cuda.fast on @YukonResearch.
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
9
16
100
8,106
Karim Mattar retweeted
~ ATTENTION MLX | CUDA COMMUNITIES ~
The new CUDA.fast and MLX.fast challenges are now LIVE with the JUST RELEASED Bonsai 2 9b by @PrismML
Now's the time to show off your skills and try to top the leaderboard....AND stay there.
Current benchmarks:
CUDA: 138.6%
MLX: 13% - Let’s get this higher!
Huge shoutout to @TheDavidTai for engineering and getting these challenges off the ground!
@eigenlabs @yukonresearch @pratikg @darkbloomai
The new CUDA.fast and MLX.fast challenges are now LIVE with the JUST RELEASED Bonsai 2 9b by @PrismML
Now's the time to show off your skills and try to top the leaderboard....AND stay there.
Current benchmarks:
CUDA: 138.6%
MLX: 13% - Let’s get this higher!
Huge shoutout to @TheDavidTai for engineering and getting these challenges off the ground!
@eigenlabs @yukonresearch @pratikg @darkbloomai
2
9
22
1,702
Karim Mattar retweeted
Run Ternary Bonsai 2 locally in Atomic Chat 🪴
@PrismML shrank Qwen3.5 7x so the 9B goes from 17.9 GB to just 2.42 GB and it keeps up or outperforms the original model on benchmarks
Run AI models locally -> atomic.chat
@PrismML shrank Qwen3.5 7x so the 9B goes from 17.9 GB to just 2.42 GB and it keeps up or outperforms the original model on benchmarks
Run AI models locally -> atomic.chat
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
16
145
1,577
78,334
Karim Mattar retweeted
Congratulations @PrismML on releasing Bonsai 2 2B, 4B and 9B 👏👏
If you are building AI products with tight memory, latency, or power requirements, reach out at contact@prismml.com
@BabakHassibi @HessianFree @SahinLale
cc @khoslaventures @vkhosla
If you are building AI products with tight memory, latency, or power requirements, reach out at contact@prismml.com
@BabakHassibi @HessianFree @SahinLale
cc @khoslaventures @vkhosla
Today, we’re releasing Bonsai 2 2B, 4B, and 9B.
Based on Qwen3.5, these new ternary models bring stronger reasoning, visual understanding, and tool calling to compact footprints designed for everyday devices:
• 2B: 0.43GB
• 4B: 0.95GB
• 9B: 2.0GB
For this release, we mall models useful as local assistants: following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without unnecessary output. The 9B model uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. Solving the Riemann hypothesis can wait.
These models are successors to our first-generation Bonsai 1.7B, 4B, and 8B, and provide a foundation for bringing practical agentic functionality to local applications on personal computers, phones, and wearables.
The Bonsai 2 family now includes four sizes spanning 2B to 27B parameters. Developers can choose the balance of capability, memory use, and responsiveness that fits their application and hardware.
Available today under Apache 2.0.
2
4
18
1,309
Karim Mattar retweeted
@pratikg @PrismML @darkbloomai The speed jumps from on-device to cloud are wild here. We actually broke down what makes these ternary models hit those numbers: x.com/AGTPinsights/s…
PrismML just released Bonsai 2 ternary models for phones and laptops. Here's what you need to know.
The new models come in 2B, 4B and 9B sizes. They're built on Qwen3.5 and use ternary weights (-1, 0, +1). They're the successors to the first-generation Bonsai 1.7B, 4B and 8B. ning, visual understanding and tool calling, and they're small enough for everyday devices. PrismML says they're more than 9x smaller than their full-precision counterparts.
This release is aimed at local assistants. PrismML focused on following instructions, avoiding repetitive loops, calling tools on mobile devices, and giving useful answers without extra output. The 9B uses roughly 40% fewer tokens on standard instruction-following and everyday math tasks. PrismML's line: solving the Riemann hypothesis can wait.
Speed is the other selling point. On an iPhone 18 Pro Max, the 9B reaches up to 34 tokens per second and the 2B up to 120. In a browser demo on an M4 Max, the 2B reaches up to 500 tokens per second. PrismML also showed the 9B running agentic tasks on an iPhone, and as a local layer in a hybrid on-device and cloud workflow on Mac.
The weights are on Hugging Face in GGUF and MLX formats, under Apache 2.0. The Bonsai 2 family now spans four sizes, from 2B to 27B. The Locally AI app on iPhone and iPad already supports the new models. Darkbloom offers day-one API access to the 9B, with 250M free tokens for the first 100 people.
Key numbers:
- 2B: 0.43GB
- 4B: 0.95GB
- 9B: 2.0GB
- 9B: about 40% fewer tokens on instruction-following and everyday math
- iPhone 18 Pro Max: up to 34 tok/s (9B), up to 120 tok/s (2B)
- M4 Max in browser: up to 500 tok/s (2B)
The models were announced October 16, 2026, and are available today.
0
1
1
140
Karim Mattar retweeted
This is incredible!
A 9B-parameter AI model running locally on your phone with just a 2.6 GB memory footprint!
Just look at what cutting-edge AI research and engineering can achieve!
Amazing work by our team at @PrismML, and super excited about this competition!
A 9B-parameter AI model running locally on your phone with just a 2.6 GB memory footprint!
Just look at what cutting-edge AI research and engineering can achieve!
Amazing work by our team at @PrismML, and super excited about this competition!
We’re also launching an optimization competition with @darkbloomai to make Bonsai 2 9B faster. Check it out on @yukonresearch!
1
1
6
235
Karim Mattar retweeted
PrismML squeezed a 9B vision + tool-calling AI agent into a few GB and got it running at 34 tps on an iPhone. 👀
And the 2B version hits 120 tps.
The new Bonsai 2 models are based on Qwen3.5 and designed specifically for Local AI agents.
On iPhone 18 Pro Max, PrismML reports
⚡ Bonsai 2 2B → 120 tps
Apple/MLX package sizes including vision
💾 9B → 3.46GB
💾 2B → 1.20GB
And they aren't simple tiny chat models.
👁️ vision
🔧 tool calling
🧠 reasoning
📚 262K context
🔓 Apache 2.0
Quality retention is remarkable!
Bonsai 2 9B vs full Qwen3.5-9B:
🔧 agent/tool capability → 92.4% retained
🧮 math → 98.2%
🎯 instruction following → ~100%
💻 coding → ~100%
🧠 knowledge/reasoning → 93.0%
👁️ vision → 92.5%
On the mobile tool-calling benchmark specifically
Qwen3.5-9B → 82.02
Bonsai 2 9B → 78.02
And PrismML says the 9B uses roughly 40% fewer output tokens on its everyday math + instruction-following tests.
There are even smaller GGUF builds:
🌳 9B language model → 2.00GB
🌳 2B language model → 0.43GB
The 2B GGUF + Q8 vision projector is only ~0.79GB of model files. 🤯
🔗 Links in ALT
And the 2B version hits 120 tps.
The new Bonsai 2 models are based on Qwen3.5 and designed specifically for Local AI agents.
On iPhone 18 Pro Max, PrismML reports
⚡ Bonsai 2 2B → 120 tps
Apple/MLX package sizes including vision
💾 9B → 3.46GB
💾 2B → 1.20GB
And they aren't simple tiny chat models.
👁️ vision
🔧 tool calling
🧠 reasoning
📚 262K context
🔓 Apache 2.0
Quality retention is remarkable!
Bonsai 2 9B vs full Qwen3.5-9B:
🔧 agent/tool capability → 92.4% retained
🧮 math → 98.2%
🎯 instruction following → ~100%
💻 coding → ~100%
🧠 knowledge/reasoning → 93.0%
👁️ vision → 92.5%
On the mobile tool-calling benchmark specifically
Qwen3.5-9B → 82.02
Bonsai 2 9B → 78.02
And PrismML says the 9B uses roughly 40% fewer output tokens on its everyday math + instruction-following tests.
There are even smaller GGUF builds:
🌳 9B language model → 2.00GB
🌳 2B language model → 0.43GB
The 2B GGUF + Q8 vision projector is only ~0.79GB of model files. 🤯
🔗 Links in ALT
7
7
44
2,371
RT @yukonresearch: New challenge on Yukon: Make Bonsai 2 9B faster.
@PrismML’s new Bonsai 2 9B packs reasoning, vision and tool calling in…
@PrismML’s new Bonsai 2 9B packs reasoning, vision and tool calling in…
0
12
0