Having lots of fun with local audio models…
Voxtral 2 realtime coming to MLX-Audio thanks @KarnikShreyas and @ochafik
New release this coming week once we land the PR.
I believe we can get to 100-200ms range. Perfect for iOS!
github.com/Blaizzy/mlx-au…
1
0
1
280
Olivier Chafik retweeted
You can now run apps within Claude.ai, powered by MCP Apps.
Analyze data, edit tickets, draft messages, generate diagrams and more with Claude's interactive connectors. Grateful to our team, the open source community, and our partners for this launch 🚀
Analyze data, edit tickets, draft messages, generate diagrams and more with Claude's interactive connectors. Grateful to our team, the open source community, and our partners for this launch 🚀
38
83
1,222
132,634
Had so much fun co-authoring the MCP Apps spec and helping launch it in claude.ai! 🎉
🧵 About some demo apps you may wanna check out as user or developer
🧵 About some demo apps you may wanna check out as user or developer
Your work tools are now interactive in Claude.
Draft Slack messages, visualize ideas as Figma diagrams, or build and see Asana timelines.
2
1
13
1,030
Pdf (github.com/modelcontextpr… ; also available in claude.ai Connectors) will show any arxiv PDF and allow you to ask Claude about any page / selection you need details about
1
0
3
155
Say (github.com/modelcontextpr…) uses the amazing Pocket-TTS to let Claude speak out loud to you, live!
0
0
2
114
Wanna disable thinking in llama.cpp? Try the new `--reasoning-budget 0` flag
github.com/ggml-org/llama…
Should work w/ Qwen3, QwQ, DeepSeek R1 distills, Command R7B; please report any issues!
(Upcoming per-request behaviour discussed on github.com/ggml-org/llama… @ngxson)
#llamacpp
github.com/ggml-org/llama…
Should work w/ Qwen3, QwQ, DeepSeek R1 distills, Command R7B; please report any issues!
(Upcoming per-request behaviour discussed on github.com/ggml-org/llama… @ngxson)
#llamacpp
1
8
44
2,151
@ngxson Preview above wasn't for the PR that introduces --reasoning-budget, here it is again:
github.com/ggml-org/llama…
github.com/ggml-org/llama…
0
0
5
371
Really slick integration! MCP all the (cool) things!
You really can just do things! Use *any* Hugging Face space as a MCP server along with your Local Models! 🔥
Here in we use Qwen 3 30B A3B with @ggml_org llama.cpp and @huggingface tiny agents to create images via FLUX powered by ZeroGPU ⚡
It's quite a bit crazy to see local much and just be able to understand/ infer from tool description!
There's a lot of potential here in automating video generation workflows, content curation and a lot more..
Bonus: you can plug any other Inference Provider if you don't want to run locally too!
npx @ huggingface/tiny-agents run [TASK]
oh, and we provide both typescript and python client! 🐐
0
0
4
272
Olivier Chafik retweeted
Months of @ochafik's work finally paid off, we have MCP w/ llama.cpp 😁😁
biggest takeaway of all is Qwen 3 30B A3B is slept on and you should be playing around with MCP if you haven't already!
0
2
14
1,186
llama.cpp streaming support for tool calling & thoughts was just merged: please test & report any issues 😅
github.com/ggml-org/llama…
#llamacpp
github.com/ggml-org/llama…
#llamacpp
1
6
65
5,484
Olivier Chafik retweeted
Gemma 3 VISION on llama.cpp server
Still very early WIP, but it works 🔥
Still very early WIP, but it works 🔥
4
3
76
3,915
Olivier Chafik retweeted
Docker is embracing the ggml/llama.cpp on-device future
Who is going to be next?
Who is going to be next?
14
34
331
36,544
WIP: streaming support for llama.cpp's tool calling + normalization of streamed <think> tags (for QwQ, DeepSeek R1 & Cohere Command R7B)
Still strongly conforming to tool schemas (grammar-constrained).
Testing / feedback welcome :-)
github.com/ggml-org/llama…
Still strongly conforming to tool schemas (grammar-constrained).
Testing / feedback welcome :-)
github.com/ggml-org/llama…
0
0
8
381
Olivier Chafik retweeted
Day-zero Gemma 3 support in llama.cpp 🤯
👉 4 model sizes: 1B, 4B, 12B, 27B
👉 Vision capability (except for 1B) with bi-direction attention
👉 Context size: 32k (1B) and 128k (4B, 12B, 27B)
👉 +140 languages support (except for 1B)
👉 Day-zero support on many frameworks 🚀
👉 4 model sizes: 1B, 4B, 12B, 27B
👉 Vision capability (except for 1B) with bi-direction attention
👉 Context size: 32k (1B) and 128k (4B, 12B, 27B)
👉 +140 languages support (except for 1B)
👉 Day-zero support on many frameworks 🚀
9
45
293
25,125
Loving this... hope OSS catches up quickly 🤗
Also, conforming to JSON schemas in a diffusion context is going to be an interesting challenge (schema / grammar compliance gradients anyone?)
Also, conforming to JSON schemas in a diffusion context is going to be an interesting challenge (schema / grammar compliance gradients anyone?)
Major AI breakthrough: Diffusion Large Language Models are here!
They're 10x faster and 10x cheaper than traditional LLMs.
Here's everything you need to know:
0
0
0
160
Olivier Chafik retweeted
You're seeing a minor miracle: Phi-4 3.8B doing agentic tool calls.
Doing that reliably, locally, even with 8B, has been a fools errand until @ochafik's opus of a patch series to llama.cpp (PR coming for phi-4)
Doing that reliably, locally, even with 8B, has been a fools errand until @ochafik's opus of a patch series to llama.cpp (PR coming for phi-4)
1
2
10
1,538