FULL RECAP: Apple Intelligence Group Lab, the short version for Local AI builders

Sharing highlights from this week's session hosted by @shshnkp (Apple AI/ML Evangelism Team) with @mgorbach and engineers from Foundation Models, Core AI, Siri and App Intents teams at Apple.

1) an use your open-weight model as its LLM. Apple’s new app framework can use Apple’s on-device model, Apple’s cloud model or a Core AI converted model, which is interchangeable with a one liner! And your model gets access to the same tool calling, structured output, profiling with Instruments, etc. as Apple’s models.

2) Don’t use Apple’s on-device model for coding, even according to Apple engineers! Use Core AI to bring in a model for coding. One engineer uses a setup where he can use MLX and Core AI models in different “coding harnesses” and reports good experiences.

3) Free local Chat Completions endpoint on any Apple silicon Mac with macOS 27! fm serve provides an OpenAI style Chat Completions endpoint backed by Apple’s on-device model on any Mac. Nothing to install! (shared by Shashank the day after the lab)

4) Apple’s new on-device model is 20B, but only has 8K context. AFM Core Advanced (20B) will be available on the latest iPhones, iPads and M3+ Macs. Can’t opt out of Apple Intelligence in 27, and the model will always be present. Tip: use typed units for fields and let constrained decoding handle it (instead of wasting tokens explaining in prompt).

5) Agentic Siri only works when online. Tool calls from Siri to apps are done using Private Cloud Compute. When offline, Siri will use the on-device model but won’t be able to call into apps.

6) Lessons Apple engineers follow when building agents apply to any agent:

- write down instructions to the model about how/when to use your tools in a markdown file (“skills” are the same idea as agent skills everywhere else!)
- skills are just context to the model, meaning no ordering guarantees. If something should always happen in an order, it should probably be a tool
- make tools accept lists of items to operate on to avoid agent having to call it in a loop
- unit test your tools (deterministic) before testing the whole agent (non-deterministic)

Curated Q&A relevant to Local AI builders in 🧵

A few weeks ago, I wrapped up 7 great years at Apple. I've since joined a startup!

Looking back, I’m really proud of the core developer frameworks and SDKs we built, and of helping bring free, fast on-device AI models, guided generation, and Private Cloud Compute to app developers. To me, great API design is an art form. It takes taste, empathy for developers, and a relentless pursuit of simplicity.

I’m also grateful for the opportunity to help shape lots of WWDC videos, speak twice at the WWDC Platforms State of the Union, and build multiple teams from scratch. Above all, these experiences reinforced my belief in what small, talented teams can accomplish. That feels especially true today.

Thank you to everyone who made this chapter so special. I’m staying in SF and can’t wait to share what’s next.

At yesterday's Apple Intelligence Group Lab livestream, many of you had questions about Apple Foundation Models context size. We recently published a doc Article with more details.
developer.apple.com/documentation/…

You can also see it on the terminal using the fm cli.

Also, fun fact: You tions API server using the on-device model using fm serve

This is huge news for Siri AI (and one of the last things I was working on when I stepped away)

Any app can now offer dynamic skills from their app to Siri so users can take broader actions across their apps

(App Intents instead of MCP)

@alpennece@bitrige@orca_buildp@t3dotcodess@jakemorment experience for really serious builders. Some examples:

- Providing world class simulator use tools for your agent
- A system that allows agents to work with Xcodebuild in a way that gets out of your way (priority queueing builds based on whether you're looking at them / the agent is doing them in the background, etc. More to come here soon as well, I have something really special cooking 🧑‍🍳)
- Support for sims even apple doesn't provide (CarPlay sim!)
- A great out-of-the-box suite of skills + tools that help with Apple platforms development
- Best-in-class support for letting your agent use App Store Connect so you never need to worry about things like taking screenshots, uploading builds, etc.

There are many great orchestrators out there, but there are a lot of unique things about development for Apple's platforms that a standard Agent + Orchestrator setup doesn't equip your AI well to handle. That's what Bitrig is here for!

Huge day for Creator Studio today.

Our new in-app assistants can help you get things done in record time. MCP allows you to bring your own agent if you want. And an open source format for Keynote (and Python library!) so you can create Keynote documents programmatically on any platform.

Super proud of the team. Hope you love it!

wtf i just discovered that a macbook comes with a free local LLM in your terminal

Today we also published benchmarks and a deep dive into LLM inference on M5 Ultra Mac Studio, covering MLX, GPU Neural Accelerators, tensor and pipeline parallelism, and Thunderbolt 5 RDMA.

machinelearning.apple.com/research/mlx-d…

You can now use @OpenAIDevs ‘s Decision model via Apple’s Foundation Model framework!

One API to rule them all 💍

Decision models like Jev are all the rage - they are super fast, dirt cheap, and - best of all - deterministic.

What if... you could use Jev (and other System One models) using Apple Foundation Models framework?

Meet System One models in Swift:
go.peterfriese.dev/system-one-mod…

💯

Want to use @typesafeai's Jev in your @SwiftLang apps using the convenience of Apple's Foundation Models framework?

Well - here you go (GitHub repo link in the replies)!

(h/t to @rxwei for the inspiration!)

Jev is a fantastic example of co-designing SDKs and models to unlock a new paradigm!

When we were designing the Swift API for Apple’s Foundation Models framework two years ago, we wanted to make type-safe structured responses the standard for apps on our platform. That is why we e` API, making constrained decoding approachable to app developers. This effectively eliminated structural hallucinations in both tool call arguments and responses, which were especially important for a small on-device model. developer.apple.com/documentation/…

It would be fascinating to see decision models conform to the `LanguageModel` protocol and work with `@​Generable` natively! The best part is that all the APIs are already there: Bools are "noul", enums represent choices, and `@​Guide` descriptions are instructions. A Swift model implementation simply needs to handle a schema conversation, and output probabilities can be stored directly in the response metadata. developer.apple.com/documentation/…

I clearly made some typos in the code example! 😅 The request should look like this:

It supports both the on-device model and Private Cloud Compute! We built it to enable effortless automations for Mac users. It's free, zero setup, no privacy compromises.

I've been using the on-device Apple foundation model to test out the SoC performance on the iPhone 17 Pro Max and 18 Pro Max

A19 Pro
Long Prompt: 2340ms TTFT, 45 tok/s
Medium Prompt: 946ms TTFT, 70 tok/s
Short Prompt: 638ms TTFT, 100 tok/s

A20 Pro
Long Prompt: 1125ms TTFT, 55 tok/s
Medium Prompt: 437ms TTFT, 71 tok/s
Short Prompt: 354ms TTFT, 150 tok/s

With iOS 27 Apple introduced its most powerful on-device model: AFM 3 Core Advanced

It’s a 20 billion parameters model activating only between 1 to 4 billion parameters depending on the request

Here’s the model running on iPhone 17 Pro

At our WWDC26 Recap event this week, I shared an overview of bringing intelligence into your apps on Apple platforms.
1/ Foundation Models Framework: Native Swift API for on-device, PCC & third-party LLMs (e.g. @GoogleDeepMind gemini, @claudeai, and others.)
2/ Core AI: Purpbringing custom models into your apps on Apple Silicon
3/ MLX: Open-source framework for model research, fine-tuning, local inference servers, and distributed computing
4/ Agentic Experiences: Multi-model workflows and context handoffs with specialized agents
5/ Evaluations: Automated tools to measure quality, uncover failures, and help you ship with confidence

Watch: youtube.com/live/V2i8f_NeK…

The overview maps everything else, including the bit nobody mentions: macOS 27 ships an fm CLI, and fm serve gives you a local OpenAI-compatible endpoint backed by Apple's models. I'll update the post as I publish the rest: ivanmagda.dev/posts/wwdc26-f…

My favorite part: Dynamic Profiles. One session swaps instructions, tools, and the model mid-conversation via a SwiftUI-style body. I built these mechanisms by hand for my coding-agent series, so I mapped Apple's versions onto ours, footguns included: ivanmagda.dev/posts/foundati…