Your Mac has a fast, offline LLM

2025-12-14

OK, the title might be a little click-baity, but it's true (given that you've updated to the latest MacOS version). I'm late to the party, so maybe you already know about Apple's Foundation Models framework, but I didn't until recently. It was announced in June 2025 and shipped in iOS/macOS 26 in September 2025.

It's a local LLM that lets you do most of what you'd do with a hosted LLM. Don't expect it to perform like ChatGPT or similar, but it's fairly capable.

I was surprised to find that Apple doesn't use this as an offline backup for Siri or provide a typical chat interface to it, but that makes more sense once you try it. It's noticeably worse than hosted LLMs and more locked down, but it's not really designed to be a hosted LLM replacement. Though, I do think it can be useful in a pinch if you're offline and addicted to using LLMs like I am.

To run it, you need:

To develop, you need Xcode 26.

Quack is the source code for a simple example that I built—mostly vibe-coded in less than an hour since I don't know Swift.

This is what it looks like:

Screenshot of a simple chat interface saying hi to an LLM

Trying some things out, you'll quickly learn that it's not as free-flowing as something like ChatGPT.

Screenshot of a chat interface with the LLM apologizing it can't answer training data questions

Anything covered by Apple's Acceptable use requirements for the Foundation Models framework is pretty locked down.

But it's generally pretty good at giving some info about history.

Screenshot of a chat interface with the LLM responding with historical facts about 2020

Or doing some simple coding examples:

Screenshot of a chat interface with the LLM responding with a python script

Or writing a short story:

Screenshot of a chat interface with the LLM responding with a short Lord of the Rings-style story

Or many things you might use a hosted model for!

I found a couple other chat apps that are way better than mine like

so I would look at those. But it's also fun and easy to create your own!

The coolest thing is that this already works entirely offline on any device with Apple Intelligence. It'll be interesting to see what people build with it, but may be a hard sell to tie API calls to Apple-only devices and only available in Swift with some weird restrictions.

Overall, I found it pretty awesome!