engineer
San Francisco, CA
Joined September 2009
- Tweets580
- Following354
- Followers5.1K
- Likes2.1K
Richard Wei retweeted
Today we also published benchmarks and a deep dive into LLM inference on M5 Ultra Mac Studio, covering MLX, GPU Neural Accelerators, tensor and pipeline parallelism, and Thunderbolt 5 RDMA.
machinelearning.apple.com/re…
Muse Spark Open -- weights releasing today -- runs great on new Mac Studios thanks to work with @johnternus and Apple.
Richard Wei retweeted
You can now use @OpenAIDevs ‘s Decision model via Apple’s Foundation Model framework!
One API to rule them all 💍
Richard Wei retweeted
Decision models like Jev are all the rage - they are super fast, dirt cheap, and - best of all - deterministic.
What if... you could use Jev (and other System One models) using Apple Foundation Models framework?
Meet System One models in Swift:
go.peterfriese.dev/system-on…
Richard Wei retweeted
Want to use @typesafeai's Jev in your @SwiftLang apps using the convenience of Apple's Foundation Models framework?
Well - here you go (GitHub repo link in the replies)!
(h/t to @rxwei for the inspiration!)
Jev is a fantastic example of co-designing SDKs and models to unlock a new paradigm!
When we were designing the Swift API for Apple’s Foundation Models framework two years ago, we wanted to make type-safe structured responses the standard for apps on our platform. That is why we created the `@Generable` API, making constrained decoding approachable to app developers. This effectively eliminated structural hallucinations in both tool call arguments and responses, which were especially important for a small on-device model. developer.apple.com/document…
It would be fascinating to see decision models conform to the `LanguageModel` protocol and work with `@Generable` natively! The best part is that all the APIs are already there: Bools are "noul", enums represent choices, and `@Guide` descriptions are instructions. A Swift model implementation simply needs to handle a schema conversation, and output probabilities can be stored directly in the response metadata. developer.apple.com/document…
Richard Wei retweeted
I've been using the on-device Apple foundation model to test out the SoC performance on the iPhone 17 Pro Max and 18 Pro Max
A19 Pro
Long Prompt: 2340ms TTFT, 45 tok/s
Medium Prompt: 946ms TTFT, 70 tok/s
Short Prompt: 638ms TTFT, 100 tok/s
A20 Pro
Long Prompt: 1125ms TTFT, 55 tok/s
Medium Prompt: 437ms TTFT, 71 tok/s
Short Prompt: 354ms TTFT, 150 tok/s
Richard Wei retweeted
With iOS 27 Apple introduced its most powerful on-device model: AFM 3 Core Advanced
It’s a 20 billion parameters model activating only between 1 to 4 billion parameters depending on the request
Here’s the model running on iPhone 17 Pro
Richard Wei retweeted
At our WWDC26 Recap event this week, I shared an overview of bringing intelligence into your apps on Apple platforms.
1/ Foundation Models Framework: Native Swift API for on-device, PCC & third-party LLMs (e.g. @GoogleDeepMind gemini, @claudeai, and others.)
2/ Core AI: Purpose-built framework for bringing custom models into your apps on Apple Silicon
3/ MLX: Open-source framework for model research, fine-tuning, local inference servers, and distributed computing
4/ Agentic Experiences: Multi-model workflows and context handoffs with specialized agents
5/ Evaluations: Automated tools to measure quality, uncover failures, and help you ship with confidence
Watch: youtube.com/live/V2i8f_NeKDI…
Richard Wei retweeted
The overview maps everything else, including the bit nobody mentions: macOS 27 ships an fm CLI, and fm serve gives you a local OpenAI-compatible endpoint backed by Apple's models. I'll update the post as I publish the rest: ivanmagda.dev/posts/wwdc26-f…
Richard Wei retweeted
My favorite part: Dynamic Profiles. One session swaps instructions, tools, and the model mid-conversation via a SwiftUI-style body. I built these mechanisms by hand for my coding-agent series, so I mapped Apple's versions onto ours, footguns included: ivanmagda.dev/posts/foundati…
Richard Wei retweeted
PrivateCloudComputeLanguageModel: a 32K-context server model with reasoning, switched on with one line. No account, no keys, no per-token bill for eligible apps. Each user spends from a daily quota instead, and that quota deserves real UI. Deep dive: ivanmagda.dev/posts/foundati…
Richard Wei retweeted
Last June, Foundation Models was a Swift API to one small on-device model. After WWDC26 the same session API runs Apple's server model, Claude, Gemini, or a local Qwen. I spent the week in the betas. Three posts on what changed:
Richard Wei retweeted
We had a blast at the Machine Learning & AI Group Lab today! ML technologies, frameworks, models, evaluations, and lots of fun with @trebolloc, @LouisDhauwe and others.
🎬 Video is up on YouTube - questions and discussions on Core AI, MLX, Foundation Models framework, Evaluations and more!
WWDC26 was amazing for AI/ML. Super excited for all the incredible experiences you’ll build with these technologies! Thank you!
youtube.com/live/n6TdbD7nh2k…
Richard Wei retweeted
You can now use Gemini via Apple's Foundation Model framework.
This means:
- same calls for on-device and cloud-hosted inference
- you can use Grounding with Google Search / Google Maps
- access to Gemini's large context window (up to 1M tokens)
Richard Wei retweeted
Foundation Modelsめっちゃ進化してる😲iOS 27では、オンデバイスモデルにVision機能が追加されたことにより、画像に関する質問にも答えられるように。
UIImage, NSImage, CGImage, Core Image 型, CoreVideo, Pixel Buffer, ファイル URLの形式に対応。
大きな画像ほどトークンを消費する #wwdc26
Richard Wei retweeted
If you missed our AI/ML Group Lab yesterday, we're doing another one on Friday June 12, 2026 9:00 AM – 10:00 AM PT. We'll discuss Core AI, Foundation Models framework, Evaluations framework and MLX. Join us!
developer.apple.com/wwdc26/s…
We wrapped up an amazing panel discussion on the latest in AI and Machine Learning at WWDC26. Thank you for all your questions about agents in Xcode, Core AI, Evaluations framework, Foundation Models framework, MLX. Here is the recording ICYMI.
youtube.com/live/bXb18GwYQS8…
