Pinned Tweet
I decided not to invest in a hardware upgrade. So I built MoEShift to run Qwen3.8 Flash Next on Macs with just 16–32 GB of RAM.
Turns out the bottleneck was software this time.
4
8
926
Here's MoEShift running behind @mfranz_on's Local Office!
You can create and work with documents using AI entirely on your Mac, without relying on cloud inference.
Here's a demo of the agent writing a doc about Python, with MoEShift handling inference locally.
In the video: Qwen 3.6 35B A3B, running locally via MoEShift on a 24GB M4 Pro, with a lot of memory free for other apps.
MoEShift: github.com/davide-troiani/mo…
Local Office: github.com/mfranzon/local-of…
4
93
Two MoEShift updates today 😊 You can now run models supported by stock llama.cpp alongside the optimized Qwen setup. Plus adjustable timeouts for longer requests.
Try it on your Mac and let me know how it goes!
github.com/davide-troiani/mo…
7
211
MoEShift currently supports Qwen 3.8 Flash Next and Qwen3.6-35B-A3B.
I’m looking into which MoE architecture to support next.
What model would you like to run on your Mac?
2
84
MoEShift just hit 100 GitHub stars! ⭐️
A few days ago I was testing it on my own Mac. Now others are running it on theirs and sharing benchmarks.
Really cool to see this happening. Thanks to everyone who’s tried it so far!
6
308
Worth trying. This will definitely need an efficient inference backend for lower end hardware!
Every office will have this Local AI tool soon: open a contract, type "make the payment terms more formal", and watch the edit appear in the file.
Same for spreadsheets, filled from your own invoices.
Local Office runs it all on your PC, offline with local models.
1
3
430
Yesterday I asked people to benchmark MoEShift on their Macs.
We now have results from M4, M5, M5 Pro and M6 machines running MoE models up to 125B parameters.
Really interesting to see how it performs across different hardware.
All results are public:
github.com/davide-troiani/mo…
If you have an Apple Silicon Mac, give MoEShift a try!
I’m collecting benchmarks across different chips, RAM sizes and workloads to see how it performs in the wild.
Good or bad, I’d love to see your results.
Setup and benchmark instructions:
github.com/davide-troiani/mo…
1
131
New independent benchmark of MoEShift running @UnslothAI’s Qwen 3.6 35B A3B Q6_K_XL (32GB on disk) on a 32GB M5 Macbook Pro.
Prompt processing 474 t/s
Generation 15 t/s
7.7 GB process memory footprint
2
95
MoEShift is open source, quick install and documentation: github.com/davide-troiani/mo…
1
22
I built MoEShift to run large MoE models on Macs with limited RAM.
On a 24GB M5 Pro, Qwen 3.8 Flash Next (125B): 374 tok/s prefill, 14.7 tok/s generation, 8.5GB peak process memory. Fast enough for chats and agents.
If you try it, share your numbers!
github.com/davide-troiani/mo…
💥 1250 亿参数的 Qwen3.8 Flash Next,塞进 24GB 的 Mac,内存只用 7.5GB。
专家从硬盘一块块读进来。长提示 63 秒跑完,另一种跑法要 130 秒。
温馨提示:作者在 M4 Pro 上自测,量化版;还是测试版,硬盘大约要空出 120GB。
来源:MoEShift / @iltroiani👇
github.com/davide-troiani/mo…
5
207
Spending thousands of dollars a month on tokens is no longer a flex.
The real competitive advantage is getting great results from non-premium models.
Less brute force, more engineering.
Even better when it’s local.
2
248
Qwen 3.8 Flash Next IQ3 XXS (78GB on disk) benchmarked on MoEShift with a 24GB M5 Pro:
380 t/s prefill
14 t/s decoding
8.5GB peak process memory
Pretty usable, ain’t it?
Get ready for Qwen4: github.com/davide-troiani/mo…


