Here's a quick installation video for Edge0-145B. Thought it might help anyone trying to get the model running.
Github: github.com/Edge0-AI/Edge0
1
9
288
A robot doesn't need to write a 2,000-word essay while avoiding a chair.
Smart glasses don't need to run a massive model just to understand “turn up the volume.”
And your phone doesn't need a cloud-sized setup for every AI task.
We talk a lot about how big a model can run on a device.
Maybe we should talk more about what that device actually needs the model to do.
2
8
755
Airplane mode has always meant giving up a few things.
Local AI doesn't have to be one of them.
Once Edge0-35B is downloaded and set up, you can disconnect from the internet and keep chatting with the model.
A small change in your settings. A pretty big change in how AI works.
1
5
451
AI that works in airplane mode?
Welcome to on-device AI: the model runs on your phone or computer.
Inference can work offline, avoid cloud round trips, and keep your data on the device.
Here’s where those benefits matter.
1
1
329
Your data doesn't have to leave your device for AI to work.
With local AI, your prompts, files, and model outputs can stay on your own device.
No need to send every request to someone else's servers.
And you don't need expensive hardware to do it.
3
5
225
Stop paying per token for AI tasks you can run locally.
Hosted AI APIs charge for the tokens your model reads and generates. With local AI, your own hardware does the work.
And no, you don't need an expensive workstation to get started.
Here's how.
1
2
168
As @rauchg said: "Own your intelligence." We agree.
That includes running AI locally, choosing which model your application uses, and deciding when to update it.
At @Edge0_AI, our 35B and 8B model weights and inference code are available under Apache 2.0. You can download a model, run it locally, and inspect or modify the runtime.
Your application can call that deployment through a local API. You can keep a tested version running while evaluating updates, and use hosted models alongside it.
1
3
221
1/6 Edge0’s Mac Metal engine decodes at 23–25 tok/s on Apple M5 with 16 GB RAM, while keeping its expert bank on SSD.
How do you keep a 35B MoE moving when a needed expert’s weights may still be on storage?
Our Prerouter predicts expert selections one token ahead, giving the engine advance notice of which weights to prepare. 🧵
1/8 why run ai locally when a cloud api is just one call away?
you get more control over your data, fewer network dependencies, and no per-token api fees.
the catch? your device has to handle the computing.
here's when that trade-off makes sense.
Your quantized model fits in memory. It even loads successfully.
Then you send the first prompt, and the runtime crashes with an out-of-memory error.
Why does a model that fits on paper still run out of memory?
Because model weight size is not peak runtime memory. The runtime needs memory for much more than weights.
1/ your phone can run an ai model without sending every prompt to a server.
that's on-device ai. the model runs on your phone or laptop, and can work offline.
but your device now has to do the computing itself.











