OpenBMB (Open Lab for Big Model Base) aims to build foundation models and systems towards AGI. Connect with us: https://xitter.cf/t.co/N9pevTnoOa
Joined February 2022
- Tweets2K
- Following394
- Followers12.8K
- Likes3.1K
OpenBMB retweeted
A 35B vision-language model running locally on a phone with about 3GB of memory.
OpenBMB reports MiniCPM-V 4.7 reaching 13+ tokens/sec on an iPhone 17 Pro Max.
It’s a MoE with roughly 3B active parameters per token. Expert weights stream from storage as needed.
Lookahead Expert Scheduling predicts which experts the next layer will need and preloads them while the current layer computes. If a prediction misses, the required experts still get loaded and evaluated.
That works alongside layer-wise quantization and DSpark speculative decoding.
Open-source, with on-device deployment support for iOS, Android and HarmonyOS.
The clip below is their iPad M4 demo, recorded directly on the device.
OpenBMB retweeted
💥 MiniCPM-V 4.7 35B-A3B just dropped, and we put it through five real tests on an H100
📱 Open-source MoE built for on-device use, now running on a local GPU
🔹 Setup and serving on a single H100
🔹 Coding: a chimney cake fire simulation
🔹 SQL: find the hidden bug and fix it
🔹 Vision: a handwritten note in three languages
🔹 Physics: reasoning over a page of equations
🔹 Translation: one Brazilian headline into 80 languages
🔥 Full test and honest verdict in the video below 👇
youtu.be/zFJiwyhDFrA?si=3DJY…
@OpenBMB
OpenBMB retweeted
We hope to build an inference framework whose optimizations can be explained both when they work and when changing conditions make them fail. It is also the kind of engineering practice I hope more of us will undertake together in SGLang Omni.
OpenBMB retweeted
🖥️ 上头了，AI Jarvis 这个开源桌宠会一直看着你的屏幕，你打游戏它在旁边搭话，你上网课它顺手把笔记记了
它跑的是 OpenBMB 的 MiniCPM-o 4.5 多模态模型，语言、视觉、音频三块全在你自己机器上算。安装包里连编译好的推理运行时都带着，不用自己去装 Python、CMake 或者 CUDA。
以前边看网课边记笔记是这样的：讲到关键那页要暂停、截图、切到笔记软件、打字，一节课来回切十几次，还总有几页没截着。AI Jarvis 的课程模式是它每秒看一帧画面、听最近一秒的系统声音，自己挑出关键画面，整理成知识点和画面说明，你从头到尾不用动手。
平时它就在桌面上待着，一个快捷键叫出来说话；打游戏的时候只在旁边发弹幕提示，不会替你操作。日常采集的原始画面和声音默认不长期保存，不想被看的时候有隐私模式能一键暂停感知。
目前只认 64 位的 Windows 10/11，第一次启动要联网下 6.32 GiB 模型，之后就是纯本地了，显卡不够会自动退回 CPU 跑。MIT 协议。
助理这东西，能自己看见你在干嘛，才算真的省事。
GitHub：github.com/LYiHub/pub-local-…
OpenBMB retweeted
New article: how accurate are local models vs Jev, a cloud System One classifier?
I ran 8 on a Mac, same JevBench questions. A 4B model matched Jev on short texts. On complex ones, all were 17+ points behind.
rockyshikoku.medium.com/jev-…
Video: minicpm5-2b classifying messages.
OpenBMB retweeted
Everyone is posting 3 and 4 Spark clusters this week. Here is what ONE DGX Spark does for one user, all measured on my box:
MiniCPM5-2B, OpenBMB's drafter on: 100.8 tok/s
Qwen3.6-35B-A3B: 89.7
Ling-3.0-flash: 69.5
Qwen3.8-Flash-Next: 55.5
DeepSeek-V4-Flash: 47.9 in EXL3
Gemma-4-E2B: 39.2
Qwen3.8-27B: 33.4 EXL3 + DFlash2
Tinfield-1 at 2-bit: 30.7
One model at a time, 256 tokens in, 256 out
OpenBMB retweeted
2B 参数的小模型，现在也开始能干活了？
OpenBMB 的 MiniCPM5-2B，主打本地小硬件运行。有人把它接进 CI 修复 Agent，丢了一个“结算重试导致运费重复计算”的 Bug 给它。
模型自己复现问题、翻代码、定位原因，最后直接打补丁，18 个测试全部通过，而且全程本地运行。
几个亮点：
① 编程和工具调用能力不错
② 2B 参数，对本地设备更友好
③ 权重和训练数据都开源
小模型不一定只能聊天，拿来跑本地 Agent 也开始有点意思了。
🔗 huggingface.co/openbmb/MiniC…
OpenBMB retweeted
The first demo showed 32 local agents generating text at once. This time I gave them tools and put GPT-6 Astra in charge.
MiniCPM5-2B handled the workers on my DGX Spark. Astra coordinated through Codex CLI.
Their job: restore 32 services in a simulated city blackout using shared, limited repair supplies.
Then a second failure closed a bridge and knocked six restored services offline.
You can see tools reject actions, workers get blocked, and Astra send more fuel before assigning another round of repairs. It chose to leave the bridge closed and use the alternate route.
232 actual tool calls. 19 rejected actions. All 32 services verified online in 51.5 seconds.
This is the setup I wanted to test: a stronger model coordinating small local models that can act and check their work.
Full live recording below. 30 fps, no speedup.
OpenBMB retweeted
open source is on fire this week! for some weeks, i've been playing with very small LLM models in my free time (gemma e4b, qwen 4b, etc.) just to see what “small” stuff these LLMs can do.
I got access to this 2 billion dense param model MiniCPM5-2B, and this thing is a beast for its size. 128k context window and better than every model in the same size range running fully locally on my macbook pro m5 (via llama.cpp)
OpenBMB retweeted
Introducing JustRL II 🚀
Building on JustRL (xitter.cf/HBX_hbx/status/1988474…), we took a closer look at how GRPO behaves in long-CoT RL (128k). The group-mean baseline is a great fit for short traces, but over tens of thousands of tokens it gives a coarse, response-level signal.
JustRL II keeps GRPO's group structure and adds a critic for token-level credit assignment. Just as simple, keeps improving where GRPO levels off: AIME25 61→81 on a 2B model.
The same recipe powers the RL stage of MiniCPM5-2B (huggingface.co/openbmb/MiniC…), making it SOTA among models under 4B.
Data + checkpoints are open. Code lands this week. 📖 panhaoxuan.notion.site/justr…
✨What if the simplest RL recipe is all you need?
Introducing JustRL: new SOTA among 1.5B reasoning models with 2× less compute.
Stable improvement over 4,000+ steps. No multi-stage pipelines. No dynamic schedules. Just simple RL at scale.
📄 Blog: relieved-cafe-fe1.notion.sit…
OpenBMB retweeted
GGUF quantization may be getting a size slider of sorts with a new FIT-GGUF quants!
Normally, when downloading a local model, you choose a quant level
Q3 → Q4 → Q5 → Q6
…and hope it fits your RAM/VRAM budget.
FIT-GGUF flips that around.
Give it a target size or fidelity tier, and it can mix precision tensor-by-tensor to find a GGUF that meets that target.
OpenBMB highlighted it on MiniCPM5-2B:
📦 Quality — 1.46 GiB
⚖️ Balanced — 1.28 GiB
🗜️ Compact — 1.21 GiB
🤏 Mini — 1.14 GiB
Those labels represent progressively looser fidelity to the BF16 model, measured primarily with KL divergence and not different skills like coding vs. reasoning.
And they run in normal llama.cpp / LM Studio.
⚠️ FIT does not claim its tensor allocation is universally optimal yet.
🔗 Link in ALT
ALT https://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/tree/main
OpenBMB retweeted
What can a 2B open-source model actually do on your phone?
MiniCPM5-2B from OpenBMB is 2B params, #1 under 4B on the Artificial Analysis Intelligence Index, and tops their Agentic Index outright, 20 vs 9 for the next small model (official). That's the Densing Law in action: model capability density doubles roughly every 3.5 months, and this one fits on an iPhone.
The thing I want from a model that size is an iMessage agent: reads my texts, makes reminders and events, looks things up, replies for me, nothing leaves the phone. So I tested it. Six tools, eight real-world scenarios, three runs each. My results, not official ones.
Worked:
"remind me to send Jordan the invoice tomorrow at 9am" → correct reminder, 6/6.
Delta confirmation text → clean JSON of flight, times, seat, 6/6, ~1.5s.
"is the 6 train running Saturday?" → searched, read the result, replied correctly: take the 4 or D.
"book it and tell her yes" → calendar event plus reply to Maya, 2/3.
Didn't:
"dinner thursday 7" became Sept 14, not the 10th, in most runs. Once it reasoned the right date and then wrote the wrong one anyway.
Decided Lucali is in San Francisco. It's in Brooklyn.
Thinking mode ran away: 4,096 tokens on a Swift date parser, no code, three times.
HuggingfaceRepo:huggingface.openbmb.cn/model…
OpenBMB retweeted
OpenBMB MiniCPM-V4.6 é um dos sinais mais claros de que a IA multimodal está ficando pronta para edge AI.
~1.3B parâmetros.
Roda localmente em hardware de consumo.
E ainda lida muito bem com OCR, dashboards, gráficos e compreensão de documentos.
Testei no iPhone + Mac. 🧵
🤝 Paid partnership
OpenBMB retweeted
我们期待一起建设的推理框架，能够说清楚优化为什么有效，条件改变后又为什么失效。把一个反常结果追到可以解释、可以复现，再把这些认识写进代码和测试里，本身就是很扎实的系统工作，也正是我希望更多朋友在 SGLang Omni 中共同完成的工程训练。
OpenBMB retweeted
I wanted to see how useful a small local model could be on my older computer, so I gave @OpenBMB’s MiniCPM5-2B a simple job.
The machine: an Intel i5-9400F with 16GB of RAM, running Linux.
I built MiniCPM News Desk around something I already spend time doing: keeping up with AI and tech news. ☕
On my machine, it’s connected to an hourly official source collector and configured to send one Telegram recap covering the previous 24 hours.
A lead story, other meaningful updates, and quick hits with links to the original sources.
Enough information to understand what happened and whether it matters to me. If I want more detail, I can open the article.
The code handles collection, filtering, and delivery.
MiniCPM selects useful passages from supported articles, while the system preserves important dates, requirements, and limitations. Smaller updates use clearly labeled publisher excerpts or headline links.
A small, practical project using hardware I already own.
I started with AI and tech because that’s what I follow.
The same idea could be adapted to other interests.
Here’s what the Telegram output looks like:
OpenBMB retweeted
🏥 This is the best reason for a tiny Local AI model.
⚕️ OpenBMB and OpenMed now have a open weight healthcare workflow built around MiniCPM5-2B.
🩺 OpenMed is a whole local-first clinical AI stack:
🔒 PHI / PII detection + de-identification
🧬 Clinical entity extraction
📋 FHIR + HL7 workflows
🔧 MCP + agent tools
📱 Apple / Android / browser support
🌐 33 model-backed PII languages
📚 2,266 model entries
After the required models are downloaded, its core processing can stay on hardware you control and even air-gapped.
Then MiniCPM5-2B becomes the agent brain.
And this is tiny
🧠 2.52B total parameters
⚡ 1.98B non-embedding
📚 131K context
Yet OpenBMB specifically trained it for agents:
🤖 500K agent-training samples
🔧 native tool calling
🧠 long-context reasoning
💻 coding + search agents
And you can actually run it locally ...
✅ GGUF / llama.cpp
✅ Ollama
✅ LM Studio
✅ MLX 4-bit
✅ GPTQ 4-bit
This is why I think small Local AI models may become incredibly important.
You don't necessarily need a 70B model reading every sensitive medical record.
You may need a 2B local agent sitting next to the data, calling specialized tools and models that each do one thing extremely well.
Keep the private data local. Bring the intelligence to the data. 👀
⚠️ Local processing and de-identification can improve privacy, but OpenMed explicitly says using the SDK alone does not establish HIPAA compliance.
🔗 GH: /maziyarpanahi/openmed
🔗 HF: /openbmb/MiniCPM5-2B
OpenBMB retweeted
This is my MacBook Air M2 (16GB) with Hermes Agent powered by MiniCPM5-2B. 96k context, and a very respectable output of 20-24 tok/s.
This is totally usable, and MiniCPM5 can do all the tool calls to manage the basics with Hermes in a personal agent context.
I wouldn't use it to build the next Salesforce, but for the home enthusiast this is a powerful and fast local model that can really punch above its weight, and runs on almost anything. Look at it go.. Amazing!
