CEO & Co-Founder | Entrepreneur (1 exit), Investor (2 exits), Advisor and Speaker on AI. I help companies and entrepreneurs adopt AI to grow and become cognitiv
Rome, Italy
Joined September 2008
- Tweets28.7K
- Following3.1K
- Followers16.5K
- Likes17.4K
AI agents shouldn't get full powers. #Plenipo is now open source (Apache 2.0): you approve a mandate from your #EUDIWallet with scope, spend limits and expiry, and any service verifies it before the agent acts.
Revoke it anytime. github.com/plenipo-net/pleni… #AIAgents
v0.4, experimental: tested end to end with a real walt.id wallet, not yet for real money. Try it live, or connect Claude to the sandbox as an MCP server: plenipo.net/demo #OpenSource #AgenticAI #DigitalIdentity
Fabio Lalli retweeted
@mudler_it al #linuxdaybenevento ci parla di #LocalAI. Seguici in diretta su #YouTube
youtube.com/live/WuT9p0uu-tE…
#linuxday #AI #opensource
Fabio Lalli retweeted
▲ ~/𝚛𝚊𝚞𝚌𝚑𝚐/𝚘𝚜𝚜-𝚐𝚛𝚊𝚗𝚝𝚜/ 𝚕𝚜▐ (v2)
I'm announcing v2 of my open source grants of $1,000USD to 35 awesome contributors. The themes:
① Agent skills & tools
Skills are now a valuable form of software, like Emil's world-class design eng work. Projects like 𝚙𝚐𝚋𝚘𝚝, 𝚔𝚗𝚒𝚙, 𝚞𝚗𝚕𝚒𝚐𝚑𝚝𝚑𝚘𝚞𝚜𝚎 create amazing AI engineering loops.
② Local AI
Projects like 𝚜𝚕𝚘𝚝𝚜𝚝𝚛𝚎𝚊𝚖, 𝙻𝚘𝚌𝚊𝚕𝙰𝙸, and 𝚌𝚘𝚕𝚒𝚋𝚛𝚒 are making inference not just a demo but for many a real and growing utility. Exciting to watch local AI unfold.
③ Performance
There's no excuse for software to be slow anymore. Projects like 𝚏𝚏𝚏 and 𝚝𝚊𝚔𝚞𝚖𝚒 leading with speed, compilers like 𝙿𝚎𝚛𝚛𝚢𝚃𝚂 and 𝚙𝚘𝚛𝚏𝚏𝚘𝚛 that can help us upgrade entire ecosystems to native, and 𝚐𝚙𝚞𝚒𝚡 pushing the GPU rendering speed frontier.
④ High quality foundations
AI is still at its best when it can reuse great components than regenerating every time from first principles. Lots of love to folks working on @shadcn, Svelte(kit), Nuxt, Node.js and security foundations.
⑤ Cool experiments
I couldn't resist donating to gloom.sh, a 'Finance terminal in the terminal', or beautiful funky things like 𝚍𝚒𝚝𝚑𝚎𝚛-𝚔𝚒𝚝.
🔗 rauchg-oss-grants.vercel.app…
PS: These are personal, not affiliated with Vercel, and fully unconditional. If you are listed, I'll be DM'ing you or using your public contact info!
Fabio Lalli retweeted
Thanks @AMD 🫶
They have sent a box over... so this means you are gonna get first class @AMD support in vllm.cpp, @LocalAI_API and all our projects.
It was very painful before, as we had to work only with logs and reported issues, expects things to improve a lot of you have one of these!
Fabio Lalli retweeted
A 370 GiB model just answered me on a machine with 119 GiB of RAM (and it doesn't even need that much).
@Alibaba_Qwen's Qwen3.8-2.4T, 2.4 trillion parameters, the UD-Q1_0 checkpoint from @UnslothAI, running on one DGX Spark with vllm.cpp.
It works because the expert weights never get loaded. They stay in the GGUF on disk and each token reads only the slices its router picks. 62 GiB ends up resident.
It takes 11.05 seconds per token using CPU only. The CUDA path is roughly twice as fast. Seconds, per token. I don't care. A frontier model runs on my desk, offline, no strings attached.
Link's in the replies
Fabio Lalli retweeted
a 118B model on one DGX Spark, in 4-bit, and vllm.cpp is ahead of vLLM on it
44.46 tok/s against vLLM's 43.10 on @poolsideai's Laguna-S-2.1. same output, token for token. default config, nothing to switch on
And most importantly: no python in the process. no pytorch. just one 66 MiB binary and a portable C ABI you can hook onto
and it's not just this model. qwen3.6-27b matches or beats vLLM at every concurrency we ran, cpu prefill is 1.18x llama.cpp on the same gguf, and we're 1.14x the fastest gguf engine we could find on deepseek-v4-flash!
vulkan is next, since that's what most of you asked for. after that the board is open, what goes on it?
Fabio Lalli retweeted
Il corpo? È la nuova interfaccia del potere hi-tech
🖊️Su @startup_italia, un estratto del libro di @fabiolalli: tinyurl.com/StartupItaliaPel…
📖“Pelle digitale – Come l’intelligenza invisibile sta riscrivendo la realtà, il corpo e la società”: tinyurl.com/PelleDigitaleEge…
Boom
been writing a C++20 port of vLLM for a while now, and was quite silent on it, because I don't like to talk until I have results. So I was in "cave mode". Name TBD. Until then it's vllm.cpp
paged KV cache, continuous batching, prefix caching, no python at inference, reads GGUF natively
today it beat the reference engine on a 300B model running on one DGX Spark, so I guess it's becoming real
Fabio Lalli retweeted
📚 Scopriamo la "Pelle digitale" all'AI WEEK
📌Mercoledì alle 12.45, a Rho Fiera Milano, l’appuntamento con @fabiolalli: cutt.ly/WtBjxrWT
📖Il libro, qui: tinyurl.com/PelleDigitale
Fabio Lalli retweeted
rf-detr.cpp: native C++/ggml (@ggml_org ) inference for @roboflow 's RF-DETR (my go-to for object detection and segmentation!) from the @LocalAI_API team.
All 11 variants (5 detection + 6 segmentation), running at PyTorch speed (slightly, ~8% faster on CPU benchmarks), without Python dependencies at f16.
Models available in @huggingface 🤗
Thanks to @ggerganov for ggml and @SkalskiP for rf-detr to make this possible!
🧵
Fabio Lalli retweeted
44 GGUFs on @huggingface under mudler/rfdetr-cpp-*. Packaged also as a @LocalAI_API backend: `local-ai backends install rfdetr-cpp`, then just POST
/v1/detection API :) Apache-2.0.
Code: github.com/mudler/rf-detr.cp…
Models: huggingface.co/mudler
Fabio Lalli retweeted
LocalAI ( @LocalAI_API ) 4.2.0 is out, just few numbers and facts:
- +392 commits ( we squash these 😄 )
- +11 Backends: voice and face recognition, vibevoice.cpp (from me), LocalQVE from @jichiep and among @sgl_project , @__tinygrad__ , @no_stp_on_snek 's Turboquant, ik_llama.cpp, sam.cpp from @el_PA_B
- Many new QoL improvements, increased sglang and VLLM support and hardening on distributed mode
- 16+ new contributors ! Thanks to the community!
LocalAI is all about give you flexibility to run the latest from the community, and ds4 support from @antirez is on its way!
This is the year of Local AI!
Fabio Lalli retweeted
Say hello to vibevoice.cpp, @Microsoft 's Vibevoice in pure C++ with @ggerganov 's ggml (@ggml_org).
TTS and ASR (with diarization). CPU + CUDA + Metal + Vulkan via ggml backends. Quantized models live on @huggingface.
Built with ❤️ from the @LocalAI_API team
github.com/mudler/vibevoice.…
Fabio Lalli retweeted
🟣 Appuntamento venerdì 10 aprile alle 18:00 presso lo Spazio Attivo di Lazio Innova a Valle Faul #Viterbo per la presentazione dell’ultimo volume di @fabiolalli edito da @egeaonline
#medioera
Fabio Lalli retweeted
📚Pelle digitale: come l’intelligenza invisibile sta riscrivendo la realtà, il corpo e la società
📖Il nuovo saggio di @fabiolalli, ora disponibile in libreria e online, qui: tinyurl.com/PelleDigitale
Ci sono momenti storici come quello che viviamo, che, mentre li attraversi, continuano a sembrare perfettamente normali. La loro forza non sta nell’annuncio e nemmeno nell’hype che generano. Sta nel ritardo con cui vengono percepiti.
fabiolalli.substack.com/p/la…
Fabio Lalli retweeted
A call to open source maintainers. How I Built a 100% Local Autonomous Dev Team to help maintain LocalAI, and why you should too
Spinning up an autonomous dev team is no longer sci-fi. It is here, and it works 100% locally. Minimax really changed the game.
👇link to my blog
Fabio Lalli retweeted
📚 La mente adattiva: pensare insieme alle macchine
📲 Pubblicato nella collana interamente digitale di Egea – Bookmark – il libro di @fabiolalli è disponibile online, qui: tinyurl.com/LaMenteAdattivaB…
