Roman Shemet retweeted
Presenting R² Index, a global leaderboard of the top 800k "de-sloped" AI researchers, along with papers, patents, institutions, founders, startups, investors and countries.
I designed the algorithm to filter job applicants without relying on fancy metrics like alma-matter, but it ended up rediscovering common beliefs, while including industry underdogs like Georgi (llama.cpp author) in the top percentile.
I explain the algorithm in this thread, but the full ranking is here: r2index.fyi
Roman Shemet retweeted
Whistle now supports live transcription streaming and long audio file transcription.
Repo: github.com/cactus-compute/ne…
Roman Shemet retweeted
We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash.
One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5.
Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess.
That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction.
It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser: cactuscompute.com/needle
Roman Shemet retweeted
Needle 2 was made to be fine tuned.
Show it your environment and some data samples - and the 14MB model beats DeepSeek v4 Flash.
Read more: cactuscompute.com/needle#fin…
Roman Shemet retweeted
We release Needle 2: A 14MB agentic LLM for phones, wearables, smart home, robots and microcontroller. The whole model is a single 14MB binary that runs a full session in 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine.
Needle 2 has 45m parameters trained from the ground up on 140B tool call, device use structured generation tokens. On mobile device use benchmarks, Needle 2 trades wins with frontier small LLMs like LFM2.5 230M, Apple FM Gemma-270m, at 5× to 70× smaller, and 2 bits against their f16.
Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, between 400–1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges 300–700 on sub-$200 phones such as the Samsung A-Series. Needle also runs on newer microcontrollers like ESP32.
A conventional transformer of Needle's width and depth spends 164 MFLOPs per token, and even one squeezed down to Needle's parameter count spends 87, Needle spends 70. Even on a high-end phone, an always-on assistant lives inside a power budget; every MFLOP is milliwatt-hours, and Needle spends 7x to 85x fewer of them per token than the smallest performant LLMs.
Read more: cactuscompute.com/needle
UPDATE: a new local LLM repository for ESP32 chips just hit the radar.
the Needle from Cactus Compute framework just dropped, and it completely solves the headache of running offline function-calling on hardware with only 512KB of SRAM.
because of this, we just pushed a massive update to the ESP32 Swarm Masterclass. instead of flashing raw LLM weights and hoping the output matches your macros, the guide now includes a full architecture drop for the new framework.
you can now define your physical room in strict JSON, finetune a 26M parameter model in 2 minutes, and deploy it to an $8 chip for guaranteed, offline function calling.
what is new in the masterclass:
- Step-by-step needle framework setup
- Memory optimization tricks for 512KB SRAM
- A new offline swarm deployment architecture
- Real-time latency benchmarks
the full guide and updated code are attached in the original article.
Roman Shemet retweeted
Hybrid models efficiently route complex tasks to larger cloud models, but how do you go about creating one? 🤔
Cactus solved this by post-training Gemma 4 to return a confidence score for every prompt. Process the prompt locally when confidence is high, route to a larger model when confidence is low. Routing only happens 15%-55% of time based on domain benchmarks.
Roman Shemet retweeted
Hackathon alert! London, SF, Boston. This Friday! 👀
@nothing is teaming up with @cactuscompute and @huggingface to hack on redefining on-device AI experiences!
Come build something memorable, meet the teams, and ship in 24 hours!
Signups are wild so far 🔥
Roman Shemet retweeted
The era of AI app subscriptions is coming to an end…
Thanks to the advent of lightweight edge AI models and frameworks like cactus - developed by @RomanShemet (backed by @ycombinator) AI apps can run 100% on the phone
My offline Plant ID app is proof:
👉 apps.apple.com/br/app/plant-…
No more cloud fees. No more monthly charges.
Just fast, private, one-time apps.
Roman Shemet retweeted
I've been talking to the Cactus folks about getting their local LLMs for mobile into dartantic. In the meantime, don't let that stop you from bringing them into your mobile Flutter apps: medium.com/@shemet0roman/how…
Roman Shemet retweeted
Cactus (cactuscompute.com) runs AI on smartphones. They help developers deploy local, private, and offline AI cross-platform in their apps.
Congrats on the launch, @RomanShemet & @Henry_Ndubuaku!
ycombinator.com/launches/Nwp…
Roman Shemet retweeted
cubic (@cubic_dev_) is Cursor for code review.
An AI-native platform that helps teams like Cal.com, n8n, and the Linux Foundation ship code 28% faster.
Congrats on the launch, @NeuralPaul and @allisonee2!
ycombinator.com/launches/NYz…





















