Roman Shemet (@RomanShemet) header
Roman Shemet (@RomanShemet) profile picture

Founder @ Cactus Compute (YC S25)

San Francisco, CA
Joined January 2022
Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)Photo by Roman Shemet (@RomanShemet)
Presenting R² Index, a global leaderboard of the top 800k "de-sloped" AI researchers, along with papers, patents, institutions, founders, startups, investors and countries. I designed the algorithm to filter job applicants without relying on fancy metrics like alma-matter, but it ended up rediscovering common beliefs, while including industry underdogs like Georgi (llama.cpp author) in the top percentile. I explain the algorithm in this thread, but the full ranking is here: r2index.fyi
6
7
1
64
397,011
Roman Shemet retweeted
Whistle now supports live transcription streaming and long audio file transcription. Repo: github.com/cactus-compute/ne…
12
13
7
174
173,502
Roman Shemet retweeted
We release Needle 3: A Sliceable 8-29MB automation foundation model that can match DeepSeek V4 Flash. One set of weights, every depth from 2 to 20 layers a model of its own, 25-121M parameters at CQ2-bit, built on our Simple Attention Networks and running locally at up to 4k tokens/sec decode speed on a Raspberry Pi 5. Needle does not chat. Every turn is a function call: give it the tools your app exposes and it picks the right ones and fills every argument from what the user said, or hand it a schema and it returns a typed record. Ask for something no tool covers and you get an empty list, not a guess. That trade is lets 121M parameters trained on 360B tokens of structured data beat models 10x their size on mobile tool calls and match 2-3x bigger models on structured JSON extraction. It runs on mobiles, wearables, smart home devices, small robots and microcontrollers, with prebuilt engines for macOS, Linux, Windows, Android, iOS, watchOS, tvOS, the browser and WASI hosts. Try it in your browser: cactuscompute.com/needle
106
267
117
2,912
600,573
Roman Shemet retweeted
Needle 2 was made to be fine tuned. Show it your environment and some data samples - and the 14MB model beats DeepSeek v4 Flash. Read more: cactuscompute.com/needle#fin…
8
13
5
276
13,790
Roman Shemet retweeted
We release Needle 2: A 14MB agentic LLM for phones, wearables, smart home, robots and microcontroller. The whole model is a single 14MB binary that runs a full session in 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. Needle 2 has 45m parameters trained from the ground up on 140B tool call, device use structured generation tokens. On mobile device use benchmarks, Needle 2 trades wins with frontier small LLMs like LFM2.5 230M, Apple FM Gemma-270m, at 5× to 70× smaller, and 2 bits against their f16. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, between 400–1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges 300–700 on sub-$200 phones such as the Samsung A-Series. Needle also runs on newer microcontrollers like ESP32. A conventional transformer of Needle's width and depth spends 164 MFLOPs per token, and even one squeezed down to Needle's parameter count spends 87, Needle spends 70. Even on a high-end phone, an always-on assistant lives inside a power budget; every MFLOP is milliwatt-hours, and Needle spends 7x to 85x fewer of them per token than the smallest performant LLMs. Read more: cactuscompute.com/needle
101
245
61
2,412
294,805
Roman Shemet retweeted
UPDATE: a new local LLM repository for ESP32 chips just hit the radar. the Needle from Cactus Compute framework just dropped, and it completely solves the headache of running offline function-calling on hardware with only 512KB of SRAM. because of this, we just pushed a massive update to the ESP32 Swarm Masterclass. instead of flashing raw LLM weights and hoping the output matches your macros, the guide now includes a full architecture drop for the new framework. you can now define your physical room in strict JSON, finetune a 26M parameter model in 2 minutes, and deploy it to an $8 chip for guaranteed, offline function calling. what is new in the masterclass: - Step-by-step needle framework setup - Memory optimization tricks for 512KB SRAM - A new offline swarm deployment architecture - Real-time latency benchmarks the full guide and updated code are attached in the original article.
finally! local AI is accessible to everyone. we replaced our massive smart home server with a single $8 ESP32-S3 chip processing voice commands entirely on-device. now anyone can build this. here is the full guide:
6
7
2
67
6,955
Roman Shemet retweeted
Hybrid models efficiently route complex tasks to larger cloud models, but how do you go about creating one? 🤔 Cactus solved this by post-training Gemma 4 to return a confidence score for every prompt. Process the prompt locally when confidence is high, route to a larger model when confidence is low. Routing only happens 15%-55% of time based on domain benchmarks.
14
36
3
356
25,284
Roman Shemet retweeted
Check out Cactus AI!
This quoted post is unavailable.
2
1
18
5,758
Roman Shemet retweeted
Hackathon alert! London, SF, Boston. This Friday! 👀 @nothing is teaming up with @cactuscompute and @huggingface to hack on redefining on-device AI experiences! Come build something memorable, meet the teams, and ship in 24 hours! Signups are wild so far 🔥
9
19
2
200
48,739
The era of AI app subscriptions is coming to an end… Thanks to the advent of lightweight edge AI models and frameworks like cactus - developed by @RomanShemet (backed by @ycombinator) AI apps can run 100% on the phone My offline Plant ID app is proof: 👉 apps.apple.com/br/app/plant-… No more cloud fees. No more monthly charges. Just fast, private, one-time apps.
2
2
6
373
Roman Shemet retweeted
I've been talking to the Cactus folks about getting their local LLMs for mobile into dartantic. In the meantime, don't let that stop you from bringing them into your mobile Flutter apps: medium.com/@shemet0roman/how…
2
3
1
30
2,997
Cactus (cactuscompute.com) runs AI on smartphones. They help developers deploy local, private, and offline AI cross-platform in their apps. Congrats on the launch, @RomanShemet & @Henry_Ndubuaku! ycombinator.com/launches/Nwp…
15
9
3
111
15,462
cubic (@cubic_dev_) is Cursor for code review. An AI-native platform that helps teams like Cal.com, n8n, and the Linux Foundation ship code 28% faster. Congrats on the launch, @NeuralPaul and @allisonee2! ycombinator.com/launches/NYz…
25
39
9
409
51,914
Roman Shemet retweeted
The sweet sweet moment you find product market fit (wait for it)
317
916
274
9,768
1,435,151
How do you monopolize any industry? According to Mukesh Ambani, Asia's richest man, you give away $25 billion. His competitors laughed... Until he bankrupted 11 companies, created $100 billion & disrupted India's economy. This is history's most ruthless takeover: 🧵
204
1,659
133
12,972
2,539,889