Making @kaptionai - among other things.
Austin, TX
Joined November 2008
- Tweets12.8K
- Following389
- Followers932
- Likes20.9K
1/ Sam Altman argued this week that AI running on everyone's own computer is a misallocation of the world's limited memory and chips, which do more good shared in data centers.
I checked the numbers on a real, free AI model. Here it is in plain English. 🧵
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
Something we can agree on!
Except for audio… that is 100% solved.
sam: local inference is a terrible allocation of scarce resources
“a lot of people want local inference, and somehow they assume that the problem is solved now because we have open-source models, we're going to be able to run them on our computers, and so we're not going to need all the stuff in the data centre
but from a global allocation perspective, actually it's quite terrible to have everybody, like, have a bunch of the world's limited supply of DRAM sitting in their own computer and being used five percent of the time
you want maximum utilisation in the cloud if you're trying to deal with this very supply-constrained environment
somehow many people's intuition is the opposite of that”
There’s 2 types of people in the world…
Pro FSD/SelfDriving and dumbasses like this one.
COLUMN | The first and worst part of the week was being invited to sit in a Tesla Cybercab, a vehicle – I can’t call it a car – with no steering wheel and no pedals, painted in Trump Gold, which is likely to be running a Tesla Robotaxi service on Sydney’s streets by this time next year. Stephen Corby has more: bit.ly/3WpKfIt
Readers added context they thought people might want to know
Tesla’s Cybercab is gold as a futuristic NYC yellow cab, per designer Franz von Holzhausen (2024). Musk called it a “golden era of transport,” with no Trump link.
x.com/elonmusk/statu…
teslamagz.com/tesla/robotaxi…
This
Railway solucionaba la velocidad
Ya no
Loco, ya están grandes para seguir usando esas plataformas como Railway para deployar su web, ahora que tienen AIs que hacen absolutamente todo saquen un VPS (DigitalOcean, Hetzner, Vultr) donde quieran, le pasan la IP del VPS a su AI y le dicen configurame todo y levantame este proyecto ahi.
Listo, se ahorraron mucha plata por mes, dolores de cabeza con filtraciones y caidas.
Cristian Pereyra retweeted
Well, we just rotated every single key across a large number of projects very quickly, and I'm struck by the fact that this would have been an absolute nightmare not too long ago
The defensive aspect of AI for cybersecurity is really underrated
Where the nuclear weapon analogy breaks down with AI is that you can't stop a nuclear weapon with another nuclear weapon
But you can stop an AI attack with an AI
