open source inference || prev @MSFTResearch
Joined May 2026
- Tweets361
- Following224
- Followers30
- Likes1.7K
Pinned Tweet
new account here :) im an inference engineer and researcher and will be posting about high performance inference systems. if that interests you, do consider a follow :)
deep learning inference in libraries like tensort is so unoptimized that it virtually guarantees that if you want to host any model outside of an LLM you would have to so some sorts of kernel engineering to get to real production standards
spent around a $100 to get to top 100 worldwide on the meta OKC challenge a couple of days ago, stopping now incase someone wants to sponsor GPUs for me
pythongiant retweeted
every time i hit #1 on kernel leaderboard my macos opens queen and plays we are the champions
here we go!
We're launching a $100,000 challenge to see who can make Muse Spark run fastest: openkernelchallenge.dev/
it feels very nice to know that im top 100 in the world at something
We're launching a $100,000 challenge to see who can make Muse Spark run fastest: openkernelchallenge.dev/
pythongiant retweeted
Jensen Huang holds up a Vera Rubin node on stage: what took two hours to assemble on Grace Blackwell now takes under five minutes, with no cables and 100% liquid cooling
“So Vera Rubin is in full production. It’s nice to kind of see it. This is the first computer in the world that we’ve created. No cables, no wires, completely 100% liquid cooled. And it’s just really delightful...Really delightful to be able to pull it out. And this is one Vera Rubin computer. Okay, so this has two GPUs."
"What used to take, in the case of Grace Blackwell, two hours to assemble one node, and when you see it, you’ll know what I mean. Cables and wires everywhere, 80% liquid cooled."
"Now Vera Rubin is 100% liquid cooled, and it takes less than five minutes to assemble it. And so, incredible, incredible achievement."
"Of course, this translates to higher cycle times in manufacturing, much, much better reliability, and of course, a lot better performance.”
Nebius CEO Arkady Volozh says GPU compute should be auctioned like internet ads or taxi rides. Nebius launched a GPU auction yesterday, “and the prices went up”
“And pricing is one of the components which helps you to overcome all these bumps in the road. The pricing should be market driven. On the markets where supply and demand are not balanced yet, you need to introduce something like an auctioning system."
"Something which actually we had in our previous lives experience, like internet advertising or all these taxi markets or delivery markets, where supply and demand should be balanced through some mechanism."
"I think on this market, on the infrastructure market, there should be some mechanism of balancing supply and demand, should be automated, and everything should be built around it."
"And that’s what we started to build. We just launched it. This is the first small sector where we launched it. It’s so-called preemptible capacity, very easy to get and to give up. And we launched it yesterday, and we got the first traction, the prices went up.”
the lack of context is crazy here. its 70 USD per hour and you can only work for one hour a week lmfao
also tcgen05 MMA is asynchronous
a single mode throughout the kernel reduces the number of execution protocols that must be reconciled.