Karan Brar (@deepmatmul) profile picture

ceo/co-founder of autoresearch, inc. | prev ml enjoyer @reductoai @dynamo_ai, math enjoyer @uoft

San Francisco, CA
Joined May 2025
Photo by Karan Brar (@deepmatmul)Photo by Karan Brar (@deepmatmul)Photo by Karan Brar (@deepmatmul)Photo by Karan Brar (@deepmatmul)Photo by Karan Brar (@deepmatmul)Photo by Karan Brar (@deepmatmul)Photo by Karan Brar (@deepmatmul)
Karan Brar retweeted
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario). He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails. He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it. And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion. Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
166
201
141
1,743
491,011
this is too real. the age old question, do i have psychosis, or do the normies?
i can't talk to friends or family about ASI and what might happen next year without sounding like a chronically online person lost in sci-fi fantasies. it's increasingly frustrating that trying to be rational about what's happening sounds delusional to normies. like i'm the one detached from reality when, from where i'm standing, they're the ones missing what's unfolding right in front of them. the gap in perception is getting so absurd it's making me feel like I am the schizo who needs to touch grass but im just trying to inform them because the media doesnt
1
1
3
284
There lies his crown in waters deep, till Waleed finally gets his PhD (and joins @autoresearchinc)
3
4
242
About as much as every other field is cooked. I’m not a fan of these takes because it implies that math is somehow over. If anything, we’re entering the golden age of the industrialization of mathematics.
5
379
lol what a gross post
Welcome Xi! He's joining us from @OpenAI after 18 years as a Professor of Physics at Harvard 🐐 As AI eats mathematics, the best theoretical physicists seem to be reorienting towards bridging theory and experiment. This is much harder to hill-climb, but incredibly impactful!
6
13
1
435
83,035
We’ll certainly have a lot more exotic optimization algorithms that combine classical techniques + LLM driven optimization. WSP is only the beginning.
I’ll take the other side of this bet. If one understands any reinforcement learning, it should be clear why weight space editing is a last mile touch up and not a backprop or heck even ZO opt replacement
2
7
569
Karan Brar retweeted
pro tip: don't punch down on a 1000x smaller competitor for not updating their comparison page, it's not the flex you think it is
pro tip: don’t make shit up about the competition and then complain when you get caught
16
3
3
246
37,094
I found all the people saying negative things about Sutton and quoting bitter lesson to be quite surprising. It's as if they completely forgot how basically everyone was talking (including Dwarkesh Patel with Sutton on his podcast) when pretraining only was the dominant paradigm (pre o1 etc) and everyone thought just keep scaling and you'll get PhD level math models with just pretraining and then reasoning models came and completely shattered that ceiling becoming a great step up in capabilities. Sutton is 100% right (as he usually is) here btw and if you don't see it you have major skill issue and don't even understand bitter lesson properly
Brute force computation is not enough for intelligence. Clever new methods are needed. The science of AI is far from finished. IMHO
7
6
113
7,939
Karan Brar retweeted
The vibe type theory explosion is bound to occur. Axiom and others will realize their systems can make new type theories that are more expressive than the hand crafted ones we use.
2
3
1
44
2,134
Karan Brar retweeted
Post-train a formal theorem proving model to reach frontier performance to push the frontier of math 🔥🔥🔥
Replying to @lmsysorg
An in-house training stack for mathematical formalization. Tobias Gessler from @axiommathai shares how the team approaches building training infrastructure for this specialized domain.
2
5
23
2,647
should we do it
bad business idea: abliterated ai model inference provider but you report your customers to the FBI
8
441
Karan Brar retweeted
What if a swarm of AI agents could learn to work together on its own? 🐝 No leader. No roles. No turn order. Just one shared reward. Our new paper s³: multi-agent RL lets LLM swarms self-organize, beat best-of-N, and scale past their training size! scalingswarms.com/ 🧵
30
59
19
342
41,904
Still curious if formalization of mathematics will yield any gains in terms of our ability for discovery, or it’s mostly just to accomplish the autistic dream of a formally verified encyclopedia of mathematics. Most mathematicians I’ve spoken to see this as a vanity project with no real applications, but the rest is to be seen.
Using the Stacks formalization we built at Axiom, we formalized the Weil conjectures in a single day. With the foundations in place, formalizing research-level algebraic geometry is now within reach for anyone working in the field.
18
4
1
21
9,229
Using the Stacks formalization we built at Axiom, we formalized the Weil conjectures in a single day. With the foundations in place, formalizing research-level algebraic geometry is now within reach for anyone working in the field.
We shared our formalization of Stacks. Is it reusable? The true test of a foundational library is what you can build on it. Now, we release the Lean formalization of the proof of the renowned Weil conjectures (I-IV), one of which gives Riemann Hypothesis over finite fields. ✨
1
4
1
17
3,976
Karan Brar retweeted
The mask has slipped. The heart of the pro-AI movement is contempt for humanity. Not progress. Not 'Abundance'. Just frustration that ordinary Americans simply refuse to vote the way that AI devs and economics professors think they should.
Human disempowerment now. It's time for the age of the folly and cruelty of human politics to end. The median AI is so far above the median human voter in both knowledge and decency it's embarrassing. Humanity should just take the L.
55
169
4
1,546
30,575
re: weight-space programming (WSP) craziness… just tried something that blew my mind.. i asked the question: is Astra + WSP a better optimizer than BP on simple extrapolation learning on any turing computable function.. (let alone interpolation).. so far my answer is yes.. just tried a bunch of functions like: x^2 sin(x) x^x tanh(x) add sub rolling_sum sort fib WSP gets to beyond machine precision in many cases for interp and extrap.. (exactly matched every tested 64-bit reference output) (DISCLAIMER: very prelim results so far mind you..) and BP does not. WSP beats BP on every task I gave it.. iso-architecture/iso-param/iso-initialization/iso-inputs.. I have been stuck on the following problem for 12 years... if UAT is true, why cant we find a decent root in the loss landscape.. its bc BP sucks.. WSP is the first optimizer that actually can in a lot of cases. this is a whole new area of optimization burgeoning..
15
20
1
203
9,897
Karan Brar retweeted
"I am prepared to risk murdering innocent children for a chance of immortality"
Without ai, we are all going to die. But there is a chance, with Superintelligence, that I get to live forever. I am therefore prepared to risk everything.
79
21
301
9,360
stfu already man you’ve been shitting yourself the past 48 hours we get it you hate the dems and love Elon you don’t need to continue virtue signalling
This is the nightmare scenario they want. Total AI Dystopian 1984. We must fight this tooth and nail. The resistance begins now.
1
10
528