Pinned Tweet
It's so tragic that universal love is all you need and it's already right there but you also need to proactively defend against those who don't believe either of those claims to be true.
wife: labor pain really wasn't what I expected
me: how so
wife: like... It was definitely painful, but it would probably be equally painful if you got shot in the leg, you know?
3
10
350
no, I don't know and it sounds just about as horrifying as it did before
but I have now had multiple moments where I thought I could die for this kid, so I guess that's spiritually similar? @chiaragerosa
7
56
In Rotterdam, @MaximeStauffer @MopTheTerritory @chiaragerosa and I lost our minds over his stuff. In particular this hyperreal painting of NYC from 2016. A masterpiece, without hyperbole. Just sublime. vimeo.com/163882531
@nandabeniniii thank you for reminding me of this!!
3
81
I’ve read this a couple times now, and while there is, frankly, a lot of sloppy argumentation (which I will get to), the charitable, simplified read of Dwarkesh's position seems to be:
1) AI needs major breakthroughs to achieve AGI/ASI (continual learning, sample efficiency, etc.) — this is a view he’s held for some time now, in line with Ilya
2) These breakthroughs, and the ML research skills required to obtain them are verifiable, and thus current, pre-breakthrough AIs can hill-climb on these sorts of problems and capabilities to solve them (like we saw in math)
3) Once these breakthroughs occur, we *may* see rapid, recursively compounding gains to intelligence, leading to ASI in a very short period of time
There's a ton of other stuff in the essay, but I think this is the basic crux. Christiano basically articulated this same argument in his Dwarkesh appearance, and if you look at AI-2027, it essentially follows the same trajectory: Agent 2/3 makes a ton of algorithmic progress, allowing Agent 4/5 to explode.
In general, I think the high-level narrative here is...fine? It’s certainly one plausible outcome. But, imo, it is a much narrower path than I think many are giving it credit for—a lot of things actually have to go just right for this to play out. And here's where I return to the other, messier arguments, because many of the intuitions and arguments driving Dwarkesh's view in favor of near-term ASI (e.g., EO2027) as a highly probability outcome (~50%) are, imo, quite bad:
- “[We'd see] a jump as big as GPT-3.5 to GPT-6 Astra in just 12 months” -> No, under this scenario we’ll see ~4 years of AI progress in 1 year. If there are diminishing returns, then this won’t be as extreme as the jump we saw from GPT-3 to GPT-6. It very well could be, but that would require innovation to overcome the diminishing returns we'd expect in the base case.
- “The other big piece of evidence for transfer is human evolution” -> Except if AI was following the sort of pattern we saw for human evolution, we should have seen better transfer long before AI started solving millennium problems. This spikiness has been one of the big surprises of AI!
- “Rather, my claim is that scaling up RL training will give AIs the abilities necessary to ramp quickly in new contexts…it’ll go through an evolutionary process that involves centuries of corporate competition to figure out the optimal organizational techniques.” -> These are in direct conflict! If ramping requires RL runs that simulate centuries of competition within “digital economy full of millions of other AIs” then that’s quite bearish for ramp speed (relatively) given the compute and wall-clock requirements!
- “I have a lot of 20-something-year-old friends whom I’ve watched accomplish incredible things and build world-leading companies and ventures in domains they entered into with no previous experience.” -> lol. rip leopold. But also, again, these are humans, with human sample efficiency. If AI gets there, sure. But how soon it gets there is the big crux.
- “Obviously, some pieces of feedback—about how your actions impacted the quarterly earnings, say—require months of serial time. But even granting that, we should expect these future AIs to be able to get to top human performance in almost all domains within months of being deployed.” -> …no that’s not how that works at all because one set of earnings isn't enough to robustly learn from and you can’t re-run the same real-world conditions.
- “[Sample efficiency & online learning are] AI’s equivalent of a Millennium Prize problem. And mathematics has already shown us what happens to such problems” -> Sure, but there’s a difference between “AI can solve a Millennium-level problem” and “AI can solve *this specific Millennium-level problem*” -- if the blocker is a specific problem, the timing spread becomes much wider.
- “My answer to this [experiment/wall clock/compute] objection is that each agent will still have access to a ton of compute. Frontier labs have been more than quadrupling their compute every single year.” -> But that’s what they’ve needed to do keep up with their current pace! Even if the labs maintain their current pace of compute growth (and like, will they even? capital markets really do not seem accommodating and Dwarkesh's roommate just lowered his compute projections), we'd still need a step change in pace above that!
And that's not even covering the biggest crux of all: whether the returns to intelligence continue to recursively compound faster than diminishing returns eat into them.
Now, are any of these objections fatal? No. Do they “disprove” hard RSI/an intelligence explosion? No, ofc not. Am I kind of nitpicking here? Yeah, 100%.
And to Dwarkesh's credit, he does anticipate and acknowledge a scenario (the “spark”) where these objections play out.
But the reason I've bothered to rant this long is that I see a lot of AI/RSI discourse follow this pattern: where 50% of the arguments are reasonable and grounded and appropriately hedged, and then the other 50% are these sorts of sloppy intuition pumps and what-ifs that include a ton of detail but a noticeable lack of rigor. And I can’t help but get the sense that it is the latter 50% that is really driving people’s predictions and expectations here, as it leads to sentences like this:
“In the spark scenario…it takes years to get to the same endpoint that the software-only singularity would have achieved in months: a superintelligence that is to us what we are to chimps.”
Note that Dwarkesh thinks these two outcomes (spark vs. software-only singularity) are equally likely. 50/50. I’m sorry, but that’s absolutely nuts. Even Christiano put the likelihood of ASI by in 2027 *absent any slowing down* at only 1 in 3, and I still think that's too bullish.
Like, yes, I fully anticipate that we will have automated AI researchers (or something close to them) in 2027, and the universe where we go from that to a software-only singularity is not entirely out of the question. But obtaining that universe relies on a confluence of various frictions, diminishing returns, and wall-clocks being overcome, a somewhat specific set of algorithmic innovations being made or obviated, and an assortment of other factors playing out just-so to enable that perfect storm. Even a fairly rapid exponential doesn't qualify when you're talking about full-blown ASI in 13 months. It may not be impossible, but it's certainly not a 50/50 outcome.
konrad retweeted
I don't know if you actually need legislation to get real commitments out of these companies. But you definitely need the threat of legislation. You need them to believe that the alternative to doing this is MORE transparency and oversight.
There's so much reason to be so hopeful for humanity. We have come so far in so little time. It's hard to think about how it could have gone much better much faster.
I have believed this for a very long time. But the first two days with my daughter have made it even clearer.
It's so tragic that universal love is all you need and it's already right there but you also need to proactively defend against those who don't believe either of those claims to be true.
18
475
konrad retweeted
New EA cause area: flood the Bay Area with babies
Works in Progress' @RuxandraTeslo says Silicon Valley's willingness to hand humanity's future over to AI may stem from how few people there have children:
"A lot of people think about this only from the perspective of their own lives, not about the future."
Could sb pls explain to me how much we can expect @infomaniak to contribute to Bulles heating costs per capita with this data center, and how you would expect that to scale if one were to build data centers 1, 2, and 3 orders of magnitude larger than that? @grok
1
82
While I broadly subscribe to the vibes of this message (i.e. think that people are overestimating the promises of transhumanism), I have to point out that, theoretically, supernatural/religious belief could be considered transhumanist. Another horse shoe Peter Thiel entertains in exemplary fashion.
Medicine is on the gradient of transhumanism.
(by these definitions, yes, I'm a transhumanist)
Today, the promises of transhumanism and posthumanism are multiplying. These are ideological currents according to which, sooner or later, technology will eliminate all disease and pain, as well as aging and even death itself. Precisely because scientific progress is a supreme achievement of human intelligence, it must not be confused with a privilege reserved for millionaires seeking immortality, while entire peoples suffer from poverty and oppression. In the face of this mirage, which promotes the illusion of human beings perfected by cybernetics and biotechnology, Christian anthropology reminds us that a limitation is not a manufacturing defect, but rather the very condition of human grandeur. #ApostolicJourney #Peru #MagnificaHumanitas vatican.va/content/leo-xiv/e…
2
3
238
The panic vibes around first the hugging face incident and now the ai r&d acceleration report are really confusing to me.
Either, a bunch of people who've been making accurate predictions are being fake dramatic, hoping it doesn't look like being dishonest to pretend to update drastically in light of roughly on par events OR they didn't take their forecasts seriously themselves OR they feel powerless and desperate (or any combination of these).
I feel you guys. But the panic vibes are really not helping us. Ground your nervous systems and get to work, there are lots of people looking for support right now who have woken up to the dangers of the game we're in. The last thing they need is more advocacy noise. If you cannot access any such people, consider that you might be best off building technical solutions. Otherwise, help people understand the game we're in to defeat the odds.
But telling decision-makers the odds has approximately 0 marginal value. And maybe more importantly, will backfire hard whenever you're wrong. And you will be wrong. That's fine. Nobody cares if you can make yourself concretely useful. It's not most people's job to be right, that's the job of a tiny sliver of humanity.
2
16
799
konrad retweeted
Introducing Claude Psychiatry: add Claude to your group’s signal chat or discord and it builds detailed psychological profiles, diagnoses, and dispenses medication via select partners while you’re still talking.
Starting to roll out in beta on Team and Enterprise plans.
konrad retweeted
Replying to @politicalmath
my mute list probably has thousands of entries at this point
i credit my enjoyment of this platform to muting freely
konrad retweeted
Was funny to watch everyone in Tech finally, collectively, realize tech's PR was awful and that the general public was gonna eat Tech alive in short order if it wasn't helped.
So, days later, every big Tech figure on X tweets simultaneously "It's okay for the powerful tech CEO to have affairs and do whatever he wants, accountability is corruption"
Like, yes, of course, Jensen leaving Nvidia is bad, BUT you people are really really not helping the broader issue facing the entire ecosystem.
It's almost like tech subconsciously wants to be crucified
can estrogen save jensen...
NVIDIA announces CEO transition. Read the release: nvidianews.nvidia.com/news/n…





























