Ramez Naam (@ramez) header
Ramez Naam (@ramez) profile picture

Climate and clean energy investor. Author of 5 books. Energy & Environment co-chair @SingularityU. Trying to build a better world.

Seattle
Joined May 2007
Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)Photo by Ramez Naam (@ramez)
Despite this election, I remain an optimist about America and the world. Humanity will continue to produce new ideas and new innovations to improve our lives. Good people will continue to come together to improve the world. And the political tide will turn. We'll make it so.
40
23
8
336
169,269
This is very good, on models, singularities, and the real world.
I recently moved to San Francisco to start a postdoc at Stanford, and I have quickly assimilated to the local culture by writing a 10,000 word essay about semi-endogenous growth models and the software singularity.
1
18
2,936
Ramez Naam retweeted
One of the few things harder than building a transmission line today is passing legislation to make it not quite as hard to build a transmission line tomorrow.
12
29
1
471
24,185
Ramez Naam retweeted
RSI is a process that's been happening at least since ~600,000 years ago
rsi is a process that’s been happening at least since the renaissance
29
58
4
810
44,602
Ramez Naam retweeted
The funny thing about the IEA is that most of the data from IEA modeling isn't publicly available unless one opens their wallet to cross their paywall, crossing the paywall only grants access to a modicum of additional data, and the modeling itself is hardly replicable by outside researchers, yet the IEA remains a very-accepted, common source to cite in academic publications (as I myself have often done).
The IEA now projects solar additions flat at 2025's level until 2040.
6
13
2
85
5,515
Ramez Naam retweeted
It simply has to be the Presidio.
NEW: California is bidding to host the United States Space Academy in the Bay Area, @CAGoBiz announced today. The state is one of many making their pitch to NASA, including Florida, Colorado and Virginia.
9
11
1
251
9,785
You gotta respect their commitment to the bit though...
The IEA now projects solar additions flat at 2025's level until 2040.
18
37
1
469
13,941
Alex is one of the top cyber security people in the industry so worth listening to him. I read him as both more pessimistic about the near and medium cyber future than many in his industry, and also pointing the path forward to an actually secure world (his last bullet).
No. 1. In the presence of superhuman bug finding machines, humans cannot write safe code without extreme levels of care and planning. 2. That care has almost never been applied to the critical systems in our lives. Not because software engineers don't care, but we aren't really engineers, not in the way civil engineers are engineers. The title is stolen valor. 3. Even careful, experienced "engineers" have difficulty writing code that is safe against modern models in C/C++ and other low-level languages. 4. We are finding that CVEs are overrated; huge numbers of organizations are being owned by custom vulns in the 100s of billions of lines of non-standard enterprise code out there or misconfigurations in heterogeneous systems, which LLMs can find and write exploits for live. 5. @dinodaizovi is right, we need architectures for which one RCE is not game-over and that buy us time for AI-driven response. 6. Everything is going to have to be re-written by AI, in type/memory safe languages, with minimal libraries, using formal methods, contracts, solvers, etc...
19
66
3
1,073
149,541
Ramez Naam retweeted
It is a truth universally acknowledged, that a single researcher in possession of good stock options, must be in want of opportunities for impact. Well, at Epoch we have more ambitious AI evaluation ideas than budget. If you'd like to fund some million-dollar experiments, get in touch!
1
16
129
4,829
Ramez Naam retweeted
The largest IPO in history at $81.34 billion is just a milestone for @AnthropicAI You may like or hate them, but they are on generational run Congratulations, and please make the most out of this, for all humans (and the lightcone!)
1
5
43
2,139
Ramez Naam retweeted
Young men in the midterms rejected MAGA 2.0 more dramatically than I've ever seen any major demographic turn against a group that had supposedly won them for life. I mean, these numbers are insane! I've never seen anything like this??
96
301
51
2,002
159,321
Hahaha. Hahahaha! HAHAHAHAHA!! Facepalm.
The IEA now projects solar additions flat at 2025's level until 2040.
31
22
3
390
19,712
Ramez Naam retweeted
It's always great to see quantitative predictions because they are falsifiable. Good to see @GoogleDeepMind doing this research. 1.5× more compute per dollar 2.4× more hardware spending ~3× better algorithmic efficiency per year = ~11× annually --> ~150,000× more effective compute in five years
2
5
28
2,503
After drawing level in September, India has now solidly surpassed the US in EV sales share in October. India: 6.5% battery electric United States: 6.0% battery electric The US will continue to fall behind India and others in this key technology if nothing changes.
26
110
17
585
32,514
Ramez Naam retweeted
AI safety is a marathon, not a sprint The views of the AI safety community are quickly converging on a fast takeoff within the next year. But this relies on a few low-quality data points from AI companies and contested models of the world. I worry that this convergence will degrade the epistemic environment and make good decisions about AI policy and deployment harder. Forget about predicting the future; it will be (and sometimes already is) hard to adjudicate claims about the present. It also pushes the community’s energy toward blunt instruments such as generalized bans on AI development, at the cost of decentralized, society-wide approaches that improve resilience. And it normalizes extraordinary government interventions and authoritarian actions. I hope that even people who are worried about imminent superintelligence and takeover risks can see that the epistemic homogeneity in the AI safety community is unhealthy. The community’s entire focus seems to be on one threat model, backed by contested data points, when it should be prepared for a range of scenarios. Five ways in which this shows up: 1) Updating heavily on limited evidence. Prominent commentators have shifted their views based on benchmark accuracy numbers. Of course, benchmarks have some signal. But anything that can be benchmarked is by definition verifiable, so it can be optimized for with reinforcement learning. Still, the community has coalesced around the view that these numbers point to superintelligence emerging in 2027. 2) Focusing analysis and interventions exclusively on AI companies. Little of the community’s analysis covers how AI is diffusing, or how institutions outside the AI industry are adapting. So almost every incident is seen as a story about model capabilities. The recent outages at Microsoft, United, and the NSA, and the FAA ground stop, all saw vagueposting about the latest cyberattack by an advanced AI model; none of them was the result of an AI cyberattack. The NSA’s outage came from patching vulnerabilities that were found with AI tools, which is a consequence of AI diffusion. But this is not nearly as compelling as the AI cyberattack story, so it didn't get a lot of attention. 3) Ignoring institution- and capacity-building in favor of blunt (and potentially authoritarian) interventions. In response to concerns around imminent superintelligence, many of the policy recommendations rely on extraordinary government action that veers into authoritarianism. Most are generalized bans on AI training; few build the institutional capacity to absorb AI’s impacts across society. But the cost of building frontier models is decreasing rapidly, so even if we were somehow able to ban AI development today, these bans wouldn't be effective at preventing societal risks. They will also leave us worse prepared for when a capable model is openly released, or an adversarial nation state develops one, since our institutions will not be ready for the fallout. 4) Adopting a “soldier mindset” instead of forming independent views. One dynamic I have observed firsthand is that many people in the safety community take the word of prominent commentators as gospel and do not question their claims or methods, which compounds these problems. Even the debates happen within the community’s frame, such as its reliance on subjective probability estimates for AI’s existential risks. So the community is homogeneous in its methods as well as its views. 5) Treating AI safety as a sprint. To be clear, I think AI capabilities will continue to increase over the next few years, and we need to dramatically improve our responses to AI risks. But that means preparing society to absorb a transformative technology, which goes well beyond heading off the current worry about an intelligence explosion. Remaking our institutions is a decades-long process; we can’t solve it by pushing AI back in the box. My worry is that today’s views lead us into a perpetual state of unproductive alarm. Even if “superintelligence in 2027” doesn’t materialize, AI progress won’t stop, so I expect the community will keep its narrow focus on AI development. A far more productive outcome would be to update its causal models of the world and treat resilience, institutions, and diffusion more seriously.
15
34
6
184
24,532
Ramez Naam retweeted
Solar is now China's 2nd largest source of electricity. Coal used to be the primary driver of growth, but now solar dominates. Total generation has continued to skyrocket while coal has flatlined (down ~2% from the peak).
36
160
21
837
54,943
OpenAI has released data on AI research acceleration that's far more detailed, granular, and longitudinal than Anthropic. And in this data we don't see anything like runaway super-intelligence. I respect the METR team who went into Anthropic quite a bit. But without this level of data transparency, it's hard to be convinced that Anthropic really is seeing an AI-driven acceleration. To be clear, the METR team had their hands tied by Anthropic, who redacted the clearest evidence. The data Anthropic has released and allowed METR to release is insufficient for independent conclusions. Anthropic says that they expect to hit the early stages of RSI (and pass one of their safety thresholds) in a forthcoming model. Given such a statement, the responsible thing for Anthropic to do is to release data that matches OpenAI's in granularity and over a similar span of time, so anyone can independently analyze it to understand the capabilities of Anthropic internal models and their trajectory.
Seems like the OpenAI research acceleration blogpost is not what you would expect if we were definitely on the cusp of artificial superintelligence - models that are more cognitively capable than ~ALL humans on ~ALL tasks. Agent workdays have grown very jaggedly, swarm sizes haven't grown very much, and spending on tokens hasn't actually outpaced spending on human workers* although the growth rate is clear. There also hasn't been an acceleration in research experiments per experimenter, or in the internal ECI scores of OpenAI's models. Perhaps most importantly (and most surprisingly), the task success rate for <15 min tasks hasn't grown very much at all, and the share of tasks done by agents across various time horizons is both low and not growing for the most part. It's possible that Anthropic is *much* further along in automating AI R&D than OpenAI, and that METR saw this in their investigation. It's also possible that we see nothing much right up until a shocking acceleration in capabilities, like we saw in mathematics. We certainly can't rule out the possibility of an intelligence explosion. But there's been a huge belief cascade in AI in general and AI safety in particular that seems largely unwarranted given the evidence that we have today. It's entirely possible that we can't fully automate all AI R&D tasks in the imminent future, or that full automation of AI R&D doesn't massively accelerate AI capabilities progress, or that capability gains concentrate in easily verifiable tasks and fail to generalize to the messy real world. These are all open research questions
18
39
3
337
33,322
Ramez Naam retweeted
It's great to see OpenAI publishing so much useful information about AI R&D automation. I hope Anthropic does the same soon
We're sharing new data on how AI is changing research at OpenAI. We find researchers' AI use continues to rise exponentially, and agents are completing longer tasks, but human involvement remains central in research. Researchers are also spending more time on safety and alignment, with human effort in these areas growing faster than in any other individual research area. This report aims to increase transparency around research acceleration and inform the governance of increasingly capable AI. We believe that AI labs should be required to track and publicly report indicators of recursive self-improvement (RSI): the process by which AI accelerates the development of future AI systems. Our hope is that this expanded report will help sustain and advance shared measurement norms across the industry. openai.com/index/research-ac…
5
4
1
113
4,850
Great graph. Each increment of AI progress is harder than the last, showing typically power law or log scale dimishing returns. That's been shown as a function of compute, data, money, and now looks like it applies to lines of code.
Replying to @tszzl
one maybe less interesting fact is that the utility of each line of code seems to be going down drastically in terms of creating ECI deltas
6
8
63
5,642
Ramez Naam retweeted
There is a common belief in the AI field—that advanced AI means the inevitable forfeiture of human primacy in the world. Even many who disfavor that future believe it is likely. I call this the silent consensus, and it must not be allowed to remain silent any longer.
140
222
152
1,889
391,252