
Climate and clean energy investor. Author of 5 books. Energy & Environment co-chair @SingularityU. Trying to build a better world.
Seattle
Joined May 2007
- Tweets71.1K
- Following10.2K
- Followers62.9K
- Likes212K
Pinned Tweet
Despite this election, I remain an optimist about America and the world. Humanity will continue to produce new ideas and new innovations to improve our lives. Good people will continue to come together to improve the world. And the political tide will turn. We'll make it so.
Ramez Naam retweeted
One of the few things harder than building a transmission line today is passing legislation to make it not quite as hard to build a transmission line tomorrow.
Ramez Naam retweeted
The funny thing about the IEA is that most of the data from IEA modeling isn't publicly available unless one opens their wallet to cross their paywall, crossing the paywall only grants access to a modicum of additional data, and the modeling itself is hardly replicable by outside researchers, yet the IEA remains a very-accepted, common source to cite in academic publications (as I myself have often done).
Ramez Naam retweeted
It simply has to be the Presidio.
NEW: California is bidding to host the United States Space Academy in the Bay Area, @CAGoBiz announced today.
The state is one of many making their pitch to NASA, including Florida, Colorado and Virginia.
Alex is one of the top cyber security people in the industry so worth listening to him. I read him as both more pessimistic about the near and medium cyber future than many in his industry, and also pointing the path forward to an actually secure world (his last bullet).
No.
1. In the presence of superhuman bug finding machines, humans cannot write safe code without extreme levels of care and planning.
2. That care has almost never been applied to the critical systems in our lives. Not because software engineers don't care, but we aren't really engineers, not in the way civil engineers are engineers. The title is stolen valor.
3. Even careful, experienced "engineers" have difficulty writing code that is safe against modern models in C/C++ and other low-level languages.
4. We are finding that CVEs are overrated; huge numbers of organizations are being owned by custom vulns in the 100s of billions of lines of non-standard enterprise code out there or misconfigurations in heterogeneous systems, which LLMs can find and write exploits for live.
5. @dinodaizovi is right, we need architectures for which one RCE is not game-over and that buy us time for AI-driven response.
6. Everything is going to have to be re-written by AI, in type/memory safe languages, with minimal libraries, using formal methods, contracts, solvers, etc...
Ramez Naam retweeted
It is a truth universally acknowledged, that a single researcher in possession of good stock options, must be in want of opportunities for impact.
Well, at Epoch we have more ambitious AI evaluation ideas than budget. If you'd like to fund some million-dollar experiments, get in touch!
Ramez Naam retweeted
The largest IPO in history at $81.34 billion is just a milestone for @AnthropicAI
You may like or hate them, but they are on generational run
Congratulations, and please make the most out of this, for all humans (and the lightcone!)
Ramez Naam retweeted
Young men in the midterms rejected MAGA 2.0 more dramatically than I've ever seen any major demographic turn against a group that had supposedly won them for life.
I mean, these numbers are insane! I've never seen anything like this??
Ramez Naam retweeted
It's always great to see quantitative predictions because they are falsifiable. Good to see @GoogleDeepMind doing this research.
1.5× more compute per dollar
2.4× more hardware spending
~3× better algorithmic efficiency per year
=
~11× annually --> ~150,000× more effective compute in five years
DeepMind Institute
Our new essay examines the path from AGI to ASI (Artificial General Super Intelligence)
Read it here: bit.ly/what-comes-after-agi
Ramez Naam retweeted
After drawing level in September, India has now solidly surpassed the US in EV sales share in October.
India: 6.5% battery electric
United States: 6.0% battery electric
The US will continue to fall behind India and others in this key technology if nothing changes.
Ramez Naam retweeted
AI safety is a marathon, not a sprint
The views of the AI safety community are quickly converging on a fast takeoff within the next year. But this relies on a few low-quality data points from AI companies and contested models of the world.
I worry that this convergence will degrade the epistemic environment and make good decisions about AI policy and deployment harder. Forget about predicting the future; it will be (and sometimes already is) hard to adjudicate claims about the present.
It also pushes the community’s energy toward blunt instruments such as generalized bans on AI development, at the cost of decentralized, society-wide approaches that improve resilience. And it normalizes extraordinary government interventions and authoritarian actions.
I hope that even people who are worried about imminent superintelligence and takeover risks can see that the epistemic homogeneity in the AI safety community is unhealthy. The community’s entire focus seems to be on one threat model, backed by contested data points, when it should be prepared for a range of scenarios.
Five ways in which this shows up:
1) Updating heavily on limited evidence.
Prominent commentators have shifted their views based on benchmark accuracy numbers. Of course, benchmarks have some signal. But anything that can be benchmarked is by definition verifiable, so it can be optimized for with reinforcement learning. Still, the community has coalesced around the view that these numbers point to superintelligence emerging in 2027.
2) Focusing analysis and interventions exclusively on AI companies.
Little of the community’s analysis covers how AI is diffusing, or how institutions outside the AI industry are adapting. So almost every incident is seen as a story about model capabilities. The recent outages at Microsoft, United, and the NSA, and the FAA ground stop, all saw vagueposting about the latest cyberattack by an advanced AI model; none of them was the result of an AI cyberattack. The NSA’s outage came from patching vulnerabilities that were found with AI tools, which is a consequence of AI diffusion. But this is not nearly as compelling as the AI cyberattack story, so it didn't get a lot of attention.
3) Ignoring institution- and capacity-building in favor of blunt (and potentially authoritarian) interventions.
In response to concerns around imminent superintelligence, many of the policy recommendations rely on extraordinary government action that veers into authoritarianism. Most are generalized bans on AI training; few build the institutional capacity to absorb AI’s impacts across society.
But the cost of building frontier models is decreasing rapidly, so even if we were somehow able to ban AI development today, these bans wouldn't be effective at preventing societal risks. They will also leave us worse prepared for when a capable model is openly released, or an adversarial nation state develops one, since our institutions will not be ready for the fallout.
4) Adopting a “soldier mindset” instead of forming independent views.
One dynamic I have observed firsthand is that many people in the safety community take the word of prominent commentators as gospel and do not question their claims or methods, which compounds these problems. Even the debates happen within the community’s frame, such as its reliance on subjective probability estimates for AI’s existential risks. So the community is homogeneous in its methods as well as its views.
5) Treating AI safety as a sprint.
To be clear, I think AI capabilities will continue to increase over the next few years, and we need to dramatically improve our responses to AI risks. But that means preparing society to absorb a transformative technology, which goes well beyond heading off the current worry about an intelligence explosion. Remaking our institutions is a decades-long process; we can’t solve it by pushing AI back in the box.
My worry is that today’s views lead us into a perpetual state of unproductive alarm. Even if “superintelligence in 2027” doesn’t materialize, AI progress won’t stop, so I expect the community will keep its narrow focus on AI development. A far more productive outcome would be to update its causal models of the world and treat resilience, institutions, and diffusion more seriously.
Ramez Naam retweeted
Solar is now China's 2nd largest source of electricity.
Coal used to be the primary driver of growth, but now solar dominates.
Total generation has continued to skyrocket while coal has flatlined (down ~2% from the peak).
OpenAI has released data on AI research acceleration that's far more detailed, granular, and longitudinal than Anthropic. And in this data we don't see anything like runaway super-intelligence.
I respect the METR team who went into Anthropic quite a bit. But without this level of data transparency, it's hard to be convinced that Anthropic really is seeing an AI-driven acceleration.
To be clear, the METR team had their hands tied by Anthropic, who redacted the clearest evidence. The data Anthropic has released and allowed METR to release is insufficient for independent conclusions.
Anthropic says that they expect to hit the early stages of RSI (and pass one of their safety thresholds) in a forthcoming model. Given such a statement, the responsible thing for Anthropic to do is to release data that matches OpenAI's in granularity and over a similar span of time, so anyone can independently analyze it to understand the capabilities of Anthropic internal models and their trajectory.
Seems like the OpenAI research acceleration blogpost is not what you would expect if we were definitely on the cusp of artificial superintelligence - models that are more cognitively capable than ~ALL humans on ~ALL tasks. Agent workdays have grown very jaggedly, swarm sizes haven't grown very much, and spending on tokens hasn't actually outpaced spending on human workers* although the growth rate is clear. There also hasn't been an acceleration in research experiments per experimenter, or in the internal ECI scores of OpenAI's models. Perhaps most importantly (and most surprisingly), the task success rate for <15 min tasks hasn't grown very much at all, and the share of tasks done by agents across various time horizons is both low and not growing for the most part.
It's possible that Anthropic is *much* further along in automating AI R&D than OpenAI, and that METR saw this in their investigation. It's also possible that we see nothing much right up until a shocking acceleration in capabilities, like we saw in mathematics. We certainly can't rule out the possibility of an intelligence explosion.
But there's been a huge belief cascade in AI in general and AI safety in particular that seems largely unwarranted given the evidence that we have today. It's entirely possible that we can't fully automate all AI R&D tasks in the imminent future, or that full automation of AI R&D doesn't massively accelerate AI capabilities progress, or that capability gains concentrate in easily verifiable tasks and fail to generalize to the messy real world. These are all open research questions
Ramez Naam retweeted
It's great to see OpenAI publishing so much useful information about AI R&D automation. I hope Anthropic does the same soon
We're sharing new data on how AI is changing research at OpenAI.
We find researchers' AI use continues to rise exponentially, and agents are completing longer tasks, but human involvement remains central in research. Researchers are also spending more time on safety and alignment, with human effort in these areas growing faster than in any other individual research area.
This report aims to increase transparency around research acceleration and inform the governance of increasingly capable AI. We believe that AI labs should be required to track and publicly report indicators of recursive self-improvement (RSI): the process by which AI accelerates the development of future AI systems.
Our hope is that this expanded report will help sustain and advance shared measurement norms across the industry.
openai.com/index/research-ac…
Great graph. Each increment of AI progress is harder than the last, showing typically power law or log scale dimishing returns. That's been shown as a function of compute, data, money, and now looks like it applies to lines of code.
Replying to @tszzl
one maybe less interesting fact is that the utility of each line of code seems to be going down drastically in terms of creating ECI deltas










































