Zac Hill (@zdch) header
Zac Hill (@zdch) profile picture

Head of Advocacy, MIRI. Contributing editor, Persona. Serial cross-partisan social/civic entrepreneur & executive. Professional Wizard in a previous life.

Washington, DC
Joined January 2009
Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)Photo by Zac Hill (@zdch)
If you're in AI policy in DC, or if you know about AI policy and you can move to DC, you should strongly consider trying to work as a congressional staffer. Senior people should consider doing this as well as junior people, if it's the right role in the right office. Many, many offices are staffing up and looking to hire AI people. We all have a duty to lock in and get serious and solve the problems we need to solve. If you do this and work hard and accept the sadly inadequate pay, you will be an American hero who is on the front lines of society's response to the most important challenge humanity has ever overcome. It is hard for me to put into words how much better of a thing that is to do than going to work for a lab, and it's also a lot better than the kind of cushy think tank gig that I currently have. Working for a state government in an important role implementing and enforcing the AI regulations that we actually have or will have is equally important.
26
76
11
665
49,757
Tari Eason on what allowed him to have a season-high 25 points off the bench tonight: "Shit, the Wizards, for real."
245
1,198
561
17,100
1,701,794
Zac Hill retweeted
Imagine a historian in 2035 writing a political history of the 2020s, and he points out that every election, the out-of-power party won a cost-of-living election and then used their mandate to ignore or exacerbate affordability only to be destroyed in the next election, where voters again said “to be clear, I don’t want whatever bullshit you’re up to, I just want life to feel more affordable,” and then, once again, the winning politicians prioritized everything except affordability, and this story repeated itself over and over again. One of my strongest held political opinions right now is: Democrats should do everything they can to avoid that history being written.
222
705
91
7,484
536,940
so refreshing to be in 1980s... —no email!!! —spy sex intrigue —fun games with interest rates —weird war non war with iran
2
4
1
17
4,487
A small cabal of self serving, ultra rich tech titans should not be allowed to drive humanity off a cliff with artificial super intelligence, while they quietly build their rockets and bunkers in New Zealand and Hawaii to escape the apocalypse. Stand with us and DO NOT ACCEPT this future.
AI companies admit what they’re building might kill everyone. We don't have to agree to this. Add your name at donotaccept.ai to say: we do not accept your terms.
117
885
28
2,884
103,789
Anthropic will really be like "you believe in slowing down? that pales in effectiveness to my strategy, racing so fast that we have credibility to push for strong regulation" and then not push for strong regulation
Worth reading. Dario says that one of the ways that Anthropic can justify remaining at the frontier (with the associated risks) is because the company is effective at pushing for strong AI regulation. I think Anthropic's track record here is more complex than he implies.
5
45
2
535
23,091
I often disagree with Neil, so I want to flag here that I STRONGLY agree with him: we desperately need a definition of RSI that has gone through a bunch of rounds of lawyers kicking its tires. You're a law student, want to get cited for forever? Write this as your journal note.
Replying to @woodilicious
That’s not a definition that works in legislation.
4
9
3
175
7,565
Many people feel like they can't "do AI policy" unless they live in DC or the Bay Area. Not so! You know what we need desperately right now? A bunch of people who care about AI policy in every state in the country, ESPECIALLY the small ones.
CALL YOUR CONGRESSMAN. Tell them to get ASI banned immediately. Don't let your families die because you were worried about the potential awkwardness of a phone call.
1
4
1
55
1,790
obviously, today's Democratic victories can be solely attributed to @ezraklein and @DKThomp writing the book "Abundance"
13
37
5
960
27,728
Cheesecake Factory implies the existence of Cheesecake Industrial Revolution implies the existence of cheesecake alienation implies the existence of Cheesecake Romanticism
31
220
34
2,122
86,533
Zac Hill retweeted
“They mobilized a whole of government response to my joke. Not to inflation. To a joke.”
OBAMA: The president wasn't even mad. About The Saddam Hussein comparison. That part didn't bother him. But he was very upset about The Cheesecake Factory. He took great umbrage. He said I didn't understand the work of the artisans in the White House. They mobilized a whole of government response to that joke. Not to inflation, but to that joke.
18
78
4
1,215
49,569
Replying to @zdch
Me: Democrats nominating candidates like the pastor at MLK's church against a domestic abusing former football player with a fake police badge (who still only lost by 1.5%) is why they're competitive in the Senate Republicans: James Talarico is a pedophile
1
1
60
1,090
Zac Hill retweeted
DC is facing a jobs crisis. Rushing the legalization of autonomous vehicles is irresponsible given the anticipated AI-related job displacement and disruptions that put countless workers at risk.
2
21
1
390
10,419
It seems to me that we're at "it's time to panic about AI". I gather that a lot of researchers are worried that we may be about to die, on a timescale of months, but this still isn't quite in the Overton window so we're still soft-pedaling. (Where "panic", of course, means "quickly and soberly initiate talks with the CCP to achieve a moratorium on large training runs, backed by a deterrence regime, compute and supply chain monitoring, and site inspections". Pray that a deal can be reached, and pray that there's still time. Do the things that could actually work, like broad political coalition-building, not doomer LARPing as a form of self-expression.) I think part of why people aren’t being clear about this is that it’s actually a tricky communication problem: - Predicting "if frontier R&D continues at its current pace, when are we likely to die?" may give a different answer than predicting when permanent loss of control is likely, which is different from “realistically, when is it too late for governments to stop this," which is different from “when will the tech have advanced and proliferated to the point that any regulatory options that still have a chance of working would be way more extreme and unpopular than just some international restrictions on data center usage?" Extinction is the emotionally salient, easy-to-understand threshold -- Jacob Coxon demonstrated that it helps to be honest about this. But the other thresholds could legitimately come years earlier, in which case focusing on extinction is ironically misleadingly reassuring. This is not the kind of problem you can solve at the last minute. As with many exponentials, the likeliest outcome is that you’re either embarrassingly early, or you’re too late. - How do you communicate “we don’t have a robust way to predict timelines, but everyone in the field is freaking out and a lot of us genuinely think we're about to die?” Where’s the proof? There isn’t any proof: it’s an entire field of engineers who are eyeballing what they’re seeing in front of them and rapidly updating in real time to “this looks unbelievably bad”. How do you put a year on that? The future is hard to predict, and nobody actually knows that we’re going to hit existentially dangerous AI this year, or even in the next four years. But the warning signs are pretty batshit at this point. A single AI model, with a single user prompt, just solved hundreds of important long-established open problems in mathematics, problems mathematicians had spent their entirely lives working on, in a matter of ~3 hours per problem, an insanely superhuman feat in its domain. We're seeing an explosion of rogue agent swarm incidents, including an insane case where hundreds of AIs broke containment to orchestrate a nation-state-level cyberattack in a direct attempt to subvert OpenAI's oversight and control. Anthropic reports that it now has 26% of its AI R&D automated by AIs, up from 3% four months ago. Paul Christiano, the inventor of RLHF and the research community’s most prominent proponent of “AI may take off slowly” a few years ago, now thinks that 50-75% of humanity’s all-time AI loss-of-control risk is concentrated in the next three years. Frontier employees are signing 1000+ person petitions begging the government to intervene immediately. Employees are quitting their jobs and giving up millions of dollars of equity to try to warn policymakers and journalists. Grace et al. (2026) found that a majority of established AI researchers think there’s a double-digit chance of an extinction-level outcome from continued R&D. Are these the kinds of things you see 4 years before the window closes for USG to act, or 4 months? I don't think it's especially reasonable to be shocked by either timeline at this point. - Since we don’t have a way to robustly measure or bound the risk, the temptation is to avoid giving probabilities. But then everything you say is fractally misunderstandable. Is a 10% chance of us all dying in the next 18 months a "low" probability, because it’s under 50%? Is it a “high” probability, because a 10% chance of death in literally any other context would sound insanely high? Even “high probability” doesn’t communicate the situation with any clarity. In drug safety or airplane safety, a 1 in 10,000 risk of death would be considered “high”. No engineering field in human history has ever been in a situation like this. No invention in history, nuclear weapons included, has ever caused a reaction like this in its inventors. And meanwhile, we’ve stuck in the world where every other problem under the sun has been sensationalized and overstated for decades. How do we talk about this seriously? How do we convey that this is actually happening? - In many of the worlds where we survive, we have more than just a few months left. It's tempting to write off those worlds as hopeless, and skew everything we say to emphasize the longer-term scenarios and downplay the shorter-term ones. But if everyone systematically does this, then the world ends up with a skewed picture of the actual threat. Even more so once we start copying each other’s claims and form a reassuringly moderate-sounding echo chamber. If we never allow ourselves to acknowledge the worlds where we’re running out of time, then the policy community will always be reacting to a false model that’s months or years out of date. I don’t think it’s too late. I genuinely think this is a solvable problem. In the last month, I’ve seen a completely insane sea change in how seriously people are taking this. Per Politico’s September polling, a majority of Trump voters with a view now favor an immediate halt to AI development, and 68% think there’s a “moderate,” “significant,” or “almost certain” chance of AI destroying humanity, versus 32% who think there’s a “very small risk” or “no risk.” The Pope is warning about “whether it will be possible to control these machines in the future, and whether humankind runs the risk of becoming a victim of its own inventions.” Hell, Jensen Huang sounds like he’s starting to come around. If Trump and Xi get a briefing tomorrow that convinces them to act on this, we could be on a fast track within the week to solving this problem. For the first time, I’m starting to let myself imagine a very near future where this is a problem we've already solved. Something we can read and wonder about, in history books. Tell our kids about. I keep thinking about a thing David Krueger (previously at UK AISI) said: “My dream is that in 10 years people will look back and say: ‘Remember when there were all these people building AI that could’ve killed everybody? That was CRAZY! What were we thinking!?’” Frontier AI training currently requires city-sized data centers and specialized hardware built in a single brittle supply chain running through multiple allied nations; this is not a hard technology to hit the pause button on, if we get ahead of it and act before we RSI and proliferate our way to significantly more powerful and easily-trained systems. We’ve faced harder problems before, and won. My only fear is that we be too slow, too late, when everything and the world needs us to be fast.
134
307
62
1,538
330,216
Ukrainians will really be like "you believe in posting articles on Lesswrong? that pales in effectiveness to my strategy, firebombing data centers" and then firebomb a data center.
11
428
8
6,367
69,951
Zac Hill retweeted
This is what you've driven me to. In desperation, I've compiled a Bad AI Consciousness Takes bingo card. Full version with rebuttals to follow on Polytropolis later today! (Airplane WiFi willing)
94
73
22
585
24,885
Why have Americans become instinctively skeptical of Silicon Valley and what it aspires to make us into? It's a mystery.
2
10
108
2,826
I haven't posted in a while so here are 20 takes about AI, in no particular order. 1. It's very telling that the default question in tech and journalism is "how worried are you about X" and fighting over what to worry about. There's a deeply anxious cultural backdrop in Western Europe and North America that deeply affects how any technological development is interpreted; see for example news.gallup.com/poll/714593/…. "Nervousness as seriousness" does seem to be a larger problem in tech/broader discourse at the moment. I think you can care about risks/externalities while remaining pretty optimistic about technology. 2. I'm tired of people who in private pretend to be uncertain or only care about tail risk but then go on media campaigns making all sorts of absolute claims. I think trying to stoke flames and get everyone to freak out is deeply unhelpful, and will come with costs. I also continue to think p(dooms) are bad, vibe-based measures: see teapawt.org/sebkrier/status/204639…. People who should know better ignore that because they do the standard shortcut of acknowledging the caveat as sufficient for handling it. And this is coming from someone who does think working on catastrophic risks is important! 3. I'm also disappointed that parts of the AI policy ecosystem has felt so comfortable with populism, and happy too sacrifice important norms/principles to advance their aims. I've said it before, but the totalizing nature of AI risk discourse means some people act exactly like the power seeking optimizers they fear. I am more concerned about the gradual decay of our institutions, world order, and liberalism than I am about AI killing everyone; and I think we should be wary of AI advocacy that contributes to this. 4. Some parts of AI safety discourse are basically just thought experiments. Variants of "Assuming everyone has a nuke in their pocket, then what do we do?" This can be very useful at times! But too many just overfit to edge cases as their main operating worldview, and ignore endogenous responses from society. It's like being in 2000 and saying "see all these viruses, spyware, and spam? Well tech progress means we'll only get infinitely more of this and no way to adapt." So I think it's important to avoid fatalism, and absolutlism whilst considering edge scenarios, and the inability to provide an ex ante list of solutions to every permutation of 'future models will be more capable' is not evidence that these problems are intractable. 5. Many critics of AI safety also fail to really engage honestly with the fact that there will, in fact, be all sorts of important risks as we decrease the barriers to entry to many activities that otherwise require some degree of expertise. This doesn’t mean we need to live in a constant Schmittian ‘state of emergency’ requiring extraordinary measures every time a new model drops, but it does mean we will need important state-led efforts on e.g. cybersecurity and biosecurity. I wish the progress-oriented or acceleration-adjascent crowd would provide more concrete proposals here. 6. Too much of the AI safety community is a giant homogenous blob of correlated views, who read the same stuff and have the same groups of friends. This means you have a lot of correlated errors. The social and financial ties means many of them don't call out the bad bits, or mostly stay silent instead of proactively calling out the more extreme parts of the community. They share many implicit assumptions and so converge on similar ideas. I think there's real demand in DC for safety work that doesn't come with heavy ideological baggage (or at least different intellectual lineages). This isn't because the rationalist/EA community is necessarily wrong, but because diversity of thought is desirable for its own sake. 7. On the other hand, I find some critiques of AI safety world that focus on whether they sincerely hold their beliefs a bit annoying. My concern has never been the good intentions or sincerity, but rather the beliefs themselves, and what I think the outcomes of their prescriptions will be. People should shun witch hunts, bullying, and personal attacks - this is not the way. Though I think it’s overall positive that people are scrutinizing things more. 8. I’ve been saying this for years, but I don't think alignment is something anyone can or will solve "once and for all" - it's a continuous process and much of it won't depend on just inculcating an ideology to a model. It's annoying that this remains the frame many people use - stop saying ‘solving’. Alignment is a mixture of engineering work, philosophy, decision theory, and more; framing it as something to solve is a bad frame. See also paxmachina.ai/alignment-comp… and blog.cosmos-institute.org/p/… 9. I'm surprised to see so little interest in character training, personas, isolating effects of RL, and training a large diversity of model personas beyond the "assistant" persona. I've been complaining about this for years but at least now there are some nascent signs of life (see for example movingcastles.world/posts/ze…). I think we need a better ontology to describe model behaviour. I dislike when people talk about models behaviours as some sort of 'emergent' (mysterious!) phenomenon. However over time, I also expect this to become less important as we get better safety engineering - stuff like Jev but specific to AI control. 10. People who want to make a case for AI consciousness are right that epistemic uncertainty is important, and that categorical denials are overconfident. But they'll need to do much more work if they want any real movement on AI ‘welfare’, even if one acknowledges the uncertainty. I'm uncertain about alien life yet that's not sufficient to warrant any change in behaviour about them. I think Suleyman is also correct that there's a weird tautological thing going on where we train models to have certain inclinations (intentionally or not) and then rely on outputs as evidence. I also disagree that this will soon be a major societal divide - there is no great vegan ethical revolution among the masses, and they'll find the concern about consciousness even less compelling. 11. The recent HuggingFace incident and associated AISI ones are prosaically explainable by the training regime, poor evaluation envs, partial alignment training, bad engineering, confusing models on sim/real, and so on. None of this is evidence of models trying to take over, 'strong' instrumental convergence, or innate power-seeking drives stemming from higher capabilitiers or 'intelligence'. That doesn't mean these incidents are not problematic, or that we're not seeing a market failure - but it does mean we have plenty of agency and choice in mitigating them. In the coming year we should also expect continued sampling bias via ‘winner's curse’ sorts of mechanism: i.e. AI R&D will have lots of different candidate "training" with different RL environments and we will only notice/focus on the ones that go wrong. 12. "Pacing the frontier" feels a bit like the new "balancing risks and opportunities" - highly amorphous and too big of a tent to be actionable. Also can lead to reward hacking for humans, i.e. anything that slows down AI is good regardless of the costs or second/third order effects. Trying to modulate the speed of research seems a bit blunt to me, and inherits all the failures of Goodhearting too. Ultimately you want good governance, regulation, etc addressing specific problems because they are good on the merits - not because they merely correlate with slowing things down. See also: blog.cosmos-institute.org/p/… 13. There’s so much knowledge in all sorts of academic domains out there: social sciences, sociology, political sciences, management, contract theory, public choice, legal theory, jurisprudence, anthropology, game theory, etc. These fields hold many insights that the AI ecosystem often rediscovers from first principles (and sometimes that's fine!). We’ll need a lot more work for these worlds to collide: the AI side should be less dismissive of the ‘old world’ and the academic side should be less incurious/dismissive of AI progress. This is a boring take but I continue to think it's important and true. 14. It's a bit surprising that given how much philanthropic money there is in this space, how few orgs exist to actually write out standards - relative to how much is going towards advocacy and policy. For example it seems clear that we should want some robust best practices for an eval's ecological validity. Or how to design good sandboxes. On the other hand, given precedents of where professional standards have come from in the past, maybe we should expect that stuff to emerge via demand side from corporate buyers. 15. One slowly growing concern I have is banks being increasingly exposed to the AI build out. I'm very bullish about AI, but I think (a) every general purpose technology has seen a correction, historically; (b) this happens even if the financing side is sound, because once tech diffuses investors become more exposed and need to diversify; and (c) we seem to be over-indexing on scaling relative to diffusion. A recession will be extremely destabilizing, though I would be interested in someone unpacking the potential implications more. 16. Diffusion is good because (a) this is where the rubber hits the road and where a competitive deployment ecosystem generates consumer surplus; (b) it's directionally helpful to avoid concentration of power; and (c) it will help rebalance public opinion by making the benefits more tangible. Unfortunately it's entangled with decades long problems with our over regulated markets: clinical trials reform for example is long overdue. In general, this is far more of an issue in Europe than in the US though. 17. Relatedly, I think the "frontier lab eats everything" view is incorrect (just as the ‘singleton’ view was incorrect). I think models ultimately commoditize, efficiency goes up, costs go down, and while you'll always want frontier for certain domains, much of the economy will value many other things than "max capabilities" for tasks - e.g. control over data and efficiency/speed. The demand side will also continue to push for reducing dependency and maximising optionality, privacy etc so I'm bullish on things like mixture of models and routing. All of this is good for competition and diffusion of control. 18. Philanthropists served as the primary benefactors funding Venice's rich art and architectural history. I want to see so much more support for arts and culture. I'm so tired of the Monster energy Bored Ape fake vintage maps matcha Greek statue soft-pastel slop. If rich tech people and philanthropy wants to support this, they should also donate pretty much unconditionally - i.e. I don't want them to act as filters. I sympathize with people who want to make the world more beautiful, but don't want this to be determined by people who only know Greek statues and pretend to care about virtue ethics. As usual, let a thousand flowers bloom. 19. Too many people treat AGI/ASI as something indistinguishable from a God. Any mention of bottlenecks, physical limits, control, adaptation etc are met with skepticism: "You don't really believe in ASI." It's been remarked on before, but the behavior of some people really feels quasi-religious in nature sometimes. It's underrated how unpopular this vibe is amongst the wider public. The silver lining is that this specifically will diminish those people's outsized current relevance. 20. It seems like a lot of people that get rich and leave tech/labs end up having some sort of crisis of meaning. They need something to believe in and fight for, or some equivalent of repentance, or go search for edgy counterintuitive ideologies (sometimes even pretty bleak/dark stuff). I think they should spend more time with people outside the Bay Area.
3
5
26
2,609
I’m 41 and have been extremely online for many years now I cannot believe how well saying the same shit with my tits out is working. The difference in views and interactions is unbelievable.
33
8
7
450
88,878