Metascience @IFP

Joined April 2022
Hunter Wieman retweeted
i sometimes encounter this strange logic where people argue that, if we want to pace ai development, we need to start by expanding our lead over china by speeding up. as this report demonstrates — there are other ways to widen our lead! things like countering adversarial distillation and limiting overseas compute access allow us to retain our lead abroad, while pursuing more cautious development at home.
Big new IFP report from @GeorgiaCAdamson, @KonstantinPilz, and me on the US-China AI competition. We spent the last 6 months digging into how Chinese AI companies’ access to compute and distillation of American models translate into the current US-China capabilities gap. The result is a 23k-word research report, including an interactive analytical model that visualizes changes in the US AI lead under different policy interventions. Our findings: 1. The US AI lead over China is currently only around 6 months. 2. The US could roughly double this lead, to over a year, using authorities already available to the federal government. 3. On chips: China’s access to AI chips will roughly double over the next year, with chips housed in data centers overseas accounting for roughly half of China’s 2027 compute. 4. On distillation: the current ~16x US training compute advantage would normally yield a 13-month lead. But we estimate adversarial distillation halves this to around 6.5 months. 5. Increasing the lead would have some obvious benefits: - Give the US broadly a greater cyberdefensive advantage against Chinese threat actors. - Give US AI companies breathing room to spend more R&D resources on making their systems safer and more secure, without fear of being overtaken by less safety-conscious Chinese competitors. 6. Provide the US government more leverage in negotiations with China about the future of AI. 7. The two most important interventions are: - For BIS: Create a new end-use control to restrict China’s remote access to training clusters abroad — China’s largest and fastest-growing source of frontier-scale compute. - For DOJ/FTC: Create a narrow safe harbor to allow US companies to coordinate on countering Chinese adversarial distillation. 8. Both interventions (and all the others we discuss in the report) are things the federal government could do today. 9. To estimate the effects of each policy intervention, we model the US lead as a function of its compute advantage over China and China’s ability to adversarially distill the capabilities of the US frontier. 10. The model makes many simplifying assumptions, but provides a useful way to assess and rank the impact of different policy interventions at a high level. Read the full report here: ifp.org/how-to-increase-amer…
3
6
81
3,258
Hunter Wieman retweeted
I have made a good-faith effort to learn from MMT and I can report that I learned nothing. If I was supposed to learn that a country that issues its own currency can inflate away its debt, I already knew that. (And I worry about that.)
The fact that economists continue to post charts like this as though they self-evidently support this statement means we still have much to learn from MMT.
79
90
26
1,021
103,501
Hunter Wieman retweeted
This one makes me feel sad. I hate the suggestion at the end that we “can't keep Jews in Denver safe while Israel is executing a genocide.” I’ve got Jewish kids here in Denver. The safety of everyone in this city is an imperative, no matter what’s happening around the world. I appreciate that Melat acknowledges that the Boulder firebomber was “trying to incite terror in the Jewish community.” That makes it antisemitic. Jewish safety means the safety of all Jews, whether or not we talk about Israel/Palestine exactly the way someone wants us to. Look, we can be loud critics of Israel. I am. We can argue hard about US policy. We really have to. But the safety of our kids shouldn’t be part of that debate.
DSA candidate Melat Kiros responded to criticism for not calling an attack on an Oct. 7 hostage-support demonstration in Colorado an act of antisemitism. “I think whether it was antisemitic or anti-Zionist, it was a hate crime.” 🎧 #TheConversation: politi.co/4B9muEp
46
12
336
45,981
monosemantic neurons be like
19
76
6
1,418
29,513
Hunter Wieman retweeted
If you were to ask me (which, I understand, you haven't), I would say that, in fact, my written work, in a typical or, at least, in the median case, needs no editing -- indeed, my prose leaps forth, fully formed, from my pen as Athena emerged from the head of Zeus, and I neither benefit from my editor's direction to shorten my sentences, simplify my clauses, remove jargon, eliminate unneeded elaboration, etc; nor is my writing (alas!) more intelligible, nor more digestible (if brilliant and idea-rich sentences are to your taste, that is) following the valiant, dogged, and indefatigabile work of editorial team. Nevertheless, other writers on our team would, no doubt, benefit from the talents, skills, etc of an editor. So, with that "throat clearing" aside, I will implore you: if you are such an editor as I have described above (having the requisite skills, etc, and, I now think to add, also having a burning desire to use said skills to achieve policy impact) or if you merely have designs of becoming such an editor, please apply below, if you are so inclined.
There are enough policy opportunities that we need more editors at @IFP! Shape op-eds, reports, and private memos that let us increase the upside (e.g. infrastructure) and lower the downside (e.g. biosecurity) of a dynamist future ($5k referral bonus) ifp.org/opportunity/editor-a…
6
8
1
66
2,651
Hunter Wieman retweeted
Come edit IFP’s “problem child.”
There are enough policy opportunities that we need more editors at @IFP! Shape op-eds, reports, and private memos that let us increase the upside (e.g. infrastructure) and lower the downside (e.g. biosecurity) of a dynamist future ($5k referral bonus) ifp.org/opportunity/editor-a…
4
3
28
1,331
The biggest, and stupidest, myth in European politics, particularly popular in France, is that Europe suffers the consequences of "austerity" and "neoliberalism". In the biggest 5 countries in Europe, more than half of the population makes its income from the state, less than half from the private sector. This is what makes the problem of reducing the welfare state so impossible: more than half of the voters are net recipients. No surprise that Italy's budget has not been balanced in 100 years, France's in 50 years (ooh la la, l'austérité!!): -France: 30 million from the state versus 21m private sector workers. State spending is 57% of GDP, taxes more than half the economy. The budget hasn't been balanced in the lifetime of most French (since 1974). - Germany: 35 million adults from the state, versus 34 million from the private sector. 10% of the population is registered as severely disabled. Public spending is over half of GDP. Germany has had a continuous budget deficit since 2019. -Italy: 23 million from the state, versus 20.5 million private-sector workers. Public spending is 51% of GDP. Italy has had a budget deficit every year since 1925, for 100 years. - Spain more people get their income from the state since 2024: 18.21 million from the public sector versus 17.9 million people working in the private sector. Since 2025, Spain also had more government employees than self-employed workers (autónomos) for the first time in history. State spending is at 45%, budget has not been balanced since 2007. - UK: 54% of households now receive more in state benefits than they pay in taxes. State spending is 44.5% of GDP, taxes (40%) are at their highest since 1950, the budget hasn't been balanced since 2001, and 1 in 5 new cars is paid by the government through a bizarre state disability scheme (Motability).
246
1,759
330
5,607
605,145
Enormous news if true. I'm very curious what this level of interpretability would and would not enable. Could we use it to identify and precisely remove dangerous model capabilities in virology? Or are capabilities and neurons fundamentally different objects of study?
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
58
Hunter Wieman retweeted
Anthropic's leaders need to stop commenting on the emergence of RSI like a local news anchor reporting on traffic on the 101, and acknowledge that they are the traffic, and they're not going to build something that they don't want to exist. They should move beyond "pacing" language, commit to specific policies and red lines, explain why those lines are red, and then use that costly signal to build an international consensus around norms. It's not that I trust the CCP, or that I think global lab monitoring and verification is imminent. But the status quo is absurd. You've got US lab leaders writing (or favorably quote-tweeting) essays about slowing down while simultaneously saying "things are moving faster and faster" and "btw, our internal models finished mathematics on Tuesday, next up: physics." The effect is the equivalent of Usain Bolt winning race after race while using his press conferences to bemoan the dangerous speed of modern sprinting on one's legs so would everybody please try a little less hard next race, and then ... breaking his 100m record again the following month. Honestly ask yourself: If you were China, why would you take this pausing/pacing stuff seriously?
Jack Clark of Anthropic now thinks it's more likely we see full RSI become possible in 2027 than in 2028.
57
186
18
1,481
256,367
Hunter Wieman retweeted
Meta AI Chief @alexandr_wang on China: “I've personally changed my own opinion on this matter. In the past I was very worried about China. I had read these old speeches that were given by PRC officials in the past that talked about how they viewed AI as this opportunity to leapfrog the West. But as time has gone on, I think we’re painting them to be too much of a boogeyman when they're going to see the same things that we see in terms of this is powerful technology. There's going to be a lot of shared goals in making sure that AI goes well.” Full HUGE* Conversation out now.
43
69
28
849
422,968
Hunter Wieman retweeted
More convinced than ever at @rootsofprogress: the key cultural change we need is that the goal for young people is to know how to and to value contributing to the human standard of living, rather than "standing up for justice", self-actualization/self-development, etc. 1/2
2
20
176
7,747
Hunter Wieman retweeted
MMT sowing: "Anything we can actually do, we can afford"--haha f*ck yeah! yes! MMT reaping: "Anything we cannot actually do, we can't actually afford"--well this f*cking sucks. what the f*ck
This is what crowd out looks like.
4
2
77
2,923
Hunter Wieman retweeted
This is silly. BAAJA (the permitting deal) has robust provisions to make data centers pay their fair share or MORE! Here's how it works 👇: - Data centers (*sometimes) add costs to electricity bills by requiring grid upgrades to service the new demand - When utilities have to increase grid capacity they invest in new lines, and pass the costs to the ratepayers - This means a portion of data center costs (the second order upgrades they cause the grid) get spread to you the ratepayer - What BAAJA does it require data centers to pay their ordinary costs AND pay for any costs they cause to the grid (BAAJA calls this "incremental costs") - This is a big step from what data centers pay right now, grid upgrades can be expensive - BAAJA also allows states to go EVEN FURTHER and make data centers pay even more than their incremental costs -BAAJA lets states: add even more costs, require data centers to bring their own power, require data centers to power down during peak demand time, and to treat data centers less favorably than other large industrial customers So no the permitting deal was not written to benefit data centers... I think a much better theory is that Senator Whitehouse wrote the bill because its good for the climate and that's what he cares about. xitter.cf/jeffhauser/status/2107…
Sheldon Whitehouse owning a ton of data center stock.... Huh, what else has he been in the news for lately? Oh, yeah, now the permitting "deal" makes a lot more sense!
5
9
3
59
5,122
They're calling the Senate the 100 under 100
Senator Chuck Grassley turns 93.
7
2
29
1,304
Hunter Wieman retweeted
"How do single-celled algae swim toward light sources" is the kind of project that gets DOGE'd—how does this help Americans? 20 years later, the underlying protein is a central tool in human neuroscience. We're a rich country, we should be funding way more basic science.
9
195
7
1,331
34,090
Hunter Wieman retweeted
Also really cool metascience ideas. Metascience units, X-Labs, burden busting - the works! Check out the Build More Agenda here: buildamericacaucus-harder.ho…
We applaud Build America Caucus leaders @RepJoshHarder and @RepMaloyUtah for their new Build More Agenda. They've aimed big: 5 million more homes, millions of new child care slots, a modernized energy grid, and infrastructure built faster.
2
6
605
Hunter Wieman retweeted
Replying to @JohnBal14046363
I once had a conversation with Akshay Venkatesh where he was explaining some result to me and I commented “wow, you read that [difficult] paper?” And he replied “well, I imagined I read it.” Made a big impression on me.
3
19
3
322
44,600
Hunter Wieman retweeted
FWIW in my own work it is relatively uncommon for me to find a statement of a lemma that I don't already know how to prove. This varies a lot from person to person and field to field but suggests at least sometimes the hardest part is making the conjecture.
1
1
9
413
Hunter Wieman retweeted
Ok, finished going through the OAI math list. It is bonkers and should be front page news around the world if folks understood what this meant. But if I am running strategy at a lab, #1 priority is "do this for medicine, oncology, battery efficiency, etc as fast as possible".
42
163
12
2,881
162,138
New York had to take 3 years and write 4,007 pages just to get through federal environmental review to do its own congestion pricing. But if the new permitting reform bill passed, this ludicrous, anti-environmental outcome would be history. The congestion pricing NEPA review was so ridiculous because any opponent (like New Jersey) can sue and say you should've considered some extra thing. If a judge agrees with them, the project stops. But under the new bill, lawsuits can only stop a project for substantive harms, not for writing 4,007 pages instead of 4,025 pages. Thanks to work by @SenWhitehouse, @SenatorHeinrich and others, America can be liberated from the anti-environmental consequences of our environmental laws.
Remember those giant binders that blocked congestion pricing? BAAJA fixes that - The bill would force agencies to finalize environmental assessments in one year. - If the highway department dragged its feet like it did under Trump 1.0 NYC could sue them and get a court to compel action. - The review would also be shorter: BAAJA gets rid of unnecessary alternatives analysis and the need for litigation proofing. - If New Jersey (or anybody else) sued the NEPA review they wouldn't be able to delay or block congestion pricing. - DOT would be barred from revoking any approval for congestion pricing & NYC could take legal action if they tried xitter.cf/AidanRMackenzie/status…
5
63
446
12,651