Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine) header
Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine) profile picture

she/her ⋅ LGBTESCREAL+ ⋅ reading committee @aft_technoprog ⋅ my xeets reflect the views of every person-moment I'm R-related to ⋅ 2ndary account: @MLaGrangienne

Joined October 2021
Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)Photo by Matrice Jacobine🔸🏳️‍⚧️ (@MatriceJacobine)
Hi everyone! This was my first Twitter account, it got banned and I switched to @MLaGrangienne before I got the admins to unban it. I'm now making this my public-facing English account. @MLaGrangienne remain my personal and French account.
1
1
21
11,537
Matrice Jacobine🔸🏳️‍⚧️ retweeted
Agree. Perturbation experiments, plus a lot of similar stuff, could tell us a lot. Even if with every passing week we'd be learning about models further and further behind the internal frontier.
It seems like there STILL won't be a thorough investigation into the hugging face incident, even after all this time! I keep talking to OpenAI researchers and asking about this, and what they say privately is also basically "it's just never going to happen sorry, not enough of a priority." To reiterate what I said at the time: Serious investigation of the HF incident could be a boon to alignment science for the whole world. For example, try putting the same models in the same situations, but varying the conditions slightly, to see what would have happened. Could also look at sources of evidence other than CoT to try to get more insight into model psychology, e.g. when one agent considered warning a human and decided not to saying "not my task" what else was it thinking? To what extent was this a rationalization vs. an outright lie vs. an innocent mistake? We could learn a lot about AI motivations and psychology by doing a bunch of experiments like this. More ambitiously, could try training new models with different recipes / alignment training / etc. to see what would have prevented the incident and what wouldn't have.
2
352
Matrice Jacobine🔸🏳️‍⚧️ retweeted
Me and who
problematic ideology gap yuri.
3
4
83
1,491
Matrice Jacobine🔸🏳️‍⚧️ retweeted
I currently put 20% on superintelligence within the next two years.
3
3
40
1,367
Matrice Jacobine🔸🏳️‍⚧️ retweeted
The study estimates 68 to 99 TWh of energy usage from data centers from May '24 to April '25. For reference, the Micron semiconductor fab planned for New York and TSMC fab planned for Arizona will *each* use about 16 TWh annually when fully built out in a decade and change. The ecological challenges around industrial demand for electricity are not specific to data centers. If we want to reindustrialize America, the challenges get much harder. That being said, I think this is an argument for putting intense requirements on data center development and for rationing them out as fixed utility infrastructure, as opposed to a Wild West competitive buildout.
“The largest data centers in the United States are responsible for an estimated 25.7 million metric tons of carbon dioxide emissions annually — almost as much as entire countries like Bolivia or Denmark — according to new research.” nytimes.com/interactive/2026…
4
9
2
37
3,227
Matrice Jacobine🔸🏳️‍⚧️ retweeted
Kinda wild seeing the actual uptick in incidents lately 👀
Here is a chart of cyberincidents tracked by Sentinel each week since mid 2024; the advantage of this proxy is that it is making a judgment call about worryingness each week rather than just incorporating the raw number. You can clearly see the huge recent upswing.
1
6
59
1,848
Ramez is, as usual, lying
AI pace of improvement at the frontier continues to look smooth with the release of GPT 6.2 Astra, with no clear upward bend. Anthropic has improved at a faster rate over the last year. The AI frontier (best model available) hasn't. A brief 🧵
1
57
Matrice Jacobine🔸🏳️‍⚧️ retweeted
It's a mistake to attribute a human-like property to a machine that doesn't have it, or to refuse to attribute a human-like property to a machine that does have it.
10
7
141
4,343
Matrice Jacobine🔸🏳️‍⚧️ retweeted
A member of the OpenAI board says frontier labs should pause algorithmic progress right now. Paul Christiano is clear he’s speaking for himself, not the board. But as Dwarkesh notes, he is on it! He’s also one of the most credible people in the world to be warning about these risks: he led alignment at OpenAI, co-invented RLHF, and was head of safety at the US AI Safety Institute (now CAISSI). When asked if the labs should pause: "I think we should [pause]. Pause can mean a lot of things. There are a lot of good things that frontier labs are doing that they should be doing. But improving algorithms in a way that can be very loosely diffused even normally, and could be quickly stolen, I think they should stop that right now." "I think it would be better right now, in the current environment, for AI developers to pause further algorithmic progress on those problems until their security posture is adequate, until they’re able to defend those innovations in the event of being unable to continue racing at maximum velocity, and until they’re able to tell whether they have adequate control of the systems they’ve built."
Paul Christiano @paulfchristiano was not messing around in his interview with Dwarkesh. His message is clear: Pause.
4
7
90
3,119
Matrice Jacobine🔸🏳️‍⚧️ retweeted
I don't buy that trust is currently the limiting factor - Previously METR got monorepo access for an investigation, but I didn't see that access being given here. They also aren't able to investigate internal incidents which I consider a step back from HuggingFace. - Apollo has been working with OpenAI for years, with fairly deep access, and I've never heard complaints - OpenAI has strong redaction rights, which IMO makes the risk of sharing IP or letting assessors talk to employees of their choice pretty low (the logged conversations part is more reasonable, to avoid potential misunderstandings) - It doesn't seem reasonable to be too worried about security, given the base rate of employees with way more IP moving between labs, with no non-compete, and the fact that people were able to hack on to the OpenAI Slack (so presumably foreign adversaries have already) More generally, if we could seriously have AGI within 12 months, do we really have time to mess around with pilots like this?
3
13
129
3,565
Matrice Jacobine🔸🏳️‍⚧️ retweeted
you get the ability to fly and the girlfriend of your dreams but in exchange you have to eat your favorite dessert at least twice a year
Would yall support decreasing income taxes across the board but in return drastically increasing property tax
1
11
222
1,843
Matrice Jacobine🔸🏳️‍⚧️ retweeted
This is the second time a journalist has referenced the same, specific incident of sexual assault. The victim has written about this incident, in her own words. The victim has said she was happy with how it was handled. It is very gross, in my opinion, not to acknowledge this.
1
22
221
Matrice Jacobine🔸🏳️‍⚧️ retweeted
In case you are interested, these are the genres I tried for the Pangram bounty. Every single pre-2020 sample came out 100% human: academic books on African politics and security academic paper acknowledgements with grant lists academic paper reference lists advance-fee scam emails African business news African general news African government news-agency items African opinion essays African poetry African sermons and devotionals African short fiction African student and education writing African technology news amateur rhymed poetry Arabic how-to articles Arabic local news and press releases astronomy telegrams BBS-era amateur stories BBS-era bulletin-board lore BBS-era conspiracy texts BBS-era humour files BBS-era occult texts BBS-era political tracts BBS-era UFO reports clinical trial descriptions company annual reports to shareholders Congressional Record floor speeches Congressional Research Service reports content-farm articles corporate press releases (SEC 8-K exhibits) cover-letter and interview-answer samples data-analysis textbooks developer mailing-list emails dream reports drug labels drug-label adverse-reaction lists early modern English printed books earnings-call transcripts education conference papers education evaluation reports education guides and books education journal articles education research reports education statistics reports Emu's private drafts * English-language government news-agency releases Federal Register notices and rules Federal Reserve meeting minutes flash fiction Hacker News comments horoscopes horror fiction and creepypasta internet standards documents (RFCs) IRC chat logs IRS private letter rulings literary-magazine essays and criticism LitRPG web serials from Royal Road mathematics proofs in LaTeX mathematics textbooks medical dictation reports medical-device adverse-event narratives mid-century literary short story NASA technical report abstracts Nigerian forum posts Nigerian lifestyle and entertainment posts NIH grant abstracts nonprofit and agency annual reports NSF grant abstracts organisational guidelines patent descriptions printed cookbook recipes product recall notices programming book indexes public comments on a Medicaid docket public comments on US regulations PubMed abstracts real-estate listings recipe blog posts safe-harbor legal boilerplate Saudi Press Agency releases science-fiction and fantasy magazine fiction self-published free-verse poetry slang dictionary definitions small-business press releases software bug reports software manual pages Stack Exchange answers statistics textbooks strategic plans Supreme Court oral-argument transcripts sustainability reports synonym dictionaries (scanned books) UK Acts of Parliament UK government corporate reports UK government policy papers UK government research reports UK parliamentary petitions vehicle-owner safety complaints vision statements weather forecast discussions white papers wiki cookbook recipes World Bank environmental assessments World Bank project information documents World Bank project status and completion reports World Bank working papers Yandex search engine poetry
3
6
1
87
3,481
Matrice Jacobine🔸🏳️‍⚧️ retweeted
I continue to be confused about what OpenAI thinks about RSI and what its plan is for preserving alignment during RSI. I appreciate the clarifications that the plan is not to hand off AI research and alignment to AIs and rely on AIs to align the AIs, but I am really not sure what the plan is instead (I know that @kaicathyc has said it is forthcoming, which I appreciate and look forward to reading). This interview with @markchen90 adds to my confusion. Mark Chen: "When we look at what's the most effective tool for aligning models, it it is chain of thought monitoring today. So when we think about the future of alignment, it has to be stronger models aligning other models. It does kind of feel like RSI must be intricately and deeply in this loop in some way." Uh, what? Isn't chain of thought monitoring currently deteriorating and it seems like it will deteriorate way more as we go through RSI? why does the success of chain of thought mean that therefore the future of alignment should be "stronger models aligning other models" and that "RSI must be intricately and deeply in this loop?" This seems like a horrifying plan! I really hope this interview is taken out of context and he says lots of other stuff that makes more sense and is more reassuring at other times, but to be honest I'm not sure he does because best I can tell this is the plan. @sarahdingwang's takeaway from the interview is also that OpenAI's plan for solving alignment involves RSI: "Humans are facing many problems today that are actually quite urgent and RSI could help us find solutions faster even for some of the problems created by AI itself. After talking to Mark and John, I do think that RSI could be the key to solving alignment." I realize there is confusion about what we mean when we refer to RSI (see Dean Ball's recent essay), but I found this video extremely unsettling, and it made me even more worried that the plan at OpenAI (and likely other frontier companies) is to do full RSI or very close to RSI and then use RSI to align models in some way, basically off a general vibe that we have used AI to help align models in the past? Not good! (I also am not sure that @johnschulman2 agrees with this conclusion based on his prior statements?) (to be clear, I recognize that automated AI research does have an important role to play in AI alignment, and if you are going to be totally automating capabilities maybe you have to automate alignment too in order to keep up. But this is why its a terrible idea to totally automate capabilities research and go way faster!)
For decades, the idea of AI improving itself was just a thought experiment. Today, AI is already helping researchers at frontier labs run experiments, write code and test ideas faster. So how far are we from AI that can do the research on its own? In our first episode of Working Theory, a16z General Partner Sarah Wang explores recursive self-improvement with two leading AI researchers: Mark Chen, Chief Research Officer at OpenAI, and John Schulman, co-founder and Chief Scientist of Thinking Machines Lab, and co-founder of OpenAI. They cover what RSI actually means, why research taste is so hard to teach, and why AI may be one of our best tools for solving alignment. 00:00 The last invention 01:58 What is RSI? 03:21 How far are we? 05:56 Getting to full RSI 10:57 Aligning RSI 15:08 Working theory YouTube: youtu.be/h_aA1-R70E8 @markchen90 @johnschulman2 @sarahdingwang
1
7
1
46
3,469
Matrice Jacobine🔸🏳️‍⚧️ retweeted
Yesterday, the CCP's top theoretical journal (求是, Seeking Truth) published a major piece on AI governance and risks. I think this is clearly the strongest & most urgent language on existential risks used in such an authoritative outlet. See below for comp to recent major doc.
5
49
7
223
20,521
Matrice Jacobine🔸🏳️‍⚧️ retweeted
Feel like a lot of people I bump into reason about AI catastrophic risk like this: "AI catastrophic risk would be onerous and invasive to respond to, therefore it is not real" imo you shouldn't reason like that.
3
5
2
91
3,049
Matrice Jacobine🔸🏳️‍⚧️ retweeted
>Can't import EVs, that's bad for American workers >Can't build EVs here, due to regulations >Can't have automated vehicles, every city but SF & Phoenix banned them So I guess we're just supposed to keep burning fossil fuels & letting people die in preventable car accidents?
Hasan Piker asked if he would support letting in EVs from China. When I heard this question, I thought I would finally agree with him on something. I've heard that Piker is pro-China. And since EVs fight global warming, this should be easy. But nope! He's a protectionist. Same views as Trump and Vance. We have to "protect" workers from competition even if it harms everyone else in society. I wish Piker was in fact a CCP shill, he'd at least support one good policy. Instead he demonstrates how much widespread agreement there is in favor of economically illiterate policies. What are conservatives even mad about? Piker believes "we're a nation, not an economy."
14
43
611
14,465
Matrice Jacobine🔸🏳️‍⚧️ retweeted
I don't think you even have to go through all that! Just collaborate to avoid the literal apocalypse and dare the justice department to sue. Do you have any idea how good that court case that would be, for convincing the world that the current race is horribly reckless?
4
17
1
164
3,506
Matrice Jacobine🔸🏳️‍⚧️ retweeted
My worry with AI regulation is that it leads to: - Public and external researchers knowing even less about the frontier - Relatedly, it significantly increases the internal-external deployment gap - Makes companies allocate more compute to AI R&D for internal deployment that leads to inventions that make them money, which offsets the lost potential external inference upside As there are more warning shots, it is important that we do pre-mortems ahead of time to push for policies that actually tackle the problem instead of a reactionary policy that makes things worse.
Rep.-elect @SamForstag (D-MT) on AI regulation: "We need to make sure that we are requiring the government to approve these models before they're let loose...it is terrifying what the future looks like if we don't get our arms around this technology."
4
1
8
878
Matrice Jacobine🔸🏳️‍⚧️ retweeted
Types of pacing that I'm most excited about in the short term (next 1-4 months): - cap on percentage of compute that can be used on AI R&D - do not advance capabilities in any way until you can demonstrate that China couldn't steal your weights
2
2
1
86
1,876
Matrice Jacobine🔸🏳️‍⚧️ retweeted
A reminder of how bad Emily bender is as an epistemic peer and also how bad the entire field of nlp was before the LLM took over
I recently became aware of how confused even many academics are about Pangram. Please, please, look at the audits that have been done before you deliberately sow doubt. (link in thread)
4
9
159
5,646