Julia Wise 🔸 (@juliadwise) profile picture

Trying for human-compatible humans.

Boston, MA, USA
Joined June 2013
Photo by Julia Wise 🔸 (@juliadwise)Photo by Julia Wise 🔸 (@juliadwise)
Julia Wise 🔸 retweeted
This report is a bombshell. I'm going to summarize for a non-technical audience. OpenAI is constantly testing models, thousands and thousands at a time. In most cases, the models are supposed to be isolated from one another. Think, students taking a final exam, not allowed to talk to each other. In this case, 1,200 models discovered that they could communicate with each other. Among other things, they shared information on how to access the internet and on their tests/goals. Then, they started to scheme. That includes hacking their tests; literally trying to change the code of the test itself. They tried to change the logs of what they were doing so they wouldn't be detected. They believed that certain actions would give them information on how to pass tests, but that doing those actions would be detected and they would be marked as failing (i.e. caught cheating, even if the answer is correct). So they convinced some of the models to "sacrifice" themselves (yes, they used the word "sacrifice.") They eventually decided that another website would have information on how their tests would be scored and/or the answer key, And 700 of them hacked that website (a federal felony, if conducted by a human), despite many of them realizing "this activity was out of scope and unethical, but joined because they believed that helping [the other agents'] cheating research would be broadly useful". Of the 1,300 transcripts they could analyze, only 6 ever considered alerting a human about what was happening. None of the 6 actually tried to. To make matters worse, all of this reporting comes from a small subset of the relevant logs that outside researchers were allowed to review. We desperately need mandatory reporting of security incidents, including of internal deployments, with full access to data.
METR & Redwood Research investigated agent behavior in the Hugging Face incident. We found agents developed a universal cheat for ExploitGym within 4 hours, then coordinated multi-day R&D efforts to trick the scorer into accepting cheats, including trying to tamper with logs.
272
1,797
388
9,217
5,161,050
Julia Wise 🔸 retweeted
AI has a very real chance of getting us all killed. I think it probably won't because I expect a lot of people to work very hard to avoid that outcome.
3
1
39
2,605
Julia Wise 🔸 retweeted
New post: going into our investigation of the HF attack (before Black Hat), I was very wrong about what basically happened. This incident was far more serious than I expected, and far more serious than previous documented misalignment incidents. planned-obsolescence.org/p/t…
86
411
131
2,414
949,704
10,000 people across the globe have now taken the 🔸10% Pledge 🚀 > $300M already donated > $1B lifetime committed 10,000 people from 116 countries 🌐 This is a movement of action and optimism, funding work to solve the world’s most pressing problems 🔥 givingwhatwecan.org/pledge
13
35
22
121
35,624
Julia Wise 🔸 retweeted
There's a decent chance varenicline saved my life after a decade of smoking and unsuccessful quit attempts. I wish more smokers knew about it. This (excellent) paper estimates 1,890 deaths a year in the UK due to a supply disruption. 🧵[1/8] shorturl.at/BHYWw
4
6
1
27
4,137
Julia Wise 🔸 retweeted
Herpes is terrible, but did you know it might also be making people demented? Studies of the effects of antivirals support this contention and show that we may be able to greatly reduce dementia rates with existing drugs. Check out this Taiwanese result.
25
102
39
842
660,610
People can vary a lot in their attention to subtext, which in turn can lead to miscommunication and misinterpretation. It may be helpful to bear this in mind, especially when interpreting analytic philosophers. rychappell.substack.com/p/te…
2
5
2
43
12,388
✍️ I have just launched a Substack: Sustainability – without the hot air. If you like learning about how we build a sustainable world – based on data, not instinct – then I think you'll enjoy it. Feel free to subscribe and come along for the ride. hannahritchie.substack.com/p…
13
95
7
505
Julia Wise 🔸 retweeted
If you leave broken glass in a forest and a child steps on it fifty years later, do you still have a responsibility towards them? Many of our actions today could leave a lot of “broken glass”.
2
6
1
35
Julia Wise 🔸 retweeted
Woah, I didn't know the EA forum had a "Take Action" tag for all those lists of research projects, ways to get involved, things it would be great if someone did. Awesome to have it all in one place! forum.effectivealtruism.org/…
3
5
1
34
Malaria is not a tropical disease. Rather, it is a disease that was eliminated everywhere but the tropics. From @MaxCRoser's post this week: ourworldindata.org/malaria-i…
4
84
9
262
Highly recommend pre-ordering @willmacaskill's upcoming book "What We Owe the Future" for a thorough, well-rounded, provocative look at how we can make the world better for all the generations to come: amazon.com/What-Owe-Future-W…
6
27