Sebastian Benthall (@sbenthall) header
Sebastian Benthall (@sbenthall) profile picture

Research Scientist at @ILI_NYU and @ICSIatBerkeley. A dad. Tweets are reflex, noise, and error. 🦋https://teapawt.org/t.co/t1n5c7A8Tp

An NYC suburb
Joined April 2008
Photo by Sebastian Benthall (@sbenthall)Photo by Sebastian Benthall (@sbenthall)Photo by Sebastian Benthall (@sbenthall)Photo by Sebastian Benthall (@sbenthall)Photo by Sebastian Benthall (@sbenthall)Photo by Sebastian Benthall (@sbenthall)
Sebastian Benthall retweeted
This is a great paper. It points out: 1. Humans do not even approximately behave according to rational choice theory 2. There is no reason to think advanced AI will "inevitably" maximize some utility function 3a. Human preferences are derivative / constructed, so aligning AI by matching its behavior to our stated preferences is wrongheaded 3b. We can align AIs directly to some normative ideal of a "good assistant / programmer / driver / etc." instead 4. Aggregating preferences across people is fraught with philosophical and mathematical difficulties. We should not aim to align AI to the "collective will of humanity."
Should AI be aligned with human preferences, rewards, or utility functions? Excited to finally share a preprint that @MicahCarroll @FranklinMatija @hal_ashton & I have worked on for almost 2 years, arguing that AI alignment has to move beyond the preference-reward-utility nexus!
39
134
28
974
158,619
My colleague Amy Greenwald just sent me this talk slide. It's from Jesús Fernández-Villaverde, an *economist*. We have so won. (I mean, I have *thoughts* about this content and his course slides, but also… We have so won.)
18
15
3
140
10,985
Sebastian Benthall retweeted
𝗝𝗮𝗺𝗲𝘀 𝗖. 𝗦𝗰𝗼𝘁𝘁 passed away on July 19. He was a political scientist and specialist in Southeast Asia. He was a dissident within Political Science. Yet his books opened up lines of research on central issues, such as the exercise of power and resistance to power.👇
24
1,117
294
2,763
712,142
Sebastian Benthall retweeted
AI-generated sad girl with piano performs the text of the MIT License
234
1,626
579
7,404
890,400
Sebastian Benthall retweeted
Excited to share our new paper arxiv.org/abs/2402.10877 (Oral, ICLR 2024, w/ @tom4everitt, @GoogleDeepMind). In it we answer the question, do agents need to learn causal world models? arxiv.org/abs/2402.10877. 🧵
6
74
15
348
73,083
Sebastian Benthall retweeted
Most coverage of the firing of Sam Altman from OpenAI is treating it as a corporate board firing a high-performing CEO at the peak of their success. The reaction is shock and disbelief. But this misunderstands the nature of the board and their legal duties. 1/n
110
482
138
3,864
1,546,698
Sebastian Benthall retweeted
Do not block the way of inquiry.
2
4
25
1,250
Sebastian Benthall retweeted
This might be flying under the radar (so please RT!), but the US Copyright Office is soliciting comments for its decisions on training ML/NLP/AI systems on copyrighted material (even *non*-GenAI). Researchers, please comment! Deadline Oct 30. copyright.gov/ai/
1
61
1
85
22,058
Sebastian Benthall retweeted
📛 Designing Fiduciary Artificial Intelligence 🧠 The article suggests a method for creating and reviewing Fiduciary AI systems to ensure legal compliance and address data consent issues. 🐦 1 ❤️ 14 ✍️ @sbenthall 🔗 arxiv.org/pdf/2308.02435.pdf
1
1
225
The rebrand of twitter to X, and Facebook to Meta, is so DEEPLY predictable, uninspiring, and boring. These are fundamentally engineers making these decisions. Specifically, computer engineers, who don’t even test ideas against practical physical reality. 🧵
1
1
10
586
Sebastian Benthall retweeted
Today we say goodbye to this great blue bird This logo was designed in 2012 by a team of three. @toddwaterbury, @angyche and myself, The logo was designed to be simple, balanced, and legible at very small sizes, almost like a lowercase "e", a 🧵
1,161
48,760
4,438
165,694
13,687,257
Sebastian Benthall retweeted
The "research stack" of the The Santiago Boys: - Zencastr (interviews) -OneNote (organizing research) -Atlas.ti (interview analysis) -Trint (transcription) -Custom Python scripts (named entity recognition, spelling, footnotes, etc) -The Syllabus (research) - Zotero (citations)
1
7
1
158
17,992
Sebastian Benthall retweeted
It's difficult to specify rewards for AI systems, which may incentivise systems to “reward hack”. Causal diagrams elucidate pros and cons of different mitigations: scalable oversight, mech interpretability, impact measures, and path-specific objectives alignmentforum.org/s/pcdHisD…
Today, we’re starting a blog post sequence explaining how causality underpins many critical aspects of safe AGI, including agency, incentives, misspecification, generalisation, fairness, and corrigibility. We explain past work and open questions. tinyurl.com/caus-safe-agi
5
8
925
Sebastian Benthall retweeted
Incentives capture some key forces that shape agent behaviour. In this post, we show how incentives can be analysed using causal models, and cannot be understood without them alignmentforum.org/posts/Xm4…
Today, we’re starting a blog post sequence explaining how causality underpins many critical aspects of safe AGI, including agency, incentives, misspecification, generalisation, fairness, and corrigibility. We explain past work and open questions. tinyurl.com/caus-safe-agi
4
17
1,376
Sebastian Benthall retweeted
Martin-Quinn scores for this Supreme Court term axios.com/2019/06/01/supreme…
86
123
144
819
598,321
Sebastian Benthall retweeted
Agency comes in degrees, and varies along multiple dimensions. Understanding it better may help us avoid dangerously agentic systems, and find ways to preserve human agency. In this 3rd post, we explain how aspects of agency are about causal relationships alignmentforum.org/posts/Qi7…
Today, we’re starting a blog post sequence explaining how causality underpins many critical aspects of safe AGI, including agency, incentives, misspecification, generalisation, fairness, and corrigibility. We explain past work and open questions. tinyurl.com/caus-safe-agi
6
16
1,840
Sebastian Benthall retweeted
Part 2 of our sequence gives a succinct but accessible introduction to causal models that handle interventions, counterfactuals, and agents. These models will be key building blocks for later posts alignmentforum.org/posts/9ag…
Today, we’re starting a blog post sequence explaining how causality underpins many critical aspects of safe AGI, including agency, incentives, misspecification, generalisation, fairness, and corrigibility. We explain past work and open questions. tinyurl.com/caus-safe-agi
10
25
4,406
Sebastian Benthall retweeted
Our conference paper from last year is up — with @sbenthall @alanlujan91 @ccmccomb John Liechty, Chris Carroll, and Nicholas Skar-Gislinge: "Simulating Heterogeneous Portfolio Choices and Financial Market Outcomes" ceur-ws.org/Vol-3420/paper4.…
4
7
21
7,096
Sebastian Benthall retweeted
Today, we’re starting a blog post sequence explaining how causality underpins many critical aspects of safe AGI, including agency, incentives, misspecification, generalisation, fairness, and corrigibility. We explain past work and open questions. tinyurl.com/caus-safe-agi
2
52
5
181
37,372
Sebastian Benthall retweeted
SVB was "a wealth management operation for the VC and PE industry" - with a precious banking charter. That's why we need to be careful about handing out bank charters to clever firms with 'smart' business models. We will have to subsidize them. ft.com/content/1795b4a7-65b0…
2
16
76
6,361