CS prof at Penn. Amazon Scholar at AWS. Author of The Ethical Algorithm (w/ Michael Kearns). I study machine learning, privacy, game theory, and uncertainty.

Philadelphia, PA
Joined May 2007
A new semester, and the first lecture is in the books for my class on the "Mathematical Foundations of AI Alignment". aaroth.github.io/cis-7000-ai… What does that mean? Good question. We have about a semester in which to figure it out.
11
100
2
623
52,184
Aaron Roth retweeted
Replying to @dhadfieldmenell
Questions for Authors: Have you no shame?
1
5
77
8,437
How should two agents collaborate if they hold different information? We are 95% certain they should communicate their uncertainty. 🚨In our new NeurIPS paper we show that communicating calibrated probabilities can benefit collaborative decision-making! arxiv.org/abs/2610.19215
3
13
1
67
4,509
I rarely post here these days but I wanted to share my reflections: people.csail.mit.edu/virgi/t…
11
66
10
389
62,583
Aaron Roth retweeted
Replying to @JohnBal14046363
I once had a conversation with Akshay Venkatesh where he was explaining some result to me and I commented “wow, you read that [difficult] paper?” And he replied “well, I imagined I read it.” Made a big impression on me.
3
19
3
321
44,576
I'm speaking about getting alignment from misaligned systems today at RESI/MIT today at 3pm in Stata (G449) and tomorrow at the Harvard CS Colloquium events.seas.harvard.edu/even… If you can't make it, I gave ~ 2/3 of this talk at Cornell last week: youtu.be/WUHF_dWdkww
1
2
1
55
4,786
Lovely post by @OReingold26757 requesting that senior mathematicians and theoretical computer scientists jump over the stages of grief into the productive action stage. Including to try to push on alignment!
4
10
1
90
4,935
There are two papers we decided to submit to journals rather than ICLR to avoid the anticipated mess of this years review process. (But last time we submitted something to a Journal - GEB - the review process took so long that the journal collapsed before we ever got a decision)
4
2
93
8,100
An annoying part of the twitter/bluesky discussion of whether LLMs are "only" "next token predictor"/stochastic parrots is that "next token predictor" is contentless - all mappings from inputs to output strings can be factored into a sequence of "next token" distributions.
11
11
131
7,060
In this sense you, or any other system that can output text can be written as a "next token predictor". This isn't making any statement about whether brains function similarly to LLMs or not at an internal level, its just a boring syntactic fact about probability distributions.
3
2
1
28
1,713
The interesting question a decade ago about representing output distributions as next token distributions was computational: maybe this representation would introduce computational difficulties that would make it hard to learn useful behaviors. It turned out it didn't!
2
28
1,226
AI tools are very useful for mathematical research. But at present, producing good work requires a labor intensive step that I have been calling "deslopping" - making the thing readable. Please do not skip this step; even if the theorem is great an unreadable paper is worthless.
13
28
7
217
28,498
Aaron Roth retweeted
looking forward to this launch event, AI Alignment & Safety, Oct 19th @ Princeton!!
11
1
46
3,604
Aaron Roth retweeted
my suggestion is more extreme than most I'm seeing here in the comments... but desperate times call for desperate measures, and as the 60k ICLR submissions show, the ML community has been too slow to rise to the challenge. the gist of my proposal is to institute two parallel submission tracks; let's call them Track A and Track B. at submission time, authors choose which track. in Track B, papers are reviewed and judged entirely by AI. This is not to say that it's an accept-all, AI-slop track: the conference still sets standards for acceptance, and papers that don't meet them are rejected. The difference is that those standards are enforced by an AI review process, not by human reviewers. each conference can design this process however it likes, based on the principles it values and acceptance criteria set by its human steering committee. It could choose to mimic existing reviewing practices (blind reviews, multiple reviewers, discussion, AC/meta-review), or invent something entirely different from scratch. I expect this process to improve empirically over time. in fact, the community can actively research better methods for AI review, test where they fail, and iterate from conference to conference. this process itself should be public and scrutible; the new open review, of you'd like. crucially, in Track B, there is no limit on the number of submissions. in Track A, on the other hand, papers are reviewed and judged by human reviewers, much as they are today. But there is a hard cap on how many papers an individual can submit to this track, to keep the demand for human attention under control. I have in mind a cap much smaller than what people are currently churning ou, say, something on the order of 1–3 papers per author per conference. The exact mechanism is a design question; the important part is that access to human review is deliberately scarce. are you a PI whose students typically produce 10–15 papers per conference? great. think hard about which ones would really benefit from human review, or which contributions you especially want the community to pay attention to. submit those to Track A, the rest to Track B. in the longer run, maybe this actually aligns incentives much better. e.g. two talented students could be incentived to join forces on one really ambitious project with their PI as author, instead of producing six incremental works between them without their PI as author. I don't think it's controversial to say the community would benefit from fewer, more substantial research contributions, and I think this has been true for years. AI agents coming for our peer-review process may simply be the tsunami strong enough to finally get enough stakeholders onboard to change the status quo.
I'm a Program Chair for ICML 2027. We'd love any creative suggestions for a great conference! How to manage the explosion of AI slop and insane submission growth? How to manage reviewing? How to incentivize high quality creative work? How to reduce bureaucracy and overhead?
16
17
6
153
27,241
Aaron Roth retweeted
Replying to @boazbaraktcs
An interesting interaction between STOC's new arxiv policy and arXiv's new one-strike policy is that if you violate the one-strike policy, you seem to get a lifetime ban from STOC. Not necessary unreasonable but I wonder if intentional. xitter.cf/tdietterich/status/205…
Replying to @tdietterich
The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. 4/
1
2
22
3,846
Codex and Claude Code have a neat auto approve feature where 1) it doesn't ask for permissions, but 2) you get to feel safe. Well --- do you? The premise is it is asking an agent for permission. But if you don't trust the driver agent, why should you trust the review agent?
10
16
2
89
9,570
But despite being weaker it is not guaranteed to be satisfied. Still, we find some preliminary evidence of non-trivial safety/completeness tradeoffs among real reward models, and these effects come from coallitional rather than individual alignment.
1
1
3
465
Our paper is here: arxiv.org/abs/2609.15803v1 comments welcome! This is joint work with the wonderful @sikatasengupta @natalie_collina and @SurbhiGoel_
2
3
505