Phd Student @MIT | Economics, AI, & Behavioral Science | Research: https://xitter.cf/t.co/DvYK0d3Hsb

Cambridge, MA
Joined February 2022
Brand new paper with @johnjhorton that I'm very excited to share: "General Social Agents" Suppose we wanted to create AI agents for simulations to make predictions in never-before-seen settings. How might we do this? We explore an approach to answering that question!
16
83
20
454
146,164
Benjamin Manning retweeted
Very happy to share a new paper, “Technology Bottlenecks: Measuring Where AI Could Raise the Value of Work It Cannot Yet Do.” AI exposure measures ask about the work AI is capable of doing. We ask instead where the marginal value of work AI cannot yet do may rise as AI speeds up the rest of the job. To do so, we employ 288 million U.S. job postings mapped line by line to their closest O*NET tasks and GPT-4 feasibility ratings from Eloundou et al. (2024) to construct a measure, “bottleneck pressure,” for each task in each occupation that posts it. Using a simple task-based framework, we derive this pressure at the task level, showing that for an AI-infeasible task it is the exposure of the occupation that posts it, so the same task can face very different bottleneck pressure in different jobs. Reporting bottleneck pressure alongside AI exposure shows both the scope for AI to perform work and where improving the work it cannot do may pay off more as capabilities expand.
8
82
14
221
51,449
Benjamin Manning retweeted
Berry Wang @zwangcj5125 is presenting his job market paper in the @is_spoc talk series. Register and check it out! I've been working with Berry for a few years @Wharton and he's a fantastic scholar and wonderful guy. His work on the business value of AI and IT more generally has taught me a lot. He's on the market this year too.
1
5
16
1,092
Benjamin Manning retweeted
PSA: If you're a student and need @ExpectedParrot credits for your research, don't create a bunch of fake accounts w/ emails and make self-referrals to get credits. Just email us! We run a very loose, highly progressive fiscal policy & can make credits by fiat. We also have a soft-spot for students!
4
12
79
5,919
Benjamin Manning retweeted
Replying to @DKThomp
My former PhD student, BU Professor & co-author Emma Wiles! emmawiles.com/research/ cc: @ModeledBehavior
1
2
28
1,497
Benjamin Manning retweeted
lol - a *lot* of you are apparently on P&T committees or going up for tenure. If you went through our AI interviewer to get some free credits, just hold tight - we'll get them to you ASAP (Ms @ExpectedParrot is making soup at the moment & she reviews all of these requests.) Thanks for signing up!
I wrote a "playbook" for helping you prepare lists of potential tenure letter writers---researchers that know about the work AND are unconflicted by co-authorship, institutions, advising, &c. It's designed to work w/ our @ExpectedParrot Research Agent. As it would be weird as heck to run it on anyone else, I ran it on myself to show you how it works 1/
1
1
18
3,683
Benjamin Manning retweeted
I think AI Interviewers are very cool & will have a big impact in the social sciences. However, the best ones are very much aimed at enterprises, not academics. Expected Parrot recently created a self-serve, usage-based AI interviewer we hope you'll try out 1/
In my own personal celebration of Bob Hall Nobel Prize, I'm asking AI agents pretending to be people how'd they react to sudden windfall cash using another AI playing the role of interviewer. One nerdy AI apparently went *deep* on estate planning...
2
14
3
65
8,053
google scholar should add this feature. Also, how can I get galaxy clusters as one of my categories?
it's kind of interesting to see how the composition of the citations changes over time... it really highlights how scientists need to keep publishing in order to keep their citation count going up, as most papers crest and then die out
1
16
1,428
Benjamin Manning retweeted
Recent AI progress unbundles expertise from the workers that possess them: know-how that used to be tacit and inalienable can now be codified and duplicated at scale across the economy. We document some facts about this new market for human expertise.
1/ Modern AI systems are built on human expertise which is accumulated by paying humans. We measure this market: in a new note with @zoebcullen and @andrewjkoh we track ~213,000 expert hires and ~$540m in estimated compensation at posted rates on one leading platform. iemlab.org/projects/human-wo…
2
16
1
137
7,154
Benjamin Manning retweeted
There are two polar visions of this agentic economy, the good place and the bad place. In the good place, everyone is allowed to bring their own agent (we may need legislation for this to happen). We also develop new institutions and market designs to control congestion and other externalities arising from the fall in communication costs. As a result, we get closer to the ideal of a frictionless market that benefits consumers. In the bad place, one that @johnjhorton calls Robot Ripoff Hell, we don't price the externalities and platforms force consumers to use the platform's own agent. In this world, the good ways of signaling disappear, increasing our reliance on brand and personal networks. Some slides on this and other related topics are available here: docs.google.com/presentation…
Assorted things I've been mulling over regarding agents in the economy: • Companies are in some sense the original superintelligence. It's often the case that they can use their amortized cognitive surplus to advance their interests against those of certain consumers: relative to the optimization capacity of companies, consumers can be inattentive, underinformed, irrational, etc. (Think, say, of some of the classic insurance upsells at the end of a booking process, which is typically priced substantially above what a regular policy would cost.) Herbert Simon introduced the idea of bounded rationality, and firms often end up benefiting from it. Thanks to personal agents, we will probably get somewhat more rational in how we engage with companies. • How exactly will outcomes change? Well, certain kinds of price discrimination will work less well (how do coupons work if every agent hunts fastidiously for them?), and certain kinds of customer cross-subsidy will change (today, people who forget to cancel subscriptions subsidize those who don't). Taken across the economy as a whole, I'm not sure what the incidence of all of this will be. • In corporate strategy, people have long debated the relative importance of distribution and product quality. Perhaps you've made something better, but does anyone know about it? Won't people just continue to buy the ACME Corp Mousetraps? Personal agents will, I think, act as a kind of structural subsidy for product quality. How would markets look different if individuals always spent at least 10 hours researching their purchases? On the margin, focusing on making a product better is, I think, going to become a more effective strategy for companies. This is good! • It true, a corollary is that information about product quality will become more important and impactful. Today, most people have very limited ability to aggregate and integrate such data, but agents will have insatiable thirst for such signals. (It's often said that data is becoming more valuable because it helps in the training of models. That's obviously true, but I think there's a secondary way in which it's more valuable simply as a complement to cheaper intelligence.) • A big question is whether agents will increase superstar effects (rational agents all settle on the same product) or increase dispersion (because everyone has slightly different tastes and preferences, which agents are good at eliciting, finally overcoming the artificial clustering we've been subjected to). • Many internet companies have monetized demand routing, which is in turn often predicated on short attention spans. "I'll click on the first thing I see." This has led to some amount of inefficiency, as companies try to propitiate capricious ranking algorithms and incur various placement fees. It's not all deadweight loss (willingness to pay is itself a kind of signal, which is valuable to buyers), but it's not great. How will the nature and value of this routing change in a world of personal agents? I don't know the answer, but I think that the effects might be pretty big. • Much of what I'm writing hinges on the assumption that personal agents will be on the consumer's side. I suspect that that will be the winning strategy, but whether it's true, and what it means to be true, involves a lot of other questions. Taken as a whole, the market is a massively parallelized, decentralized, and highly imperfect value attribution machine, and, taking stock of all of the effects, it seems to me that the quality point may well be the biggest: that agents will change the nature of the backpropagation such that the rewards for better products increase. Maybe this is too optimistic, but I suspect (and hope) that companies that make things that they're proud of will feel that the universe is a little more partisan in their favor.
5
8
47
3,807
Benjamin Manning retweeted
This is impressive and may very well be the future. It is much more user-friendly and informative than the traditional PDF with enormous appendices. The review process will need to adapt, but we can find a way. @AEAjournals Here is the link: nobel-prizes.devdatalab.org
What should an open scientific paper look like in the AI age? We made a demonstration project out of our paper on the childhood socioeconomic circumstances of the Nobel Laureates. Please check it out and share. Details in 🧵 1/
1
11
75
7,687
Benjamin Manning retweeted
not deadwood day came on october 10 this year! 🎉
6
2
1
31
4,219
Benjamin Manning retweeted
What should an open scientific paper look like in the AI age? We made a demonstration project out of our paper on the childhood socioeconomic circumstances of the Nobel Laureates. Please check it out and share. Details in 🧵 1/
13
98
24
395
61,013
That’s all to say, if you apply to work with @johnjhorton (which you should all do - he’s the best advisor around) but if you don’t actually want to do research. It’s going to really suck! Regardless of how great an advisor you have. 5/n
1
11
683
With all that said, PhD as surely been the best 5 years of my life: the perfect combo of passion, purpose, freedom, challenge, cool people, meeting brilliant mentors, freedom, oh and freedom! N/n
14
617
I've elicited a lot of advice from @johnjhorton - he’s a 4sd above mean advice giver - but even for him this is good advice. Not just strategically for getting into the PhD, but like “do you want to do this.” 1/n
Replying to @johnjhorton
Sometimes we see personal statements that are 'personal' --- NO! It's badly named. It's just another place to talk about research. You really want to make it super clear you understand why getting a PhD isn't a terrible life choice for you. If you have any professional goal other than doing research, PhD is a bad idea. You could trick us of course, but again - you don't want to do that b/c it's a personally bad idea.
3
6
75
13,419
6 years is simply too much of an 80 year life to try and do this as a stop gap/hedge/“I don’t know what I want but I’m good at school.” It will be so miserable 4/n
1
14
704
I think one of the things that’s hard is undergrad really doesn’t prepare you for this type of environment/process. It prepares you to think, but it’s a series of goal posts and assignments designed for you in the form of homework and tests. 3/n
1
9
856
I know a lot of people in life who are lukewarm on their job (finance, engineering, doctors, lawyers, so on….) I do not know anyone who is lukewarm in their PhD. They either love it sososo much or just hate it beyond anything. There’s just so much unstructured time where you are exploring and have no idea what you’re doing searching towards an unspecified goal and no supervision. 2/n
1
1
26
1,908