Chris Morecroft

🐙 Mega-thread of freshly baked hot takes on AGI, multi-agent systems, models/scaffolds, economics, labour, meaning, and the future. 1. Existing models will continue improving and getting better. And they will continue to be trained while accounting for all sorts of things like cost, efficiency, steerability, personality etc. as we already see today. I think it’s more obvious than ever that there is likely no convergence to the One Big Model. More capabilities will be packed in smaller, efficient models: cost/resource scarcity means that there will always be demand for "slightly less good but much cheaper models." Intelligence too cheap to meter is marketing. 2. Software, scaffolds, harnesses, APIs, affordances etc., are where the rubber hits the road. Most organizations and people won't prompt models directly, but instead interact via intermediary systems. Models offer cognition, but the value comes from the deployment layer; the model is a necessary but insufficient component. Despite the messiness we see now, what entrepreneurs will make available soon will respond to demand, both obvious demand and latent demand that potential customers aren’t aware of yet. Demand derived from value-add often bakes in a degree of complementarity: you buy systems that you integrate in your business, environment, or whatever other context where human involvement is messy and not easily separable. Fixed costs of human integration are an essential consideration, and they’ll be critically important to think about (if you’ve been following me a while, you’ll know I am actively thinking and writing about these considerations), but getting to the point where this mechanism is dominant will require the intermediate progress from… intermediaries. 3. Increasingly, the focus will be on collective and industrial intelligence. Social technologies matter hugely and are often ignored by technologists who fail to zoom out. It's a cyborgist multi-agent world all the way down. Organisational structures will change and evolve too: if you fail to do so, it will be easier than ever for a small group of entrepreneurs to outcompete you. There are a lot of fruits that were previously unreachable but increasingly hanging lower. With the rise of entrepreneurship that focuses on displacing incumbents all along steps of whichever pipelines, the friction from convincing major players will be less of a drag than it was in the era of offshoring and replacing antiquated systems with software. 4. Here, there is still a lot to work out, and I expect high complementarity with human workers for at least the next decade. Roles will evolve: as you start doing less coding, your work looks more like technical product management. As you do less document review, you become more like a paralegal manager. And so on. Over time, better agents/products for management will emerge too. Neomanagers become CEOs, overseeing hierarchies of manager-agents who in turn direct execution sub-agents. And in turn, neoCEOs will band with other neoCEOs, M&A booms will lead to new holding companies and so on. The same applies to science research, and yes even ML R&D automation. Instead of painstakingly curating datasets and testing hypotheses, you are now a research director or an FRO CEO. 5. You just keep going up layers of abstraction, and humans continue steering complex multi-agent systems, until fixed costs bite. Part of the reason why humans always stay at the top of the chain is that many decisions made are normative: about what you want to happen, where you want things to go, how you want to react to changes. This requires inherently human inputs, since there's no point in having an AI decide this alone no matter how smart without eliciting more information about what the relevant humans prefer. Put differently: the telos of the whole system is the amalgamation of what users/consumers/businesses want, and tracking whether you're actually achieving that requires human input. This is already the case today with highly complex gigantic companies that make 1000 opaque decisions a minute. 6. Remember, this doesn’t violate the basic fact that market-coordinated economic activity is downstream of consumer and business demand. Capital isn’t some sort of independent force of the universe. What is being built depends on buyers/consumers that are ultimately human, even if occasionally intermediated by agents. The "AI decides everything" frame misses something fundamental about what economic and political systems are for. But as we go through these transitions, there are also costs or externalities (both pecuniary and non-pecuniary). Some people lose jobs. New industries cause unforeseen harms. Terence Tao has a great analogy: the abundance of food solved famine, but of course also led to harms like obesity. The solution is not to slow down abundance, but to develop the right norms, technologies, and laws to curb the excesses. 7. Accounts of full disempowerment assume democracy disappears, but I don't think all roads lead to autocracy. I don’t think ‘this time it’s different’. Growth and innovation historically benefited from free trade and liberal democracy, and this will be the case here too because of its impacts on investment, human capital, institutional quality, self-correction mechanisms, and ensuing fly-wheel effects. Many lessons from the past century already need to be rediscovered. Of course many current implementations of democracy (a) are decaying and fragile; (b) need deep reform; and (c) are necessary but insufficient. The combination of liberal democracy, rule of law, property rights, and competitive markets (what North, Wallis, and Weingast call an 'open access order') remains the institutional configuration that sustains long-term prosperity and innovation at the frontier, even as we are still iterating on critical implementation details. 8. As the world goes through these transitions, we will probably continue to see many commentators gloss over the vast benefits and improvements humanity will see. Progress in longevity, cured diseases, consumer welfare, massive reduction in poverty and famine, better education and so on. The arguments for market coordination over some sort of early-Soviet or Maoist collectivism apply even more in this world, not less. The world will generally become materially richer. Large increases in rates of growth for wealth, health, and the other aspects of the good life will transform societies the world over, including through more resources availability for redistribution if need be. Consider how much easier it was for a rich (and democratic) country to afford furloughs during the pandemic, or the welfare state after WW2, than for a LMIC, even in cases where such countries messed up implementation on other margins. Scarcity forces more difficult trade-offs than wealth. In fact it’s striking how rarely people consider the impact AI will have on LMICs: just as with mobile phones and the internet (and probably even more so), they will benefit disproportionately. 9. If we allow sufficient deployment of technology, robots, AI and so on, while ensuring the supply of energy, housing, and other important inputs isn’t constrained to a strangling degree, then the production of many goods and services will go down in price. In the early 1990s, a poor person in the US couldn't afford a mobile phone; today their purchasing power is much higher for most goods, despite dysfunctional scarcities around many sectors like housing or medical care. This trend will accelerate: personalized medicine, fancy boats, cutting edge surgery, fine dining, private education, yes even services many proclaim as inevitably Baumol-constrained, the forthcoming growth will mean these are no longer limited to the elite. Many people may even no longer *need* to work to afford to live under a roof and have a good life. Of course, many will also have their employment displaced as a result of these shifts. At first, most will likely be able to move to a different profession. Others might opt out of labour markets. In general I am more concerned with customer service operators in Bangalore than I am with upper middle class white-collar professions in the West. I think FDI and aid will be critical if we want humanity to thrive. 10. But this doesn't justify regressive populist policies or a 'pause'. It's not even optimal if we were being maximally selfish, and the equivalent of saying "poverty, misery and illness should be preserved for a longer period of time, for the benefit of a particular group of workers in time." Opposing AI or technological progress is a particularly nasty version of degrowth: it kills people, it entrenches poverty, and generally locks in all sorts of tragedies for the benefit of a comfortable elite who can easily thrive with the status quo. However, this does mean ensuring the right welfare systems, democratic protections, ‘societal resilience’, public infrastructure etc is important, as many have repeatedly noted over time. Just because things net out positively doesn’t mean ignoring those who lose out in the short run is the best we can do. There’s so much work to be done still if you want to build a better world, and I think we desperately need new, better economists, scientists, sociologists, artists, and politicians more than ever. I have more faith in the zoomers than some of my peers! 11. In parallel to the economic transformations, the world of governance evolves too. I think what democracy will look like and how it will be exercised will look very different from today's decaying systems. But the core principles will either not change, or evolve in sophistication. I wrote about Coasean bargaining not long ago, but this is just one of many ideas. What is key is ensuring government institutions remain accountable, and preventing either a tyranny of the majority or the kind of vetocracy that paralyzes much of the West today. 12. In the future, I expect politics and governance to be an increasingly important component of people's lives: many will care deeply about how things are organised and managed at the local or national or international level. Personally, I think it’s fine if a large fraction don’t care much about those issues most of the time, since I don’t think there’s an obligation for everyone to have an opinion on everything, and that preference will likely be easy to satisfy. Many will devote their lives to all sorts of artistic, heroic, spiritual, and social pursuits. A proliferation of subcultures and micro worlds of wonder. This isn't "nursery for adults" but what many people already do outside of work if they can afford it. I'm a bit skeptical of "lack of meaning" narratives that seem to focus on value-add from market or non-market work. I think people can find plenty of meaning in activities that don’t require being "depended on" in an ‘economic’ sense. If the cancer researcher cares more about being depended on for status and meaning than curing cancer, then I’m afraid they’re in the wrong. I think we'll look back at such frames with disgust. I also think that more often than not, people are conflating achieving a minimum level in current status hierarchies that focus on work with what works for a happy life when such activity is no longer as critical. 13. And I do think status games will continue, albeit in a much more diverse ecosystem of sub cultures and geographies. But again: always has been. Even today plenty of people more interested in art have zero envy for techbro founder lifestyles, and conversely many engineers couldn't care less about being perceived as cultured. As people get richer, much of this will evolve too. I imagine some people will devote their lives to solving problems and creating beauty, whereas others might develop a nihilistic attachment to wealth and status: I’ve spent enough time in Geneva to see both sides of the coin. People often bring up inequality, but I wouldn't expect it to feel anything like what we have today, where there is still a strong sense of the bottom half being deprived by consumption or the whims of the wealthy. I think the gap between what will effectively be ‘the rich’ and the ‘ultra rich’ will matter less to people, but the gap in status and social hierarchies will matter more. Remember how much Elon wanted to be perceived as very good at Path of Exile 2? 14. Ultimately, AGI will bring about huge positive transformations for the world, many of which are hard to describe: could anyone at the dawn of the Industrial Revolution have told you about video games, eye surgery, deep sea diving, street tacos, and mRNA vaccines? I’m not saying this because I think safety is not important (it is, very much so!) or because I think everything will be rosy and fine. But I think there are strong incentives to point out all the ways things may or will go wrong, and few good accounts of the positives apart from bland corporate slop. So I think it’s important to continue to make the case for this important technology. 15. Lastly, so much of the field uses "this time it's different" as hand-wavey justifications for flouting norms, justifying unusual political measures, ignoring fragile progress built on centuries of trial and error, and various yet-to-be seen proposals for haphazard action (made confidently despite the uncertainty that one might guess would come with handling unprecedented phenomena). I think this is misguided: AGI will be huge, and of course will affect everything around us; but in many ways it’s also not different, and as always, there's a lot to learn from History. Much still needs to be built, except that this time you will also have millions of agents by your side to make progress. 🚀