After the surprise outcome of the Brazilian election, I was curious if AI forecasting tools predicted the outcomes correctly. It turns out a top AI tool (FutureSearch) made the same errors as prediction markets more broadly. This is an example of what @random_walker and I mean by “intrinsic limits” to prediction that have nothing to do with the AI tools in question, but rather depend on the inherent uncertainty in the world. Of course, a single failed prediction doesn’t mean that AI isn’t good at forecasting; in fact AI tools are already about as good as top forecasters, and I expect human forecasters to make heavy use of AI tools improve their own forecasts. But the inherent uncertainty in predicting social outcomes like elections means that AI tools’ performance will be limited close to that of top experts using simple automated tools (rather than being apparently unbounded with more compute, such as AI’s performance on chess).