I wanted to make a few clarifications, which we believe were clear in our paper but not in my original post (re-analyzing @METR_Evals data). Our contribution is to posit progress as a multiplicative product of sigmoids around different innovations. Given the METR data, we split it into improvements in base capabilities (data/model size) and reasoning. We show that this product provides a similar *in-sample* fit to the small datasets we observe as exponential growth. However, the implications are very different! Under our model, we would need continued innovations (akin to reasoning) to see continued exponential progress. This isn’t to say we rule out exponential progress, or that our product of sigmoids is the right model. It is simply to say there are few points and multiple possible underlying models with very different implications. Our product sigmoid fit actually fits very well when holding out GPT 5.2 and/or Gemini 3 pro. We do look worse when additionally holding out Claude Opus 4.5, but still plausible. Our goal isn’t to quibble about OOS metrics on a handful of data points, but to point out that existing forecasts are fragile, and don’t model the succession of different innovations. (There are a couple of other fits floating around X, but they don’t seem to be using our proposed product sigmoid so I can’t say what’s going on there…) I apologize for my un-nuanced earlier post – we hope people will read the paper!

Join us Wednesday at noon in AGH 414 when @hamsabastani will discuss Unpacking the Unintended Consequences of AI in Education!