
ALT Feb 2024: "I believe we're reaching the upper limits about what generative AI can do and how accurate its outputs can be." Wrong4 March 2024: "Have We Reached Peak AI?"; another prediction that hallucinations mean that AI progress is limited to then-current levels Wrong April 2024: "As I previously warned, artificial intelligence companies are running out of data ..."; another prediction that models can't improve because there's no more data Wrong June 2024: OpenAI growth is stalling (with the implication it will continue to stall), which will lead to some kind of collapse of OpenAI Wrong (it could be the case that OpenAI will collapse but, if so, it won't be due to any kind of growth stall from 2024) July 2024: "Generative AI, as I said back in March, is peaking, if it hasn't already peaked. It cannot do much more than it is currently doing, other than doing more of it faster with some new inputs" Wrong July 2024: "Gen...

ALT No. The fact that a 3T-parameter model won’t fit on one GPU explains why you need distributed inference, but it does not explain why you need AFD specifically. You could solve the “doesn’t fit” problem conventionally: Huge model │ ├── Tensor parallelism ├── Pipeline parallelism └── Expert parallelism ← especially natural for a giant MoE All of those spread the weights across many GPUs. In fact, large MoEs are routinely served with expert parallelism specifically because the expert weights don’t fit on one device. AFD is solving a different optimization problem: attention and FFN have radically different resource needs.

ALT A classical oil painting–style illustration depicting an anxious monk holding a scroll labeled "NEXT TOKEN," surrounded by small demon-like imps presenting modern software engineering tools. In front of the monk, an open book reads "PURE AUTOREGRESSIVE PREDICTION" on the left page and "THOU SHALT NOT CALL UNTOOLS" on the right page, next to a chest labeled "IMPURE: TOOLS, SEARCH, RETRIEVAL, MEMORY, TESTS, AGENTS." The surrounding imps hold framed screens labeled with various AI system extensions: "TERMINAL," "CALCULATOR," "COMPILER," "UNIT TESTS," "WEB SEARCH," "RETRIEVAL," "MEMORY," and "SUBAGENT." Above the monk, a thought bubble contains an X post from Yann LeCun (@ylecun) reading: "You obviously did not understand my statement. I was talking about auto-regressive token prediction, which is what pure LLMs do. But good code generation systems aren't pure LLMs and aren't doing mere auto-regressive token prediction."

















