Timothy John Swan ✝️

Just a note

Twenty years ago this month, I stood before a room at Google and presented work on deep learning networks. For ninety minutes the questions came, and for ninety minutes I answered them. The objection, when it finally arrived, was wonderfully mundane. "It requires too much compute." "Perceptrons are too slow." I suggested something unfashionable at the time. I suggested looking forward. Moore's Law was hardly a secret. Computing power was increasing relentlessly. If we started then, if we built for where hardware would be rather than where it happened to be that afternoon, we would arrive ahead of the curve. The response was largely the sort of practical wisdom that ages badly. The future was judged by the limitations of the present. A common habit among intelligent people. Today we live in a world intoxicated by Large Language Models. Every newspaper speaks of them. Every boardroom discusses them. Every investor discovers them with the enthusiasm of a tourist finding Paris. Yet the amusing thing is that LLMs did not appear because somebody suddenly discovered neural networks. They appeared because computation finally caught up with ideas that many had dismissed as computationally expensive curiosities. The mathematics did not perform a miracle. The silicon improved. What was once "too much compute" became routine. What was once "too slow" became infrastructure. What was once dismissed as impractical became one of the largest technological revolutions of the century. There is a peculiar vanity in assuming that the limits of today's hardware are the limits of tomorrow's civilisation. It is rather like refusing to build a cathedral because one happens to be standing in a quarry. The future rarely arrives by inventing entirely new ideas. More often, it arrives by waiting for old ideas to become affordable. And so here we are, surrounded by LLMs, watching the world celebrate what many once rejected, not because it was wrong, but because it was early. The difference between a visionary and a sceptic is often nothing more than ten years of semiconductor manufacturing. Or twenty.