Co-Founder at Accelerated Understanding. Scaling AI with physical understanding in 4D.
Joined October 2012
- Tweets3
- Following2
- Followers436
- Likes0
Benedikt Jenik retweeted
A research duo, once pitched to lead the Jeff Bezos-backed Project Prometheus, on Tuesday showed what they built on their own.
Their company, called Accelerated Understanding, says its physics-focused AI handled 5 trillion pieces of data in one prompt. reuters.com/business/ai-foun…
Fantastic to finally be able to share what we've been working on. And doing it in the biggest possible way on @Reuters is a great bonus.
Our startup Accelerated Understanding builds AI models that simulate the physical world in 4D. Their capability to intrinsically understand physics enables innovation and invention. Instead of waiting for slow lab experiments or numerical solvers the model directly forms an intuition on how to improve your design.
Our models are different from the LLMs everybody is used to. For LLMs the context grows in one dimension and the sizes stay fairly manageable. In our case it grows in 4D, 3 dimensions in space, and time.
Doing this comes with a number of interesting engineering challenges: for one of our largest inference runs at 5 Trillion context length the input and output data alone gets so big (about 22TB) that you can't easily put it anywhere, and yet you need it in accelerator memory to be fast. To make it happen you need both modern AI training approaches, many of which we had to invent from scratch because the LLM way just doesn't work at our scale, mixed with a healthy dose of good old HPC tricks.
What I am most exited about is self-improvement and we have a unique angle here: With standard training the problem is that you eventually arrive at about the average quality of whatever training data you put in the model and exceeding that gets increasingly tricky. We have the laws of physics themselves come to our rescue: in addition to checking how well we fit the training data we can also check how well we observe those laws and improve that, even if the data doesn't.
@accelerated_u
Excited to see @Reuters cover the launch of our startup Accelerated Understanding.
We are training large scale AI models that can simulate and understand physics to invent and discover. Our models understand the world directly in 4D (3D + time) and across physical phenomena. Going full 4D requires massive context length, we have pushed it to a Trillion in training and exceeding 5 Trillion at inference.
AI giving you a bigger haystack of ideas doesn’t help. The bottleneck for new inventions and discoveries is shifting from ideas to the ability to test them. With AI that can simulate and understand physics we are directly attacking this bottleneck.
People have been trying to do this for a while now, but usually by taking shortcuts. Narrow surrogates are great if you happen to have enough of precisely the right data and your design loop stays in distribution. Video models look fantastic but sweep physical accuracy under the rug, and some static world models cut out physics altogether. A lot of interesting physics isn’t visual.
What does not cutting corners look like? Space stays 3D and you also have time: so 4D in total. You also need multiple physical modalities in the same model, not just things you can see. That’s what we’ve built.
Scaling is the primary ingredient to make this work. To represent the world you need sufficient context, which in our case grows in 4 dimensions. Individual samples get so big they don’t fit into single accelerators or even full nodes anymore.
We’ve developed architectural tricks to make it work. We’ve pushed our models to 1T parameters during large scale pre-training and are able to train at up to a Trillion context when needed and do inference exceeding 5 Trillion context without any sub-sampling or patching.
Building on prior successes of AI weather forecasting, fusion simulation, design of medical devices, drugs and chips, we wanted to see if scale and universality can benefit AI for physical understanding. With our teams’ experience in large-scale infrastructure and model training we’ve been able to pull it off.
reuters.com/business/ai-foun…
acceleratedunderstanding.com…
@accelerated_u @bjenik