Director of Applied Science @GoogleAI: #ClimateChange, #AIforScience, #QuantumComputing, @GoogleColab [email protected]
Seattle, Washington
Joined June 2014
- Tweets691
- Following162
- Followers23.6K
- Likes812
We've shown (in a live trial) that we have a statistically significant method for avoiding contrails. blog.google/technology/ai/ai… #climatechange
Looks like a very sensible way of encoding v long contexts in transformers with 2 length scales of learning arxiv.org/abs/2305.07185
Nicely cleaning up an ugly thresholding hack in diffusion models.
Presenting Reflected Diffusion Models w/ @StefanoErmon!
Diffusion models should reverse a SDE, but common hacks break this. We provide a fix through a general framework.
Arxiv: arxiv.org/abs/2304.04740
Github: github.com/louaaron/Reflecte…
Blog: aaronlou.com/blog/2023/refle…
🧵(1/n)
6
2,883
From a techno-economic grid study, fusion needs to reach relatively modest capital cost goals to take off.
This Post is from an account that no longer exists. Learn more
2
1,808
First step down the road to error-correxted quantum computing quantumai.google/qecmileston…
2
865
A review of the exciting work in 2022 on natural science at Google Research ai.googleblog.com/2023/02/go…
Teaching students how to use generative AI as a tool oneusefulthing.substack.com/…
I've used Vizier: it's quite robust. Recommended.
Adding alkalinity to the ocean surface can absorb CO2, but the rate of CO2 absorption depends on where you add it. Results from an ocean GCM: bg.copernicus.org/articles/2… #CDR
Breakeven fusion may have been achieved at LLNL. Fingers crossed. Let's see the announcement. #fusionenergy
Tom Wilson, "US scientists boost clean power hopes with fusion energy breakthrough" ft.com/content/4b6f0fab-66ef…
#energy #peakrenewables
David Baker from @UWproteindesign gave a very nice talk at the #NeurIPS2022 Science workshop: diffusion models for proteins, where the diffusion step includes running structure-from-sequence prediction. Preprint: bakerlab.org/wp-content/uplo… #protein
5
An excellent summary of NeurIPS "outstanding papers", including links to demos, etc. Highly recommended.
Excited to go to NeurIPS conference tomorrow! It's an annual gala for AI. Many revolutionary ideas debuted here, like AlexNet, Transformer, & GPT-3.
I read *all 15* Outstanding Papers and can’t wait to share my thoughts with you all. Here's your front row seat to the festival:🧵
ALT Photo of a robot receiving an award at a big conference, illustration, beautiful, geometric, trending on artstation, full body, cinematic. - DALLE2 prompt
1
7
Transformers can "learn to learn" simple function classes. Interesting!
LLMs can do in-context learning, but are they "learning" new tasks or just retrieving ones seen during training? w/ @shivamg_13, @percyliang, & Greg Valiant we study a simpler Q:
Can we train Transformers to learn simple function classes in-context? 🧵
arxiv.org/abs/2208.01066
4
Very clever use of diffusion models.
Can we do model-based RL just by treating a trajectory like a huge image, and training a diffusion model to generate trajectories? Diffuser does exactly this, guiding generative diffusion models over trajectories with Q-values!
diffusion-planning.github.io…
🧵->
1
7
Clever idea from @_jasonwei et al.: if you want a large language model to do reasoning, give it detailed worked-out examples. Original paper: arxiv.org/abs/2201.11903 recent follow-on: arxiv.org/abs/2203.11171
Utterly delightful reproduction of @ylecun 1989 paper on digits (which I was heavily influenced by). Indeed: we were limited by both data and inability to tune losses/hyperparameters back then.
New blog post!⬆️ Deep Neural Nets: 33 years ago and 33 years from now karpathy.github.io/2022/03/1… we reproduce what I think may be the earliest real-world application of a neural net trained end-to-end with backprop (LeCun et al. 1989), try improve it with time travel, and reflect.
