Soham De (@sohamde_) header
Soham De (@sohamde_) profile picture

Training at OpenAI. Prev: Research Scientist at DeepMind, PhD at the University of Maryland.

London, England
Joined May 2011
Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)Photo by Soham De (@sohamde_)
Super proud to announce AlphaGenome 2 - our new DNA sequence model! 🧬 It predicts the impact of DNA variants in different specific cell types - unlocking crucial information for understanding complex diseases. Available for academic research via our API: alphagenomedocs.com/
Your heart, brain, and immune system share the same DNA sequence. So how can a single-letter change affect disease in some cells, while sparing the rest? AlphaGenome 2 is our new AI model that zooms in on specific cells to help scientists understand what drives disease. 🧵
218
459
83
3,635
200,252
Soham De retweeted
Recursive self-improvement is a critical issue, but the rhetoric on the subject is becoming more poorly calibrated over time. As we approach new stages of AI development—and RSI is one of them—we need more precise frameworks and terminology. I fear the opposite is happening.
18
27
14
173
33,089
Overdue congrats ;) happy to see you two back as a team, there's only one more person missing...!
2
1
17
6,876
The Training team @OpenAI is hiring researchers in London 🚀 Our twin missions are to train better LLMs, and serve them more cheaply Get in touch if you are excited to collaborate on architecture design, reliable scaling, and faster optimization
12
38
8
501
96,440
Excited to share #AlphaGenome, a start of our AlphaGenome named journey to decipher the regulatory genome! The model matches or exceeds top-performing external models on 24 out of 26 variant evaluations, across a wide range of biological modalities.1/6
14
206
17
899
87,739
We have a new SSM theory paper, just accepted to COLT, revisiting recall properties of linear RNNs. It's surprising how much one can delve into, and how beautiful it can become. With (and only thanks to) the amazing Alexandre and @BachFrancis arxiv.org/pdf/2502.09287
2
43
1
169
11,186
📢 New paper on creativity & multi-token prediction! We design minimal open-ended tasks to argue: → LLMs are limited in creativity since they learn to predict the next token → creativity can be improved via multi-token learning & injecting noise ("seed-conditioning" 🌱) 1/ 🧵
1
44
6
165
29,576
Excited to share what my team has been working on lately - Gemini diffusion! We bring diffusion to language modeling, yielding more power and blazing speeds! 🚀🚀🚀 Gemini diffusion is especially strong at coding. In this example the model generates at 2000 tokens/sec, including overheads like tokenization, prefill, safety filters etc.
We’ve developed Gemini Diffusion: our state-of-the-art text diffusion model. Instead of predicting text directly, it learns to generate outputs by refining noise, step-by-step. This helps it excel at coding and math, where it can iterate over solutions quickly. #GoogleIO
91
246
77
2,643
578,164
Today, we’re open-sourcing our SynthID text watermarking tool through an updated Responsible Generative AI Toolkit. Available freely to developers and businesses, it will help them identify their AI-generated content. 🔍 Find out more → goo.gle/40apGQh
27
207
68
928
408,588
Great contribution from Meta to the research community with a very easy-to-read codebase for LLM development: github.com/facebookresearch/… @sohamde_ and @SamuelMLSmith have implemented Hawk as well, which seems to have a performance comparable to Mamba.
2
19
132
14,027
We have an opening for a PhD intern working closely with (among others) me, Arwen Bradley, David Berthelot, on scientific aspects of diffusion & generative models. 1/
4
38
2
204
48,077
We’re presenting AlphaProteo: an AI system for designing novel proteins that bind more successfully to target molecules. 🧬 It could help scientists better understand how biological systems function, save time in research, advance drug design and more. 🧵 dpmd.ai/3XuMqbX
66
775
206
2,843
1,080,782
A new blog post talking about Gemma architecture explained! This time is RecurrentGemma: developers.googleblog.com/en… This is the Gemma model that is not based in the Transformers architecture but on Recurrent Neural Network! Is this the return of RNNs? #gemmaverse
2
14
1,079
Are small models still undertrained? We are releasing a 2B model that beats GPT-3.5. The crazy part is that it was distill on only 2T tokens from a small model. Distillation is the future of LLMs with the growing availability of large and efficient open models!
10
39
2
365
62,590
Replying to @sohamde_
@sohamde_ is presenting on SSM architectures and RNNs in NGSM workshop at Strauss 3 #ICML2024.
1
9
429
I am absolutely thrilled to announce the release of Gemma 2! Today, we're releasing both pre-trained-only and fully post-trained 9B and 27B models. The full technical report is here: goo.gle/gemma2report and it's live *right now* on aistudio.google.com.
21
47
5
228
26,370
Welcome RecurrentGemma 9B 🔥 > Same performance as Gemma with more than 25% lower latency and 6-7x higher tokens/ sec ⚡ > Base (9B) and Instruct (9B-IT) models released. > MMLU - 60.5, CommonSenseQA 73.2, AGIEval 39.3 - pretty strong base model to fine-tune further. > Based on the Griffin Architecture > Achieves faster inference with long sequences by replacing gloabal attention with local and linear recurrences. > Available in Transformers! 🤗 Massive Kudos to Google for continue open research for alternative architectures! GG!
8
45
3
210
37,551
Our tutorial on diffusion & flows is out! We made every effort to simplify the math, while still being correct. Hope you enjoy! (Link below -- it's long but is split into 5 mostly-self-contained chapters). lots of fun working with @ArwenBradley @oh_that_hat @advani_madhu on this
26
253
23
1,414
215,758
📣 Exciting opportunity alert! Come join our team at @CW3E_Scripps and dive into innovative AI/ML applications in environmental, ecological, and atmospheric sciences 🌎 Pls DM for more details! #NowHiring
CW3E Machine Learning Post-Doctoral Position: Seeking a researcher with a background in machine learning and related applications for environmental, ecological, and atmospheric sciences. Get the full details on the position here: cw3e.ucsd.edu/wp-content/upl…
3
7
1,530