Gamow Labs retweeted
Astra and Sol 5.2. by @OpenAI substantially push the Pareto frontier on @GamowLabs evals and take the 👑 outright on LabBench and ScreenBench by a hair.
Most interestingly, they do it very efficiently at ~20% of the cost and token consumption of Fable 5.2.
I haven't spent sufficient time with these models to comment on the qualitative aspects of their performance, but I frequently find myself dreaming of Chad IDE while waiting for agents to complete their work. I think that this trend toward brevity and speed is quite welcome even if the absolute frontier remains largely intact.
For me, I see Gemini 4 Hydrogen and Astra 5.2 fighting for this spot in my workflow and will report back when I can crown a vibes victor.
Gamow Labs retweeted
The first @GamowLabs peer-reviewed scientific publication just appeared in the American Journal of Medical Genetics Part A.
We show that we can use George, our clinical genomics agent, to identify two lethal balanced translocations in previously unsolved alveolar capillary dysplasia cases.
These diagnoses went undetected by traditional NGS methods and would have been invisible to older techniques like karyotyping due to their position.
I hope that this is the first of many and we can continue to contribute to advances in the field.
onlinelibrary.wiley.com/doi/…
Gamow Labs retweeted
Google can't stop.
@GoogleDeepMind Gemini 4 Hydrogen outperforms @AnthropicAI Opus 5.5 across most of our rare-disease benchmarks for a fraction of the price, lower latency, and faster throughput.
Hydrogen vs. Opus 5.5
RareBench: 49.2% vs. 51.6%
LabBench: 46.0% vs. 40.2%
LabRobotBench: 31.7% vs. 23.0%
ScreenBench: 70.2% vs. 47.2%
And latency and throughput don't get talked about enough. The combination of Gemini 4 Hydrogen's speed and intelligence lets it operate as a real-time agent in the loop with our laboratory robotics.
The video shows our workcell autonomously running an experiment with nothing more than a natural language prompt to Hydrogen. This performance is reflected in its very strong score on LabRobotBench as well.
More information on all of these benchmarks is available at gamowlabs.com/benchmarks
Congrats on another great launch, Big G!
Gamow Labs retweeted
Big G remains 👑 of rare-disease evals.
@GoogleDeepMind Gemini 4 Carbon and @AnthropicAI Fable 5.2 tie for first on @GamowLabs RareBench, solving a staggering 61.5% of problems.
When I started this work, GPT 5.4 only eeked out 9%. Anyone saying AI will not transform healthcare is out of their mind at this point. Stay tuned for an updated, harder RareBench v2 now that this task is close to saturated.
Gemini Carbon stands alone in first place on LabBench, which measures the model's ability to interpret real rare-disease laboratory data and predict the next experiment.
A tip of the hat to both GDM and Ant for their tremendous work in this space. I am very confident that we are getting on a trajectory to solve rare disease in the next few years.
Gamow Labs retweeted
We at @GamowLabs used our GRETA platform to measure the effect of 1,000 FDA-approved compounds on the transcriptome of human lung fibroblast cells in an attempt to identify hits that could be repurposed for use in idiopathic pulmonary fibrosis (IPF), a rare disease with high unmet medical need.
In addition to recovering drugs already used or tested in IPF, we found evidence that common hydrophobic statins can reverse the IPF gene expression signature in lung fibroblasts.
We believe GRETA can find drugs that could be repurposed for rare diseases, where treatment options are often limited.
If you’d like access to this dataset, or want to explore how GRETA could help with a particular disease or genetic variant, please reach out!
gamowlabs.com/greta-finding-…
Gamow Labs retweeted
RareBench results are in!
My adapter model (8.5M LoRA, 13.7M projector, Evo3-7B, Qwen3-14B) marginally outperforms Carbon-X (Qwen3-14B), but we are still way off the frontier. A modern agentic LLM easily saturates this benchmark.
This is more evidence that these little multimodal DNA models are fun toys but really don't generalize to out-of-domain tasks like clinical variant interpretation and their results may be a bit narrower than implied.
I do wonder how much of this outperformance is simply due to model capacity. While my FLOPS and data budget was orders of magnitude below Carbon-X, I am using 50% more parameters, which should count for something.
On that note, I requested access to @RadicalNumerics Omnii to see whether their more scaled approach to this problem could outperform the frontier while my FineDNA training run finishes. I am very curious if I can achieve similar results with a much more efficient approach with the exact same dataset.
Despite overcoming a broken GPU, results are in. It's fairly clear that beating Carbon-X with a simple adapter/projector approach is possible!
I had Claude Opus 5.5 run an autoresearch loop on my 5090 over a few days and significantly outperformed Carbon-X on 7/9 of their tasks, despite using orders of magnitude less compute and data.
This isn't totally surprising. My approach, which @adibvafa pointed out is not unique and he successfully used for BioReason, reuses the enormous amount of compute and genomic data that went into @arcinstitute Evo2. @huggingface face needed to train that into @Alibaba_Qwen from scratch.
What I found more interesting than the results was the process. I spent in aggregate a few hours on this project giving the model high-level guidance. Claude generated training data suitable for the tasks (you could argue this is overfit, but I don't think any more so than the FineDNA dataset used for Carbon-X), orchestrate a half dozen training runs, ablated a huge number of variables, and ultimately generated a very strong system.
This is insane. I worked on AI at Meta for 7 years and this work could have easily consumed a reasonably senior RS for a half and resulted in a modest publication.
Overnight, I will score my model on RareBench to see how I perform on truly out-of-sample classification tasks and train directly on FineDNA fine-tuning tasks to see how an adapter performs in an apples-to-apples comparison.
Gamow Labs retweeted
Despite overcoming a broken GPU, results are in. It's fairly clear that beating Carbon-X with a simple adapter/projector approach is possible!
I had Claude Opus 5.5 run an autoresearch loop on my 5090 over a few days and significantly outperformed Carbon-X on 7/9 of their tasks, despite using orders of magnitude less compute and data.
This isn't totally surprising. My approach, which @adibvafa pointed out is not unique and he successfully used for BioReason, reuses the enormous amount of compute and genomic data that went into @arcinstitute Evo2. @huggingface face needed to train that into @Alibaba_Qwen from scratch.
What I found more interesting than the results was the process. I spent in aggregate a few hours on this project giving the model high-level guidance. Claude generated training data suitable for the tasks (you could argue this is overfit, but I don't think any more so than the FineDNA dataset used for Carbon-X), orchestrate a half dozen training runs, ablated a huge number of variables, and ultimately generated a very strong system.
This is insane. I worked on AI at Meta for 7 years and this work could have easily consumed a reasonably senior RS for a half and resulted in a modest publication.
Overnight, I will score my model on RareBench to see how I perform on truly out-of-sample classification tasks and train directly on FineDNA fine-tuning tasks to see how an adapter performs in an apples-to-apples comparison.
The @huggingface Carbon-X DNA model @LoubnaBenAllal1 trained reminded me of the early days of vision-language models. No one knew how to best fuse multiple modalities into the same reasoning backbone, but it turned out that training a small adapter to project images/speech into text space worked really well (LLaVa, Llama 3, etc.)
I let Codex take over my 5090 today to few-shot this approach with DNA. I used @arcinstitute Evo2 7B as the frozen DNA encoder and Qwen3-14B as my LLM, which later received a task-specific LoRA.
The results were really good! With 24h of cheap compute and occasional human intervention, I trained an adapter-LoRA approach that seems to outperform Carbon-X across the board on my (very reasonable) tasks (and match Astra, which when prompted correctly gets most of them right but isn't really suited for these tasks).
I'm running on the BioReason, GenomeQA, and other Carbon-X benchmarks overnight and will report back tomorrow and vibe code everything into a @GamowLabs blog post for inspection, but I think this is a fruitful area of research and I want to see how far I can push an adapter approach with minimal compute resources.
Gamow Labs retweeted
Our mission at @GamowLabs is to democratize rare-disease diagnosis. Robotics play a huge role.
We try to one-shot experimental design and execution, but this doesn't work today partially because of the gap between model-perceived and actual robot pose.
@zdql__ set up a clever eval to measure this gap, which is essential to close for truly autonomous wet-lab work.
Please reach out if you're working at the intersection of AI and wet-lab automation and want to collaborate on solving this problem!
Robotics RL is very hard, particularly because of how expensive robotics can be to operate. In high-consequence environments like a wet lab, performing many rollouts to find out what works is impossible. Because of this, we simulate.
How do we prove realism in simulation? How can we ensure that the learning that occurs in simulation translates to real-world hardware?
These are some of the most important questions to ask in robotics RL. We focus on this in our latest LabRobotBench update. As we iterate, we bring our simulations closer to reality, allowing us to
1. Learn faster, because we can rollout in simulation
2. Be more robust in production, because we've caught failures in simulation.
Our latest blog post goes through our approach in detail.
Gamow Labs retweeted
Robotics RL is very hard, particularly because of how expensive robotics can be to operate. In high-consequence environments like a wet lab, performing many rollouts to find out what works is impossible. Because of this, we simulate.
How do we prove realism in simulation? How can we ensure that the learning that occurs in simulation translates to real-world hardware?
These are some of the most important questions to ask in robotics RL. We focus on this in our latest LabRobotBench update. As we iterate, we bring our simulations closer to reality, allowing us to
1. Learn faster, because we can rollout in simulation
2. Be more robust in production, because we've caught failures in simulation.
Our latest blog post goes through our approach in detail.
Gamow Labs retweeted
Do AI agents 🤖 check their data?
We @GamowLabs evaluated AI agents on 182 compound-screening analysis tasks, a standard procedure in wet lab biology. We introduced 7 kinds of hidden error into the data, testing whether agents notice errors that an experienced human biologist would.
Gemini 4 Argon took the 👑 with a score of 0.49. Opus 5.5 scored a close 0.47 and GPT 6.1 Sol scored 0.12. When told what kinds of faults to look for, all agents ≥0.94.
The agents miss different errors. Gemini gets 94% of source-exchange answers right and Opus 40%. On rotated plates it's the reverse, 97% for Opus and 77% for Gemini. Line swaps and well-order errors get past all three.
Blog: gamowlabs.com/screenbench-ag…
Paper: gamowlabs.com/assets/screenb…
Follow my esteemed coauthor Leah Damon @ljdamon23!
Gamow Labs retweeted
Drug development seems to be in the air, so we're proud to share the first @GamowLabs drug repurposing study.
We screened every FDA-approved drug, looking for one that could help the child in @huggingface's Real Kid, Rare Disease hackathon. One of the most interesting hits: an HIV drug.
We previously proposed that the child’s MVA was caused by biallelic loss-of-function variants in BUB1B, so we sought FDA-approved candidates to increase BUB1B expression.
Public data couldn't answer that, so we ran our own screen of every FDA-approved compound and measured each one's effect on the transcriptome.
Several compounds raised BUB1B. The most interesting is raltegravir, an HIV integrase inhibitor, which could be worth investigating.
It's an early proof of concept, but the method is real and we'll be running it at scale soon!
If you work on BUB1B, MVA, or chromosomal instability, we'd love to talk and understand how we could strengthen both these findings and the assay.
gamowlabs.com/mva-hackathon-…
Gamow Labs retweeted
Better late than never! @Kimi_Moonshot's servers have been melting since their K3.1 release, so it's taken us a few days to successfully complete a RareBench eval.
Is K3.1 benchmaxxed? I don't think so. They score a quite strong 38% on @GamowLabs RareBench, matching GPT-6 Astra exactly and improved 10 pp over K3.
RareBench is a challenging, held-out, long time horizon, agentic task that I think it is essentially impossible to overfit to. There was lots of chatter of Kimi doing this and that, but I think the proof is in the pudding here.
And they did it all for ~$250 or roughly a quarter the price of the closed frontier models. Don't count out Moonshot AI!
Gamow Labs retweeted
The @huggingface Carbon-X DNA model @LoubnaBenAllal1 trained reminded me of the early days of vision-language models. No one knew how to best fuse multiple modalities into the same reasoning backbone, but it turned out that training a small adapter to project images/speech into text space worked really well (LLaVa, Llama 3, etc.)
I let Codex take over my 5090 today to few-shot this approach with DNA. I used @arcinstitute Evo2 7B as the frozen DNA encoder and Qwen3-14B as my LLM, which later received a task-specific LoRA.
The results were really good! With 24h of cheap compute and occasional human intervention, I trained an adapter-LoRA approach that seems to outperform Carbon-X across the board on my (very reasonable) tasks (and match Astra, which when prompted correctly gets most of them right but isn't really suited for these tasks).
I'm running on the BioReason, GenomeQA, and other Carbon-X benchmarks overnight and will report back tomorrow and vibe code everything into a @GamowLabs blog post for inspection, but I think this is a fruitful area of research and I want to see how far I can push an adapter approach with minimal compute resources.
Gamow Labs retweeted
We at @GamowLabs are one of the few startups that has seen this firsthand, employing 50% PhD biologists and 50% AI engineers.
There is a clear cultural and philosophical divide between these groups that we are aggressively working to bridge.
When we get there, I think we will see crazy acceleration toward our goal of solving rare disease.
If anyone else in the same situation wants to compare notes, I would love to connect.
There’s a really pronounced divide between people who are on the AI side of things and people who are on the biology side of things. Even among people working in AI, the divide persists based on where they’re focused.
The AI side, adherents of scaling laws and the bitter lesson, believe that AI getting better means AI being more capable of novel discoveries in medicine (that go beyond discovering new drugs in well understood targets or speeding up dry lab work etc).
The biology people think the idea of a bitter lesson in biology is kind of ridiculous and that regardless of AI’s capabilities it’s going to be at least 5-10 years until you could hope to reliably simulate human toxicology in silico (right now AI/ML can’t even reliably simulate in vivo studies like binding affinities or functional assays).
Even the most AI pilled of them, who will list all the areas that AI can reduce the time spent on various areas, don’t think the concept of AI discovering novel targets or generating new understanding of causal biology is realistic in the near term. Most, in fact, will readily characterize the assumption that even an “ASI” level model could find cures hidden in our current understanding (without expanding that understanding with significantly more biological data) as patently false.
This was a big part of why we wanted to experience it ourselves by using AI for a (very) small portion of what it can do in biology. It was both mind blowing and revealed some poignant insights into how different the problem of making AI better at something like coding is to the problems of biology.
Even if it unlocked the ability to generate new molecules at a stunning pace, its simulation (in silico) of binding only had 49% accuracy. And if we wanted to improve that, we’d need to feed it back the data from the lab - data that took about 30x as long to get as it took to generate the molecules. And these were tests in a test tube (turn of phrase), imagine how long it would take if we wanted to improve the model with data that required recruiting humans and monitoring their reactions.
Human biology is very complex, and adding a drug - especially a novel drug and doubly so for a novel drug acting on a novel mechanism - means mapping interactions that have nth-order-effects that we don’t have the data to predict. Virtual cells have made advancements enabled by AI, but the perturbations equate to a small fraction of the possible.
This is before you even get into curing disease, especially uniquely human diseases. Brain function involves cellular identities that are unique to humans, meaning animal studies are of little use.
How long does it take to understand what the side effects of chronic administration of a drug in humans are? AI can speed up analyzing the data, or designing the trial or even getting it approved…but it can’t make a 6 month trial complete in 3 months.
I think its important to point out that significant advancements can be made in AI that contribute to biology, while also being realistic about how much it speeds up something as complex as curing a specific cancer or diseases of the brain.
Thanks for the shoutout @Google! Gemini 4 Argon is the top-performing agent on our RareBench benchmark, viewable here gamowlabs.com/benchmarks
Gemini 4 Argon is already helping researchers unlock new scientific breakthroughs across fusion, biology, and quantum computing:
⚛️ In nuclear fusion, Argon helped discover a broader class of counterexamples to Grad's conjecture — a nearly 60-year-old assumption in plasma physics — creating new benchmarks for stellarator 3D magnetic fields.
🧬 In biology, it designed five novel, lab-verified serine esterases in a matter of hours — enzyme engineering that would previously have taken months. This work demonstrates the potential for rapid acceleration in fields such as biomanufacturing, green chemistry, and drug development.
🩺 In rare disease research, scientists at @GamowLabs used Argon to analyze de-identified genomes from newborns with rare developmental lung disorders — pinpointing a chromosomal rearrangement that routine software missed for years, and explaining why it went undetected.
🧮 In quantum computing, Argon translated our AlphaQubit error-correction decoder into a single optimized hardware kernel — nearly doubling inference throughput — and cut spacetime costs for a key quantum subroutine by more than 100x.
Gamow Labs retweeted
Gemini 4 Argon is already helping researchers unlock new scientific breakthroughs across fusion, biology, and quantum computing:
⚛️ In nuclear fusion, Argon helped discover a broader class of counterexamples to Grad's conjecture — a nearly 60-year-old assumption in plasma physics — creating new benchmarks for stellarator 3D magnetic fields.
🧬 In biology, it designed five novel, lab-verified serine esterases in a matter of hours — enzyme engineering that would previously have taken months. This work demonstrates the potential for rapid acceleration in fields such as biomanufacturing, green chemistry, and drug development.
🩺 In rare disease research, scientists at @GamowLabs used Argon to analyze de-identified genomes from newborns with rare developmental lung disorders — pinpointing a chromosomal rearrangement that routine software missed for years, and explaining why it went undetected.
🧮 In quantum computing, Argon translated our AlphaQubit error-correction decoder into a single optimized hardware kernel — nearly doubling inference throughput — and cut spacetime costs for a key quantum subroutine by more than 100x.
Gamow Labs retweeted
In the course our our clinical genetics research, we at @GamowLabs have assembled a number of a very interesting datasets and evals that probe troughs in frontier model understanding of our tasks.
@wardbradt just put together a nice portal to explore some of these findings. These are all very high-quality, unsaturated, original datasets that I believe we can use to propel science forward with AI.
Please get in touch if any of them seem interesting.
By popular demand, we @GamowLabs have published a preview of our biological reasoning tasks at gamowlabs.com/tasks. The library spans wet lab robotics, drug design, genome analysis, and scientific reasoning.
Our goal here is to evaluate models at biological reasoning and, via reinforcement learning, improve these capabilities. If you're interested in collaborating with us on this research, please reach out.
Next week's new datasets will study compound screening and transcriptomics!
Gamow Labs retweeted
By popular demand, we @GamowLabs have published a preview of our biological reasoning tasks at gamowlabs.com/tasks. The library spans wet lab robotics, drug design, genome analysis, and scientific reasoning.
Our goal here is to evaluate models at biological reasoning and, via reinforcement learning, improve these capabilities. If you're interested in collaborating with us on this research, please reach out.
Next week's new datasets will study compound screening and transcriptomics!
Gamow Labs retweeted
I love this idea of jointly embedding text and DNA, but this first attempt underperformed my expectations on our task of real clinical genome interpretation.
I assembled 10 1k-token DNA snippets containing pathogenic variants contained in @GamowLabs RareBench, 80 1k-token DNA snippets containing a non-pathogenic but similar variant, and 10 1k-token snippets from the reference genome.
With these sequences, I tried a number of experiments to try to get Carbon-X to identify the pathogenic variant including directly prompting the model for pathogenicity, prompting the model to explain the phenotype, and computing the log likelihoods of the different mutations.
In 8/10 cases, the model was worse than random chance at predicting a given variant would be pathogenic and only confidently correct in 1/10. A frontier LLM nails all of these.
@LoubnaBenAllal1, let me know if you want to collaborate on improving here!
Introducing Carbon-X: The best open DNA model.
We midtrained a 14B model on DNA and text, and it beats every open language model and matches GPT-6.1 Sol at 1,000x lower cost.
Ask it about a sequence, what a mutation does, or to generate new DNA.
huggingface.co/HuggingFaceBi…
Gamow Labs retweeted
We at @GamowLabs recently ran a high-throughout screen to measure the effect of 2,370 approved compounds on various cellular pathways.
One of our findings was that 20 of the 30 drugs in the screen known to inhibit transport also increased expression of genes associated with cholesterol synthesis.
This is the exactly the conclusion that @OpenAI drew with their gene perturbation study. Very nice to see the same biological effect replicated with different techniques.
That said, neither study showed a change in intercellular cholesterol, which is the obvious next step in confirming the effect.
Happy to run this experiment if someone is interested!
GPT-Rosalind surfaced a new lead for drug discovery: could restoring cholesterol transport protect neurons from degeneration?
DCTN1 is a gene linked to Perry syndrome, a rare neurodegenerative disorder. In mouse-brain data, disrupting it increased expression of genes involved in making cholesterol. GPT-Rosalind suggested a possible explanation: cholesterol may not be reaching where neurons need it.
That gives scientists a specific mechanism to test and, if confirmed, a direction for developing new treatments for neurodegenerative disorders.
With @perturbai_tx and @scrippsresearch, a team of thousands of GPT-Rosalind agents worked in parallel, connecting gene-expression results from millions of mouse brain cells with scientific literature in just 32 minutes. We’re now expanding this collaboration to identify new drug targets.
openai.com/index/scaling-ai-…




























