Ben Reilly (@zdql__) profile picture

@GamowLabs AI and Genomics Researcher

Joined May 2024
Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)Photo by Ben Reilly (@zdql__)
I think it will be obvious in retrospect that OpenAI hasn't released any models because they have been and are currently assisting with a national security situation. Below, I gather the fact pattern and make some guesses. (a 🧵):
60
Turns out, frontier AI models can steer the workcell.
Google can't stop. @GoogleDeepMind Gemini 4 Hydrogen outperforms @AnthropicAI Opus 5.5 across most of our rare-disease benchmarks for a fraction of the price, lower latency, and faster throughput. Hydrogen vs. Opus 5.5 RareBench: 49.2% vs. 51.6% LabBench: 46.0% vs. 40.2% LabRobotBench: 31.7% vs. 23.0% ScreenBench: 70.2% vs. 47.2% And latency and throughput don't get talked about enough. The combination of Gemini 4 Hydrogen's speed and intelligence lets it operate as a real-time agent in the loop with our laboratory robotics. The video shows our workcell autonomously running an experiment with nothing more than a natural language prompt to Hydrogen. This performance is reflected in its very strong score on LabRobotBench as well. More information on all of these benchmarks is available at gamowlabs.com/benchmarks Congrats on another great launch, Big G!
4
168
Robotics RL is very hard, particularly because of how expensive robotics can be to operate. In high-consequence environments like a wet lab, performing many rollouts to find out what works is impossible. Because of this, we simulate. How do we prove realism in simulation? How can we ensure that the learning that occurs in simulation translates to real-world hardware? These are some of the most important questions to ask in robotics RL. We focus on this in our latest LabRobotBench update. As we iterate, we bring our simulations closer to reality, allowing us to 1. Learn faster, because we can rollout in simulation 2. Be more robust in production, because we've caught failures in simulation. Our latest blog post goes through our approach in detail.
7
7
1
210
10,132
Research highlights: 1. On a modestly difficult vision task, we can train a small image-net to match frontier vision performance at labelling. Somewhat unsurprising but good reminder. 2. We can actually hill-climb our simulator's realism, by using the aformentioned vision task. If an image-net trained on #1 in simulation performs well on real data, we can assume that our simulator is more realistic. We take advantage of this and are able to improve our simulator's realism by a meaningful margin.
4
409
oooh boy
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
1
140
gamowlabs.com/lab-robot-benc… Research update for LabRobotBench v0.2: One of the most important elements of robotics RL is the ability to simulate trials and ensure it holds up out of distribution. This blog post outlines the latest in our efforts to align this problem in our workcell.
2
5
401
looks like bogdanoff is back today
61
Drug development seems to be in the air, so we're proud to share the first @GamowLabs drug repurposing study. We screened every FDA-approved drug, looking for one that could help the child in @huggingface's Real Kid, Rare Disease hackathon. One of the most interesting hits: an HIV drug. We previously proposed that the child’s MVA was caused by biallelic loss-of-function variants in BUB1B, so we sought FDA-approved candidates to increase BUB1B expression. Public data couldn't answer that, so we ran our own screen of every FDA-approved compound and measured each one's effect on the transcriptome. Several compounds raised BUB1B. The most interesting is raltegravir, an HIV integrase inhibitor, which could be worth investigating. It's an early proof of concept, but the method is real and we'll be running it at scale soon! If you work on BUB1B, MVA, or chromosomal instability, we'd love to talk and understand how we could strengthen both these findings and the assay. gamowlabs.com/mva-hackathon-…
2
3
33
1,507
The @huggingface Carbon-X DNA model @LoubnaBenAllal1 trained reminded me of the early days of vision-language models. No one knew how to best fuse multiple modalities into the same reasoning backbone, but it turned out that training a small adapter to project images/speech into text space worked really well (LLaVa, Llama 3, etc.) I let Codex take over my 5090 today to few-shot this approach with DNA. I used @arcinstitute Evo2 7B as the frozen DNA encoder and Qwen3-14B as my LLM, which later received a task-specific LoRA. The results were really good! With 24h of cheap compute and occasional human intervention, I trained an adapter-LoRA approach that seems to outperform Carbon-X across the board on my (very reasonable) tasks (and match Astra, which when prompted correctly gets most of them right but isn't really suited for these tasks). I'm running on the BioReason, GenomeQA, and other Carbon-X benchmarks overnight and will report back tomorrow and vibe code everything into a @GamowLabs blog post for inspection, but I think this is a fruitful area of research and I want to see how far I can push an adapter approach with minimal compute resources.
3
7
2
26
7,051
Ben Reilly retweeted
By popular demand, we @GamowLabs have published a preview of our biological reasoning tasks at gamowlabs.com/tasks. The library spans wet lab robotics, drug design, genome analysis, and scientific reasoning. Our goal here is to evaluate models at biological reasoning and, via reinforcement learning, improve these capabilities. If you're interested in collaborating with us on this research, please reach out. Next week's new datasets will study compound screening and transcriptomics!
4
2
1
21
1,218
This has been so fun to work on. It's apparent to me that the future of most experimental science is robotic dark-factories working 24/7 towards discovery. We are sprinting at this at Gamow Labs!
Can frontier AI agents run a robotic wet lab? We built LabRobotBench to test how they operate equipment, manage timing, and recover interrupted experiments. LabRobotBench allows models to steer a simulation based on our real wet-lab setup in Boulder, CO! The best model tested, GPT 6.1 Sol, scores a 46% on our benchmark. Improving on LabRobotBench implies better robotic control and spatial awareness, as well as biological reasoning. read more in our latest blog post: gamowlabs.com/lab-robot-benc…
1
4
277
Excited to release our preprint for LabBench. This is a meaningful step in understanding how we can challenge agents to perform better in real-world experimental contexts.
Can AI agents decide what experiment to run next? We present LabBench, a benchmark built to challenge agents to choose and design experiments for wet-lab biology. Our core finding: agents struggle to commit to the next most-important experiment, despite considerable domain knowledge and expertise demonstrated in other parts of the eval.
5
394
Claude design is really, really good.
2
77
One of my favorite things about NYC is that there are consistent sh*t tests like this for people you can safely ignore / avoid. And then every once in a while, someone who's opinion you respect remarks that the bread is actually that good. And then you try the $60 bread, and suddenly you're the ratatouille critic having an out of body experience induced by expensive bread. And you say well okay, if you can really make bread that good you can sell it for the fair market value of $60. I don't have to eat it every day but at least I know it exists. Like the perfectly square japanese watermelons etc. And then 4 weeks later the next store opens and the cycle repeats.
I’m convinced New York is a social experiment. New bread store just opened (not a bakery, but a bread store): Rye I went in to grab a loaf, my friend and I took guesses on how much a loaf would cost I guessed $25 She guessed $80 based on the interior of the store $60 for a loaf of bread. There’s no way people with a fully developed frontal cortex are buying $60 loafs of bread
2
124
Crazy idea here but a prediction market / up-down markets for training runs could finance them. like BTC 15 min markets. This is horrible and degenerate, I wouldn't participate, but maybe some would!
I'm on my phone monitoring the MiMo v2.6 response length on coding-obg8 like a degenerate gambler.
4
214
Chat is this good?
4
174
The P(doom) stuff has broke containment...
4
254
Should have kept working on this! lol
We partnered with @baseten to post-train recursive language model (RLM) agents for M&A diligence. We found that model-harness co-optimization meaningfully improves agent performance in long-horizon environments, bringing us closer to agents that can complete M&A diligence end-to-end. We started by building an RLM harness that lets a root agent search a data room, delegate document review to sub-agents, and orchestrate the sub-agents' work to produce a final diligence memo. By distributing document review across sub-agents, the RLM harness can work through thousands of documents in data rooms containing up to 80M tokens. On LAB Diligence, our synthetic M&A diligence benchmark, the RLM harness raised average rubric criteria pass rate across models from 23% to 62%. Despite the root agent processing just 1-4% of the tokens in diligence (with sub-agents processing 96-99%), the choice of root agent had a much larger effect on performance than choice of sub-agents. This led us to focus on the root agent, training two models separately to improve diligence performance. First, we trained a GLM-5.2 root model in the RLM harness using self-distillation SFT. We selected GLM-5.2 diligence runs with high data room coverage and fine-tuned the model on these trajectories. This increased benchmark performance from 46% to 60%. Second, we trained a base Qwen3.5-122B-A10B using GRPO within the RLM harness, keeping the sub-agent models fixed. This raised rubric criteria pass rate from 30% to 63% on 50 held-out data rooms. In both cases we saw major qualitative shifts in agent behavior. The trained root agent learned to delegate more thoroughly and write its memo incrementally as sub-agent findings came back. Data room coverage rose from 62% to 96%. And the root agent learned to scale the size of its agent team proportionally to the size of the data room. We are now scaling RL training with GLM-5.3, a frontier open-weight model, as the root agent. A small amount of training gives an increase in pass rate from 51% to 59%. @nikogrupen walks through these experiments with more on harness design, training setup and results, and what we're exploring next:
2
114