Pinned Tweet
Excited to introduce ๐ถ EvoDuet: co-evolving web search and task solutions to push the state of the art across 8 scientific optimization tasks.
Young-Jun Lee retweeted
Huge congrats to @oq_35, @james_y_zou & co. for the launch of DiscoveryAgent! ๐
Agents can increasingly handle clearly defined scientific tasks (as weโre measuring with Terminal-Bench-Science). That frees researchers to focus on the work that leads to real scientific breakthroughs: asking the right questions, making sense of the results, and steering where the research goes next. Thatโs exactly what DiscoveryAgent enables.
arxiv.org/abs/2611.21950
Young-Jun Lee retweeted
Introducing FrontierML, a benchmark for evaluating AI agents on autonomous empirical ML R&D.
Most existing benchmarks for ML "research" give the agent a validation set or verifier that closely tracks its final score. In practice, human researchers rarely have that. Much of empirical research is deciding how to measure progress on a problem and how far to trust those measurements.
In FrontierML's 25 open-ended problems, agents have to operationalize progress through their own experiments, constructing local proxies and calibrating how far to trust them, while working toward an objective they never directly observe. Their submitted model or method is then scored on a held-out evaluation.
Young-Jun Lee retweeted
Welcome to the real world. Today weโre introducing FrontierEE Bench, the first benchmark to test frontier models on electrical engineering tasks at a physical lab bench.
Real-world engineering benchmarks should involve real hardware. At our lab, models control instrumentation, run experiments on physical hardware, and decide what to test next.
The best models scored 36.7% across a range of EE domains including mixed-signal circuits, power systems, & electromechanics.
Young-Jun Lee retweeted
Do not just trust your memory, and check where it came from. With WikiTrace, each lesson stays linked to its source run, so agents can catch a wrong lesson before it breaks a task they could already solve.
Proud to have worked on this with @yukyeongleee, @yumin_choi_, and @SungJuHwang1
Young-Jun Lee retweeted
Iโm very pleased to introduce AlphaGenome 2, @GoogleDeepMindโs next-generation genomics model that deciphers how DNA changes affect cells.
Cells are the basic units of life and key to our understanding of biology. It is an ongoing challenge to predict how DNA changes affect the functioning of different cells, such as for a heart muscle cell or an immune cell fighting an infection, and to uncover cellular drivers of disease.
To make progress on this challenge, we needed to design the architecture of AlphaGenome 2 so its predictions were conditional on cell types and states. This new design enables AlphaGenome 2 to predict variant effects across hundreds of human cell types and adapt to unseen cell and disease states without retraining.
When evaluated on the TenK10K immune cell dataset, AlphaGenome 2 demonstrated a major leap in accuracy over its predecessor. It reliably identified nearly twice as many causal variants (30% vs. 16%) and directional gene effects (43% vs. 23%) compared to the original AlphaGenome. This step-change improvement in precision helps researchers pinpoint previously overlooked disease-relevant variants with high confidence.
Using analysis on retrospective case studies, we were able to show AG2โs real world utility in disease understanding. A landmark study had previously shown that a variant in a distant regulatory switch drives irritable bowel disease by overactivating the ETS2 gene in gut macrophages. When used to make predictions across various immune cell types in the gut, AlphaGenome 2 singled out the exact causal variant and showed that it selectively boosts the ETS2 gene in gut macrophages, leaving other cell types and genes largely unaffected.
I would like to thank my colleagues Adam Gayoso @adamgayoso, ลฝiga Avsec @Avsecz, and our entire genomics team for their incredible contributions and dedication to bringing this model to life.
We are making AlphaGenome 2 available for non-commercial academic research via the AlphaGenome API, and soon we will release AlphaGenome 2 Skills in Google Antigravity. Iโm incredibly excited to see how scientists around the world use this model to make progress on biological understanding and treating disease.
Read more on our blog here:
deepmind.google/blog/alphageโฆ
Young-Jun Lee retweeted
Google published a paper that might end the PhD.
Itโs called โScientistTwoโ
For centuries, the bottleneck to human knowledge has been the human brain.
It takes five years of grueling labor to read the literature, find a gap, formulate a hypothesis, run the experiments, and write the paper.
But, Google automated the entire pipeline.
They built an AI that reads a problem, forms hypotheses, runs experiments, writes the paper, AND simulates its own peer review. no human touches it.
It reads millions of papers to find blind spots in the current science.
It generates novel hypotheses.
It designs the experiments. It writes the code to test them.
When the code fails, it debugs itself.
When the results come in, it analyzes the data, generates the charts, and writes the final academic paper.
It even simulates a brutal peer-review process to critique and improve its own work before it considers the project finished.
This isn't an LLM summarizing a PDF. This is a fully autonomous digital scientist.
What takes a human candidate half a decade of massive intellectual effort... the AI accomplishes in a fraction of the time, at a fraction of the cost.
The implications are staggering.
If you can spin up 10,000 autonomous researchers overnight, the rate of scientific discovery is about to go vertical. We are moving from artisanal, human-speed research to industrialized knowledge creation.
But it raises an uncomfortable reality for academia.
When an AI can read the literature, design the experiment, execute the code, and write the paper...
What exactly is the human supposed to do?
Young-Jun Lee retweeted
AI agents enter the wet lab.
In new @Nature N&V article, I discuss recent advances in agents running automated labs and ideas on how to make these systems more reliable. nature.com/articles/d41586-0โฆ
Young-Jun Lee retweeted
Iโm incredibly proud to share what weโve been building at the AItonomy Foundation: Scienceโs Last Exam (SLE). ๐งชโจ
Can AI do science? Instead of Q&A, we give agents actual research goals and real-world tools (lab data, software), then grade them against hidden expert checks.
Deeply grateful to the brilliant minds at 34 institutions (Harvard, MIT, Stanford, etc.) who helped craft 113 tasks across 9 disciplines. โค๏ธ
The results? Science is hard. Of 8 frontier models, the best scored 37.5/100. On true end-to-end research? All scored 0. We have an exciting journey ahead! ๐
(Claude Opus 5.5 & GPT-6 Astra coming soonโjust clearing bio safeguards).
Everything opens Nov 6. ๐๏ธ
๐ sciences-last-exam.org
๐ป github.com/Gen-Verse/Scienceโฆ
๐ค huggingface.co/datasets/Gen-โฆ
Thread below on how the exam works and where models break. ๐ (1/14)
#AGI #AIforScience #AIAgents #SciencesLastExam #OpenAI #Anthropic
Young-Jun Lee retweeted
What if you could synthesize an LLM inference engine customized for your model and Mac, just in time?
Meet JITEngine: custom engines built with SkyDiscover for Apple Silicon, up to 2.7ร faster than the fastest alternatives we tested.
jitengine.ai ๐งต (1/n)
Young-Jun Lee retweeted
There's a buzzy new paper about GPT-6 Astra being able to directly interpret model weights (prompts full of floating point numbers) - I tried it with Qwen 3.8 27B on my laptop and found that, after 45+ minutes, that model could achieve the same thing simonwillison.net/2026/Nov/3โฆ
HumanโAI collaboration continues well beyond the chat window. ๐ค
In Kolab, we collected 115+ hours of screen recordings from 50 professionals across 40 occupations working with AI agents on their own real-world problems.
Two things stood out:
- Frontier models struggle to predict whether people will adopt, modify, or reject an agentโs output.
- They often overestimate collaboration quality, and more workflow context doesnโt consistently help.
Really fun to be part of this one - check out the video and @shirley_ah_ 's thread below ๐
AI agents are getting better at automating long-horizon tasks. But in reality, we need agents that COLLABORATE with humans, not just complete tasks!
Introducing ๐ฅ Kolab: 115+ hours of human-agent collaboration in professional long-horizon workflows.
๐งต
21
1,004
Can AI agents find the paper you needโand tell you when no such paper exists? ๐
In CompLit, we test agents on how researchers actually search: with requirements that are implicit, grow across turns, or cannot be met by any existing paper.
Across all 8 arXiv domains:
- The strongest agents drop from 86โ87% accuracy to 66โ68% when requirements are implicit.
- When no matching paper exists, agents say so at most 53% of the time.
Really fun to be part of this one - check out @dayoon12161 's thread below ๐
๐ข ๐ต๐๐ ๐ท๐๐๐๐!
Have you ever asked an AI to find a paper and gotten something that looked ๐ณ๐ช๐จ๐ฉ๐ต but wasnโt? ๐คฏ
We find that this happens when requirements are ๐ช๐ฎ๐ฑ๐ญ๐ช๐ค๐ช๐ต, when they ๐ข๐ค๐ค๐ถ๐ฎ๐ถ๐ญ๐ข๐ต๐ฆ across turns, and when ๐ฏ๐ฐ ๐ฎ๐ข๐ต๐ค๐ฉ๐ช๐ฏ๐จ paper exists.
To evaluate these implicit, cumulative, and unmet needs, we propose ๐๐จ๐ฆ๐ฉ๐๐ข๐ญ๐, a scientific literature search benchmark covering all arXiv domains. On this benchmark, we find that search agents often find the right paper but fail to read it, or read it but still report a different paper that seems plausible but is wrong.
Why does this matter? Researchers search outward ๐ฑ, from what they have explored toward what they have not. As a result, they state their requirements implicitly, add to them over time, and often look for something no existing paper satisfies, because they are after novelty.
We analyze the agentsโ trajectories turn by turn to see where they go wrong, which can guide future improvements. Doing well on ๐๐จ๐ฆ๐ฉ๐๐ข๐ญ would be a first step toward trustworthy research agents.
Huge thanks to my mentors, co-authors, and advisor for their support! โค๏ธ
๐ Paper: arxiv.org/abs/2610.28658
๐ Project page: dayoon-ko.github.io/CompLit/
๐ป Code: github.com/dayoon-ko/CompLit
๐ค Dataset: huggingface.co/datasets/dayoโฆ
Young-Jun Lee retweeted
๐ข ๐ต๐๐ ๐ท๐๐๐๐!
Have you ever asked an AI to find a paper and gotten something that looked ๐ณ๐ช๐จ๐ฉ๐ต but wasnโt? ๐คฏ
We find that this happens when requirements are ๐ช๐ฎ๐ฑ๐ญ๐ช๐ค๐ช๐ต, when they ๐ข๐ค๐ค๐ถ๐ฎ๐ถ๐ญ๐ข๐ต๐ฆ across turns, and when ๐ฏ๐ฐ ๐ฎ๐ข๐ต๐ค๐ฉ๐ช๐ฏ๐จ paper exists.
To evaluate these implicit, cumulative, and unmet needs, we propose ๐๐จ๐ฆ๐ฉ๐๐ข๐ญ๐, a scientific literature search benchmark covering all arXiv domains. On this benchmark, we find that search agents often find the right paper but fail to read it, or read it but still report a different paper that seems plausible but is wrong.
Why does this matter? Researchers search outward ๐ฑ, from what they have explored toward what they have not. As a result, they state their requirements implicitly, add to them over time, and often look for something no existing paper satisfies, because they are after novelty.
We analyze the agentsโ trajectories turn by turn to see where they go wrong, which can guide future improvements. Doing well on ๐๐จ๐ฆ๐ฉ๐๐ข๐ญ would be a first step toward trustworthy research agents.
Huge thanks to my mentors, co-authors, and advisor for their support! โค๏ธ
๐ Paper: arxiv.org/abs/2610.28658
๐ Project page: dayoon-ko.github.io/CompLit/
๐ป Code: github.com/dayoon-ko/CompLit
๐ค Dataset: huggingface.co/datasets/dayoโฆ
Young-Jun Lee retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! ๐งต
Blog: rikiyat.github.io/blog/weighโฆ
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-spโฆ
github.com/RikiyaT/lm_undโฆ
huggingface.co/Qwen/Qwen3.5-0โฆ
huggingface.co/RikiyaT/Qwen3.โฆ
Young-Jun Lee retweeted
Rubric-based RL has become more common for non-verifiable tasks, but it can make models more verbose and prone to including unnecessary or incorrect information.
In our new work, FORK, we jointly optimize recall and precision to resolve this.
Check out @hyuncho_1905's post!
Young-Jun Lee retweeted
๐ฌ Can AI discover unknown scientific knowledge by itself?
Excited to introduce Principia: Test-Time Hypotheses-to-Principle Construction for Scientific Discovery.
Principia uses LLMs to turn experimental observations into scientific principles at test time:
๐ From observations to hypotheses
Compare related inputs and their outcomes to propose explanations for how specific characteristics contribute to the observed results.
๐ง From hypotheses to principles
Generalize recurring relationships across hypotheses into broader principles that guide subsequent predictions and candidate generation.
๐ Refine knowledge with new observations
Use new observations to revisit earlier hypotheses and reconstruct the principles built on them.
Huge thanks to my co-authors at @KAIST_AI: @SangwooPark0306, @jinheonbaek, @SoyeongJeong97, and @SungJuHwang1
๐ Paper: arxiv.org/abs/2610.32195
#AIforScience #ScientificDiscovery #LLM #TestTimeLearning #KnowledgeConstruction
Young-Jun Lee retweeted
๐ Terminal-Bench-Science is now featured in its first open-weight model release!
Kimi K3.1 Pro scores 38.1%, the strongest open-weight result on Terminal-Bench-Science to date.
Still, Terminal-Bench-Science shows BY FAR the biggest gap to SOTA (~30 points), a promising sign that itโs one of the most discriminative measures of frontier capability out there.
Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model.
๐น 1M-token context, 2.8T total / 104B active params (MoE), Multimodal.
๐น Full-project coding: games in Unreal, Unity & Godot.
๐น Excels at parametric 3D modeling in Blender & CAD.
๐น More reliable at finance work: Accurate modeling with hallucination down by 65%.
๐น Matching the strongest proprietary models across coding, agentic and knowledge benchmarks.
Kimi K3.1 Pro and K3.1 Pro Highspeed (2โ3ร faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API.
Open weights by Nov 4, 2026.
๐ API: platform.kimi.ai
๐ Tech blog: kimi.ai/blog/kimi-k3-1-pro
Great paper from Anthropic on AI agents doing ML-for-science research.
(bookmark it)
If you build AI-scientist systems, this one is worth your time.
They propose Agent4Sci, which gives frontier coding agents 20 scientific ML benchmarks across 11 domains and asks them to beat the published best model.
They beat it on all 20. Claude Opus improves on the published best by 10.4% on average, and no run was flagged for misconduct.
The gains come almost entirely from standard ML-engineering tricks. The trajectories show agents misjudging what is novel, leaving compute unused, and settling into small local sweeps.
Adding three lightweight skills for research ideation, experiment management, and scheduled checklists raises Claude Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%, with more varied and more domain-relevant changes.
The models stay the same, and the research scaffolding roughly doubles the gain.
arxiv.org/abs/2610.28763
Chat with Paper: academy.dair.ai/papers/automโฆ
Young-Jun Lee retweeted
๐ก (1/n) How do scientific ideas ๐๐๐ญ๐ฎ๐๐ฅ๐ฅ๐ฒ ๐๐ฏ๐จ๐ฅ๐ฏ๐ from an initial question to a published discovery?
We introduce the ๐๐๐ฌ๐๐๐ซ๐๐ก ๐๐ซ๐จ๐ ๐ซ๐๐ฌ๐ฌ ๐๐ซ๐๐ฉ๐ก (๐๐๐), a new representation to model ๐๐ฎ๐ญ๐ก๐๐ง๐ญ๐ข๐ ๐ซ๐๐ฌ๐๐๐ซ๐๐ก ๐ญ๐ซ๐๐ฃ๐๐๐ญ๐จ๐ซ๐ข๐๐ฌ that human experts do. It captures how ideas evolve through experiments, insights, failures, and revisionsโand crucially, the reasoning behind every transition.
Existing approaches either generate research ideas in a one-shot manner or rely on artificial workflows that capture only fragments of the RPG.
๐ arxiv.org/abs/2610.27337




































