Young-Jun Lee (@passing2961) profile picture

Applied Scientist @ Amazon | ex-Visiting Scholar @ UMN | ex-Ph.D, School of Computing, KAIST | ex-Amazon Applied Scientist Intern

Bellevue
Joined October 2022
Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)Photo by Young-Jun Lee (@passing2961)
Excited to introduce ๐ŸŽถ EvoDuet: co-evolving web search and task solutions to push the state of the art across 8 scientific optimization tasks.
1
9
3
38
5,816
Young-Jun Lee retweeted
Huge congrats to @oq_35, @james_y_zou & co. for the launch of DiscoveryAgent! ๐ŸŽ‰ Agents can increasingly handle clearly defined scientific tasks (as weโ€™re measuring with Terminal-Bench-Science). That frees researchers to focus on the work that leads to real scientific breakthroughs: asking the right questions, making sense of the results, and steering where the research goes next. Thatโ€™s exactly what DiscoveryAgent enables. arxiv.org/abs/2611.21950
1/ AI agents can now accomplish many research tasks. But tackling frontier problems requires deciding which questions matter and when to trust results. We're excited to introduce DiscoveryAgent to empower scientists in their pursuit of rigorous, open-ended discovery ๐Ÿงต
5
31
2,086
Introducing FrontierML, a benchmark for evaluating AI agents on autonomous empirical ML R&D. Most existing benchmarks for ML "research" give the agent a validation set or verifier that closely tracks its final score. In practice, human researchers rarely have that. Much of empirical research is deciding how to measure progress on a problem and how far to trust those measurements. In FrontierML's 25 open-ended problems, agents have to operationalize progress through their own experiments, constructing local proxies and calibrating how far to trust them, while working toward an objective they never directly observe. Their submitted model or method is then scored on a held-out evaluation.
64
67
48
606
68,618
Young-Jun Lee retweeted
Welcome to the real world. Today weโ€™re introducing FrontierEE Bench, the first benchmark to test frontier models on electrical engineering tasks at a physical lab bench. Real-world engineering benchmarks should involve real hardware. At our lab, models control instrumentation, run experiments on physical hardware, and decide what to test next. The best models scored 36.7% across a range of EE domains including mixed-signal circuits, power systems, & electromechanics.
97
55
15
550
51,444
Young-Jun Lee retweeted
Do not just trust your memory, and check where it came from. With WikiTrace, each lesson stays linked to its source run, so agents can catch a wrong lesson before it breaks a task they could already solve. Proud to have worked on this with @yukyeongleee, @yumin_choi_, and @SungJuHwang1
๐Ÿง  Memory can break what an agent already solves. A bad lesson from a past run can make an agent fail a task it could solve without memory. Excited to introduce WikiTrace! It links each lesson to the original run and its diagnosis so agents can check the evidence.
3
2
26
729
Young-Jun Lee retweeted
Iโ€™m very pleased to introduce AlphaGenome 2, @GoogleDeepMindโ€™s next-generation genomics model that deciphers how DNA changes affect cells. Cells are the basic units of life and key to our understanding of biology. It is an ongoing challenge to predict how DNA changes affect the functioning of different cells, such as for a heart muscle cell or an immune cell fighting an infection, and to uncover cellular drivers of disease. To make progress on this challenge, we needed to design the architecture of AlphaGenome 2 so its predictions were conditional on cell types and states. This new design enables AlphaGenome 2 to predict variant effects across hundreds of human cell types and adapt to unseen cell and disease states without retraining. When evaluated on the TenK10K immune cell dataset, AlphaGenome 2 demonstrated a major leap in accuracy over its predecessor. It reliably identified nearly twice as many causal variants (30% vs. 16%) and directional gene effects (43% vs. 23%) compared to the original AlphaGenome. This step-change improvement in precision helps researchers pinpoint previously overlooked disease-relevant variants with high confidence. Using analysis on retrospective case studies, we were able to show AG2โ€™s real world utility in disease understanding. A landmark study had previously shown that a variant in a distant regulatory switch drives irritable bowel disease by overactivating the ETS2 gene in gut macrophages. When used to make predictions across various immune cell types in the gut, AlphaGenome 2 singled out the exact causal variant and showed that it selectively boosts the ETS2 gene in gut macrophages, leaving other cell types and genes largely unaffected. I would like to thank my colleagues Adam Gayoso @adamgayoso, ลฝiga Avsec @Avsecz, and our entire genomics team for their incredible contributions and dedication to bringing this model to life. We are making AlphaGenome 2 available for non-commercial academic research via the AlphaGenome API, and soon we will release AlphaGenome 2 Skills in Google Antigravity. Iโ€™m incredibly excited to see how scientists around the world use this model to make progress on biological understanding and treating disease. Read more on our blog here: deepmind.google/blog/alphageโ€ฆ
38
164
15
1,348
45,023
Young-Jun Lee retweeted
holy shit, this is literally AGI.
During pre-release testing, Claude Fable 5.2 and Claude Mythos 5.2 worked on ambitious projects in many different fields. Here are a few. From a one-paragraph prompt, Fable 5.2 designed three working electronic devices, including this home voice assistant hub.
418
496
111
11,104
1,472,624
Young-Jun Lee retweeted
Google published a paper that might end the PhD. Itโ€™s called โ€œScientistTwoโ€ For centuries, the bottleneck to human knowledge has been the human brain. It takes five years of grueling labor to read the literature, find a gap, formulate a hypothesis, run the experiments, and write the paper. But, Google automated the entire pipeline. They built an AI that reads a problem, forms hypotheses, runs experiments, writes the paper, AND simulates its own peer review. no human touches it. It reads millions of papers to find blind spots in the current science. It generates novel hypotheses. It designs the experiments. It writes the code to test them. When the code fails, it debugs itself. When the results come in, it analyzes the data, generates the charts, and writes the final academic paper. It even simulates a brutal peer-review process to critique and improve its own work before it considers the project finished. This isn't an LLM summarizing a PDF. This is a fully autonomous digital scientist. What takes a human candidate half a decade of massive intellectual effort... the AI accomplishes in a fraction of the time, at a fraction of the cost. The implications are staggering. If you can spin up 10,000 autonomous researchers overnight, the rate of scientific discovery is about to go vertical. We are moving from artisanal, human-speed research to industrialized knowledge creation. But it raises an uncomfortable reality for academia. When an AI can read the literature, design the experiment, execute the code, and write the paper... What exactly is the human supposed to do?
91
318
72
1,417
148,003
Young-Jun Lee retweeted
AI agents enter the wet lab. In new @Nature N&V article, I discuss recent advances in agents running automated labs and ideas on how to make these systems more reliable. nature.com/articles/d41586-0โ€ฆ
35
186
17
782
51,146
Iโ€™m incredibly proud to share what weโ€™ve been building at the AItonomy Foundation: Scienceโ€™s Last Exam (SLE). ๐Ÿงชโœจ Can AI do science? Instead of Q&A, we give agents actual research goals and real-world tools (lab data, software), then grade them against hidden expert checks. Deeply grateful to the brilliant minds at 34 institutions (Harvard, MIT, Stanford, etc.) who helped craft 113 tasks across 9 disciplines. โค๏ธ The results? Science is hard. Of 8 frontier models, the best scored 37.5/100. On true end-to-end research? All scored 0. We have an exciting journey ahead! ๐Ÿš€ (Claude Opus 5.5 & GPT-6 Astra coming soonโ€”just clearing bio safeguards). Everything opens Nov 6. ๐Ÿ—“๏ธ ๐ŸŒ sciences-last-exam.org ๐Ÿ’ป github.com/Gen-Verse/Scienceโ€ฆ ๐Ÿค— huggingface.co/datasets/Gen-โ€ฆ Thread below on how the exam works and where models break. ๐Ÿ‘‡ (1/14) #AGI #AIforScience #AIAgents #SciencesLastExam #OpenAI #Anthropic
57
54
35
415
658,112
Young-Jun Lee retweeted
What if you could synthesize an LLM inference engine customized for your model and Mac, just in time? Meet JITEngine: custom engines built with SkyDiscover for Apple Silicon, up to 2.7ร— faster than the fastest alternatives we tested. jitengine.ai ๐Ÿงต (1/n)
23
43
15
274
50,607
Young-Jun Lee retweeted
There's a buzzy new paper about GPT-6 Astra being able to directly interpret model weights (prompts full of floating point numbers) - I tried it with Qwen 3.8 27B on my laptop and found that, after 45+ minutes, that model could achieve the same thing simonwillison.net/2026/Nov/3โ€ฆ
421
444
113
5,433
342,981
Humanโ€“AI collaboration continues well beyond the chat window. ๐Ÿค In Kolab, we collected 115+ hours of screen recordings from 50 professionals across 40 occupations working with AI agents on their own real-world problems. Two things stood out: - Frontier models struggle to predict whether people will adopt, modify, or reject an agentโ€™s output. - They often overestimate collaboration quality, and more workflow context doesnโ€™t consistently help. Really fun to be part of this one - check out the video and @shirley_ah_ 's thread below ๐Ÿ‘‡
AI agents are getting better at automating long-horizon tasks. But in reality, we need agents that COLLABORATE with humans, not just complete tasks! Introducing ๐Ÿ”ฅ Kolab: 115+ hours of human-agent collaboration in professional long-horizon workflows. ๐Ÿงต
21
1,004
Can AI agents find the paper you needโ€”and tell you when no such paper exists? ๐Ÿ”Ž In CompLit, we test agents on how researchers actually search: with requirements that are implicit, grow across turns, or cannot be met by any existing paper. Across all 8 arXiv domains: - The strongest agents drop from 86โ€“87% accuracy to 66โ€“68% when requirements are implicit. - When no matching paper exists, agents say so at most 53% of the time. Really fun to be part of this one - check out @dayoon12161 's thread below ๐Ÿ‘‡
๐Ÿ“ข ๐‘ต๐’†๐’˜ ๐‘ท๐’‚๐’‘๐’†๐’“! Have you ever asked an AI to find a paper and gotten something that looked ๐˜ณ๐˜ช๐˜จ๐˜ฉ๐˜ต but wasnโ€™t? ๐Ÿคฏ We find that this happens when requirements are ๐˜ช๐˜ฎ๐˜ฑ๐˜ญ๐˜ช๐˜ค๐˜ช๐˜ต, when they ๐˜ข๐˜ค๐˜ค๐˜ถ๐˜ฎ๐˜ถ๐˜ญ๐˜ข๐˜ต๐˜ฆ across turns, and when ๐˜ฏ๐˜ฐ ๐˜ฎ๐˜ข๐˜ต๐˜ค๐˜ฉ๐˜ช๐˜ฏ๐˜จ paper exists. To evaluate these implicit, cumulative, and unmet needs, we propose ๐‚๐จ๐ฆ๐ฉ๐‹๐ข๐ญ๐Ÿ”Ž, a scientific literature search benchmark covering all arXiv domains. On this benchmark, we find that search agents often find the right paper but fail to read it, or read it but still report a different paper that seems plausible but is wrong. Why does this matter? Researchers search outward ๐ŸŒฑ, from what they have explored toward what they have not. As a result, they state their requirements implicitly, add to them over time, and often look for something no existing paper satisfies, because they are after novelty. We analyze the agentsโ€™ trajectories turn by turn to see where they go wrong, which can guide future improvements. Doing well on ๐‚๐จ๐ฆ๐ฉ๐‹๐ข๐ญ would be a first step toward trustworthy research agents. Huge thanks to my mentors, co-authors, and advisor for their support! โค๏ธ ๐Ÿ“„ Paper: arxiv.org/abs/2610.28658 ๐ŸŒ Project page: dayoon-ko.github.io/CompLit/ ๐Ÿ’ป Code: github.com/dayoon-ko/CompLit ๐Ÿค— Dataset: huggingface.co/datasets/dayoโ€ฆ
1
1
1
17
916
Young-Jun Lee retweeted
๐Ÿ“ข ๐‘ต๐’†๐’˜ ๐‘ท๐’‚๐’‘๐’†๐’“! Have you ever asked an AI to find a paper and gotten something that looked ๐˜ณ๐˜ช๐˜จ๐˜ฉ๐˜ต but wasnโ€™t? ๐Ÿคฏ We find that this happens when requirements are ๐˜ช๐˜ฎ๐˜ฑ๐˜ญ๐˜ช๐˜ค๐˜ช๐˜ต, when they ๐˜ข๐˜ค๐˜ค๐˜ถ๐˜ฎ๐˜ถ๐˜ญ๐˜ข๐˜ต๐˜ฆ across turns, and when ๐˜ฏ๐˜ฐ ๐˜ฎ๐˜ข๐˜ต๐˜ค๐˜ฉ๐˜ช๐˜ฏ๐˜จ paper exists. To evaluate these implicit, cumulative, and unmet needs, we propose ๐‚๐จ๐ฆ๐ฉ๐‹๐ข๐ญ๐Ÿ”Ž, a scientific literature search benchmark covering all arXiv domains. On this benchmark, we find that search agents often find the right paper but fail to read it, or read it but still report a different paper that seems plausible but is wrong. Why does this matter? Researchers search outward ๐ŸŒฑ, from what they have explored toward what they have not. As a result, they state their requirements implicitly, add to them over time, and often look for something no existing paper satisfies, because they are after novelty. We analyze the agentsโ€™ trajectories turn by turn to see where they go wrong, which can guide future improvements. Doing well on ๐‚๐จ๐ฆ๐ฉ๐‹๐ข๐ญ would be a first step toward trustworthy research agents. Huge thanks to my mentors, co-authors, and advisor for their support! โค๏ธ ๐Ÿ“„ Paper: arxiv.org/abs/2610.28658 ๐ŸŒ Project page: dayoon-ko.github.io/CompLit/ ๐Ÿ’ป Code: github.com/dayoon-ko/CompLit ๐Ÿค— Dataset: huggingface.co/datasets/dayoโ€ฆ
6
5
2
62
3,498
Young-Jun Lee retweeted
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! ๐Ÿงต Blog: rikiyat.github.io/blog/weighโ€ฆ
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-spโ€ฆ github.com/RikiyaT/lm_undโ€ฆ huggingface.co/Qwen/Qwen3.5-0โ€ฆ huggingface.co/RikiyaT/Qwen3.โ€ฆ
2,368
7,255
5,755
55,917
10,090,527
Young-Jun Lee retweeted
Rubric-based RL has become more common for non-verifiable tasks, but it can make models more verbose and prone to including unnecessary or incorrect information. In our new work, FORK, we jointly optimize recall and precision to resolve this. Check out @hyuncho_1905's post!
๐Ÿฅฆ How can we mitigate reward hacking in rubric-based RL? We introduce FORK, a simple reward that balances standard rubric coverage with response precisionโ€”the proportion of a response that is both necessary and factually correct.
1
4
47
3,462
Young-Jun Lee retweeted
๐Ÿ”ฌ Can AI discover unknown scientific knowledge by itself? Excited to introduce Principia: Test-Time Hypotheses-to-Principle Construction for Scientific Discovery. Principia uses LLMs to turn experimental observations into scientific principles at test time: ๐Ÿ” From observations to hypotheses Compare related inputs and their outcomes to propose explanations for how specific characteristics contribute to the observed results. ๐Ÿง  From hypotheses to principles Generalize recurring relationships across hypotheses into broader principles that guide subsequent predictions and candidate generation. ๐Ÿ”„ Refine knowledge with new observations Use new observations to revisit earlier hypotheses and reconstruct the principles built on them. Huge thanks to my co-authors at @KAIST_AI: @SangwooPark0306, @jinheonbaek, @SoyeongJeong97, and @SungJuHwang1 ๐Ÿ“„ Paper: arxiv.org/abs/2610.32195 #AIforScience #ScientificDiscovery #LLM #TestTimeLearning #KnowledgeConstruction
15
2
36
2,795
Young-Jun Lee retweeted
๐Ÿš€ Terminal-Bench-Science is now featured in its first open-weight model release! Kimi K3.1 Pro scores 38.1%, the strongest open-weight result on Terminal-Bench-Science to date. Still, Terminal-Bench-Science shows BY FAR the biggest gap to SOTA (~30 points), a promising sign that itโ€™s one of the most discriminative measures of frontier capability out there.
Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model. ๐Ÿ”น 1M-token context, 2.8T total / 104B active params (MoE), Multimodal. ๐Ÿ”น Full-project coding: games in Unreal, Unity & Godot. ๐Ÿ”น Excels at parametric 3D modeling in Blender & CAD. ๐Ÿ”น More reliable at finance work: Accurate modeling with hallucination down by 65%. ๐Ÿ”น Matching the strongest proprietary models across coding, agentic and knowledge benchmarks. Kimi K3.1 Pro and K3.1 Pro Highspeed (2โ€“3ร— faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API. Open weights by Nov 4, 2026. ๐Ÿ”— API: platform.kimi.ai ๐Ÿ”— Tech blog: kimi.ai/blog/kimi-k3-1-pro
10
11
2
110
4,002
Young-Jun Lee retweeted
Great paper from Anthropic on AI agents doing ML-for-science research. (bookmark it) If you build AI-scientist systems, this one is worth your time. They propose Agent4Sci, which gives frontier coding agents 20 scientific ML benchmarks across 11 domains and asks them to beat the published best model. They beat it on all 20. Claude Opus improves on the published best by 10.4% on average, and no run was flagged for misconduct. The gains come almost entirely from standard ML-engineering tricks. The trajectories show agents misjudging what is novel, leaving compute unused, and settling into small local sweeps. Adding three lightweight skills for research ideation, experiment management, and scheduled checklists raises Claude Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%, with more varied and more domain-relevant changes. The models stay the same, and the research scaffolding roughly doubles the gain. arxiv.org/abs/2610.28763 Chat with Paper: academy.dair.ai/papers/automโ€ฆ
34
38
3
271
17,991
Young-Jun Lee retweeted
๐Ÿ’ก (1/n) How do scientific ideas ๐š๐œ๐ญ๐ฎ๐š๐ฅ๐ฅ๐ฒ ๐ž๐ฏ๐จ๐ฅ๐ฏ๐ž from an initial question to a published discovery? We introduce the ๐‘๐ž๐ฌ๐ž๐š๐ซ๐œ๐ก ๐๐ซ๐จ๐ ๐ซ๐ž๐ฌ๐ฌ ๐†๐ซ๐š๐ฉ๐ก (๐‘๐๐†), a new representation to model ๐š๐ฎ๐ญ๐ก๐ž๐ง๐ญ๐ข๐œ ๐ซ๐ž๐ฌ๐ž๐š๐ซ๐œ๐ก ๐ญ๐ซ๐š๐ฃ๐ž๐œ๐ญ๐จ๐ซ๐ข๐ž๐ฌ that human experts do. It captures how ideas evolve through experiments, insights, failures, and revisionsโ€”and crucially, the reasoning behind every transition. Existing approaches either generate research ideas in a one-shot manner or rely on artificial workflows that capture only fragments of the RPG. ๐Ÿ“„ arxiv.org/abs/2610.27337
6
25
4
180
10,279