LLMs are increasingly used to judge whether AI-generated research ideas are novel. But are they any good at it? Our new eval reveals even frontier LLMs can reach different conclusions about similar ideas depending on how they're asked to evaluate them. 🧵 github.com/allenai/novelty-e…

Come meet Ai2 at #COLM2026 👇 We’ve got researchers from across Ai2 here this week talking open models, agents, AI for science, & more. Find us around the conference and at the Ai2 booth (302).

AI’s biggest role in science may not be answering questions. It may be helping scientists find which questions are worth asking. At @ProvSwedish, AutoDiscovery surfaced an unexpected immune signal in a heavily studied cancer dataset—and follow-up research confirmed it. 🧵 allenai.org/blog/swedish-aut…

Find papers is Asta's literature search. Its new default mode, Deep search, interprets your query, evaluates whether the results actually answer it, & keeps searching if they don't.

We built SciArena to test how well AI models handle scientific literature questions, as judged by researchers. It's retiring July 15, and the results are in: ~1,700 users cast ~3,900 votes. Here's what they told us. 🧵

LLMs are no longer created w/ human data alone. They rely on other models to generate & filter data, evaluate outputs, & guide dev work. So what is a modern LLM built on? Olmo 3 → 89 model + 183 dataset dependencies; Nemotron 3 → 273 + 560 We made ModSleuth to trace this. 🧵

Now available in AstaLabs in limited research preview: MyScholarQA, a personalized version of ScholarQA for scientific deep research. ScholarQA helps synthesize evidence from 12M+ open-access papers. MyScholarQA adds user profiles to tailor that synthesis to you. 🧵

Today we’re bringing new NSF OMAI compute online with NVIDIA Blackwell Ultra-powered systems, turning a $152M national investment from @NSF & @NVIDIA into a foundation for truly open AI research. 🧵

Interim CEO Peter Clark shares his thoughts on this moment for Ai2, our commitment to open science, and where the institute is headed next. 👇 You've been part of Ai2 for many years, including as Interim CEO before. What feels different about this moment? What feels different now is the incredible pace of progress in AI—and the risk that, in that acceleration, we lose sight of some of the longer-term work that really matters. Ai2 was created to take that longer-horizon view. From the beginning, Paul Allen's vision was to advance AI in ways that push science forward while also delivering meaningful benefit to the world, and, critically, doing it in the open. That's a commitment that's become even more important in the current landscape. So for me, stepping back into this role is about renewing our commitment to that mission, and maintaining a sharpened focus on long-term, high-impact research and engineering. There's a real sense of momentum here internally. Teams are coming together, new ideas are emerging, and there's a lot of energy around what comes next. Where has that mission shown up in the projects at Ai2? It’s always been a combination of deep research and real-world impact. Early on, we helped set the stage for the LLM revolution with our project ELMo, and more recently, that work turned into Olmo and Molmo, along with new training approaches like FlexOlmo. All these projects show that you can be open and still operate at the frontier. At the same time, we’ve been applying AI in meaningful ways. Systems like AutoDiscovery are already changing how oncologists think about treatments for certain types of cancer, and efforts like OlmoEarth are helping us better understand complex Earth systems. That connection between advancing the science of AI and actually deploying these tools is really central to how we operate. Where do you think Ai2 is doing something meaningfully different? A lot of the work we focus on doesn’t fit neatly into existing incentives. Some of it is just longer-term. It’s more exploratory, where the path to impact is real but not immediate. And some of it is about openness and building in a way that others can actually understand and expand on. As a nonprofit, we have the flexibility to do that. We can spend time on problems that need sustained attention, even if they’re not tied to short-term outcomes. It also shapes the kind of environment we’re trying to create. People have the space to think long-term, to explore ideas that aren’t fully formed yet, and to work in the open—while still aiming toward something that has real impact. How do you think about the role of open models in Ai2’s work? Open models remain fundamental to what we do. They matter not just because they improve access, but because they enable a deeper scientific understanding of how these systems work. When models are open, the broader community can study them, build on them, and push the field forward more quickly. That’s also a big part of what’s driving our NSF OMAI work. The U.S. National Science Foundation and NVIDIA invested in us to create the next generation of open model development by bringing together the compute, infrastructure, and research needed to build systems that are fully open and transparent. We’re excited to share an update on the OMAI project very soon as our teams get to work developing the next generation of models and foundational research that will come from it. Where is the Institute planning to focus its time and energy going forward? There are a few areas we’re leaning into. One is continuing to advance the science of AI systems, especially around understanding how models behave and how to make them more reliable. There’s still a lot we don’t fully understand there, and industry-wide, too much of that work is happening behind closed doors. The NSF OMAI project I mentioned earlier is a big part of this. Another is AI for scientific research and discovery. We’re building Asta, an agentic ecosystem that can help researchers generate hypotheses, connect ideas, and move faster. Tools within Asta such as ScholarQA, AutoDiscovery, and Theorizer exemplify this direction, helping scientists by analyzing the scientific literature, discovering surprising findings in data, and positing explanatory theories. Something we’re also really excited to keep advancing is embodied AI. There’s a lot of interesting work happening at the intersection of language models and physical systems, and some of our early efforts like MolmoAct and MolmoBot are starting to explore what it means to build more general, adaptable systems that can operate in the real world, and we have a lot more to come this year. And then there’s our AI for the planet work—including the environment, conservation, and global systems. These are areas where our work is already having real, long-term impact, and we see huge potential for AI to help those on the ground make an even greater impact as platforms like OlmoEarth grow and expand. How does all of this translate to real-world impact? We’ve always tried to connect the research to something that can actually be used. In practice, that means moving across a spectrum from fundamental research to early prototypes to systems with real-world applications. And often, the interesting part is how those pieces connect. AutoDiscovery is a good example. It began as a research system for automated, open-ended scientific discovery and is now available as a managed solution where researchers can upload structured datasets, generate and test hypotheses, and inspect the code and statistical analysis behind each result. It’s not a linear path, but the progression from exploration to application is where a lot of the impact actually happens. Looking ahead—what kind of future is Ai2 working to build, and how can people be part of it? We’re working toward a future where AI is both more deeply understood and more broadly useful. That means continuing to make systems more transparent and reliable, while also applying those advances in ways that have real impact – whether that’s accelerating scientific discovery or addressing challenges in areas like the environment and global systems. The opportunity is to do both at once, and to do it in the open. And that really resonates with a certain kind of person: those who want to work on ambitious problems, who are motivated by impact, and who value being part of a broader scientific effort. That’s the kind of community we’re continuing to build.

New AstaBench results show frontier models making progress on scientific research, but the benchmark remains far from solved. Claude Opus 4.7 leads overall at 58.0%, while GPT-5.5 comes within 5.1 points at less than half the measured cost per problem. 🧵

🚨 The best AI gets built in the open. Next week, we’re bringing that message to #NVIDIAGTC — with panels, demos, and a window into what fully open models can do. Here's where to find us 🧵👇

Knowing which questions to ask is often the hardest part of science. Today we're releasing AutoDiscovery in AstaLabs, an AI system that starts with your data and generates its own hypotheses. 🧪