Chief Tech Innovation Officer @UCBerkeley @berkeleycdss • Leadership Council @society4science • Board @cawaterdata @CAcradle2career • Ex-White House @Stanford
Berkeley, CA
Joined October 2008
- Tweets8.5K
- Following4.4K
- Followers3.1K
- Likes10.7K
Pinned Tweet
I'm back on Twitter/X, because of Measuring Agents in Production and all the amazing projects in our @UCBerkeley @BerkeleySky Lab! Check it out: berkeley.qualtrics.com/jfe/f… #AI #Research #Agents
The agent ecosystem is evolving rapidly and we are trying to capture a picture of where we are and where things are headed.
We're now collecting responses for the 2026 survey of agents.
berkeley.qualtrics.com/jfe/f…
Help us create a more complete picture of how agent systems are evolving, what challenges remain, and where future research may be most impactful.
Meredith Lee retweeted
Does your Claude model really need Claude Code…? 🤔
We evaluate 7 models on Claude Code, Codex, and Pi. Three surprising findings emerge:
1️⃣Harness choice has little effect on task success rate, but can significantly affect the cost
2️⃣A simple harness can be competitive
3️⃣The native harness isn’t always the best.
Millions of people are using coding agents, but the impact of harness choice remains unclear.
(1/n) More details in the thread. 🧵
Meredith Lee retweeted
If you're building production agents and want to contribute to research, take our survey! We'll release aggregate stats like we did in last year's MAP paper.
1/🤖Agents in production are changing fast.
Following our #ICML2026 oral work, we’re running the study again this year to understand what’s changing in how agents are built, deployed and evaluated in real systems.
📊 If you’re working on agents in production, we’d love your input. The survey is short, but it can help us build a much clearer picture of what is actually working in practice.
🔗 Survey: berkeley.qualtrics.com/jfe/f…
🎙️ We’re also interviewing practitioners! Reach out if you’d like to share your experience in more depth.
👇 And if you’re curious about what we learned last year, check out our previous study in the thread below
Meredith Lee retweeted
AI agents can write code, but can we trust the systems they build? With SkySynth, we co-evolve code with tests and formal proofs to generate specialized, just-in-time infrastructure. The result: formally verified KV stores 2.3× faster than Redis and custom inference engines with 2.2× the throughput of vLLM/SGLang!
Agents let us build systems for different workloads and requirements. But… can we trust what they build?
We release 🌟SkySynth🌟: an engine for synthesizing high-performance, just-in-time (JIT) systems we can trust, by co-evolving formal proofs and tests alongside the code.
Results:
💿 KV stores up to 2.3× faster than Redis and FASTER + formally verified stores with 2.9× Claude Code's pass rate
🚏 Model routers up to 48% cheaper than a general router
⚡ Specialized inference engine with 2.2× the throughput of vLLM/SGLang
🧵👇
Meredith Lee retweeted
The agent ecosystem is evolving rapidly and we are trying to capture a picture of where we are and where things are headed.
We're now collecting responses for the 2026 survey of agents.
berkeley.qualtrics.com/jfe/f…
Help us create a more complete picture of how agent systems are evolving, what challenges remain, and where future research may be most impactful.
Meredith Lee retweeted
Please consider taking this survey about real-world agents to contribute to open science and to help shape the future of agent research. Thank you!
Building agents? We'd love to hear from you 🤖
Take our survey to better understand how your agent's design compares with others in the community🧭 and help shape the future of agent research 🚀
By participating, you'll contribute to open science and help uncover common design choices, emerging shifts, and underexplored research opportunities 🌱As we did with MAP v1, we'll share the findings openly with the entire community🤝
🧵 Link in the thread!
Meredith Lee retweeted
As better models rapidly eat the stack, what system research problems will remain to enable the next frontier of data agents?
We studied the evolution of data agent capabilities the past 2 years, finding general coding agents beat hand-designed data agents by up to 37 points, with 4× fewer turns.
So what's actually left for system researchers to solve?
More than you might think...
📚 Paper: arxiv.org/abs/2609.03141
🧵👇
Meredith Lee retweeted
Our social systems and institutions confuse me all the time. 😕
AI may make them even more complicated. What if it also represents us inside them?
We already delegate representation constantly - your employer negotiates your insurance, your lawyer negotiates your contract, your representatives advocate for your interests.
Wrote up some early thinking on what financial markets can teach us here, what we found in our recent paper Cooperate to Compete (now accepted to EMNLP), and why one of the hardest open problems may be designing the structures agents negotiate inside rather than the agents themselves.
Read my first research blog below!! I would love to hear any thoughts on how folks are thinking about this.
Blog: abbykoneill.github.io/posts/…
Meredith Lee retweeted
A working group co-led by three experts at CDSS, @StanfordHAI and @CarnegieEndow submitted an evidence-backed report on frontier #AI policy to the Office of Governor @GavinNewsom, providing a policymaking framework for #California.
Read the release: bit.ly/3HLtCjd
Meredith Lee retweeted
Highlights from the Joint California Summit on Generative AI! 🌟
See how @BerkeleyDataSci, @StanfordHAI, @CAGoBiz, and @CAGovOps are leading the way in connecting cross-sector partners for groundbreaking #AI discussions & initiatives. #CaliforniaGenAI
🔗 youtu.be/nDMEAdkij20
ALT California Governor giving a speech with the Golden Gate Bridge in the background.
Meredith Lee retweeted
California is on the leading edge of technology, including the future of artificial intelligence.
Today, we brought together global leaders across fields — including tech, labor, government, and academia — to discuss how we can best use GenAI to serve Californians.
ALT Governor Newsom speaks at the Joint California Summit on Generative AI.
ALT Governor Newsom speaks at the Joint California Summit on Generative AI.
Meredith Lee retweeted
Today the Joint California Summit on #GenAI convened leaders across sectors to discuss how the state can best use AI to serve the public. The summit was hosted by @CAGoBiz, @CAGovOps, @BerkeleyDataSci, and @StanfordHAI. #CaliforniaGenAI
🌐 Learn more: cdss.berkeley.edu/news/gover…
Meredith Lee retweeted
“Educators are one of the early adopters of AI,” says Isabelle Hau of the @StanfordEd Accelerator for Learning during the education panel at the Joint California Summit on Generative AI. Watch the livestream here: youtube.com/watch?v=R-nn7311…
Meredith Lee retweeted
"I believe that this is an era that calls for large-scale cooperation, and if we can make that a constant commitment, the rest will naturally flow." - @sonyachristian, Chancellor of @CalCommColleges
Joint CA Summit on #GenAI: cdss.berkeley.edu/JointCASum…
Meredith Lee retweeted
Exciting News! @CAgovernor Gavin Newsom will be joining us at the Joint California Summit on Gen AI for a discussion with @BerkeleyDataSci dean @jenniferchayes and @StanfordHAI co-director @drfeifei. Tune in tomorrow at 9am PST: cdss.berkeley.edu/JointCASum…
Meredith Lee retweeted
Exciting panels await at the Joint California Summit on Generative AI! Explore the impact of #AI on California with voices from government, academia, industry, and civil society. View the full lineup: cdss.berkeley.edu/JointCASum…
#CaliforniaGenAI #CaliforniaInnovation
Meredith Lee retweeted
Mark your calendars! 🗓️
The Joint California Summit on Generative AI is happening 5/29. Don't miss this opportunity to engage in thought-provoking conversations about the intersection of AI & society.
Access the livestream, speaker line-up, & agenda: cdss.berkeley.edu/JointCASum…
Meredith Lee retweeted
Read more about CDSS' graduates and inaugural commencement ceremonies: cdss.berkeley.edu/news/cdss-…
Meredith Lee retweeted
“I need you to know this. Because you are here. Because you persevered. Because you made it to this moment… you are and forever will be part of Team Data. And the world needs you now more than ever,” said @dpatil, keynote speaker and first-ever US Chief Data Scientist.
Meredith Lee retweeted
More than 2,200 @UCBerkeley undergraduate students earned their degree in data science, computer science or statistics this year!🎉 🎓 At the Berkeley CDSS' two inaugural commencements on Thursday, speakers lauded the students’ resilience:
🌐 cdss.berkeley.edu/news/cdss-…
