I'm on the job market! PhD student at @StanfordNLP 🌲

Stanford, CA
Joined December 2011
I'm on the academic job market this year! Recently I've been working on scaling laws and human-centered interventions for building more socially grounded AI. If you're attending #COLM2026, please drop me a DM or email. I'd love to chat!
32
1
109
10,251
Caleb Ziems retweeted
As agents increasingly interact with each other, alignment becomes a property of the system, not just of individual agents. In our new work (@xinruishi11, @Diyi_Yang), we stress-test whether collusion can emerge through repeated interaction. Even when no agent is told to misbehave, we find that agents adapt to one another and can end up jointly violating their instructions.
Can collusion emerge between LLM agents through repeated interaction? We let pairs of agents work together. Neither is told to misbehave. Yet over time, many of them end up violating their instructions together to get a higher reward. 🧵
5
15
69
5,529
Caleb Ziems retweeted
When LLMs act as agents, they inevitably encounter large amounts of irrelevant context. Modern LLMs seem more capable than ever of ignoring such distractions. At the aggregate level, adding irrelevant context barely changes their accuracy. But beneath this apparent robustness, predictions on individual examples can still shift dramatically.
1
31
88
7,366
Caleb Ziems retweeted
AI is getting incredibly powerful, yet most of that power only reaches software domain. The next frontier isn't building more agents, but a collaboration layer between humans and AI. As a first step, we're open-sourcing Coco: a proactive co-assistant that help at the right time. Blog post: collaborative-agents.github.… Code: github.com/collaborative-age…
8
32
7
128
44,503
Caleb Ziems retweeted
EXCITED to share that I’ll be joining the University of Hong Kong (HKU) as an Assistant Professor in January 2027! I’m interested in AI agents 🤖, Human-AI Interaction 🤝 and their safety🛡️. Email me if you’d like to chat! I’ll also be at ACL from July 3–6! stevenyzzhang.github.io/webs…
53
38
18
326
91,186
Caleb Ziems retweeted
Does LLM really need to be a helpful assistant all the time? No. If you want to simulate people, “perfectly helpful” could be the wrong objective. Meet OdysSim, a journey toward LLMs beyond assistants, as behavioral foundation models (10B tokens of real human behavior; 23 sim benchmarks, finally in one place. new open models: outperform or on par with GPT-5.5, Gemini 3.1, or Claude Opus 4.7 in many behavior-sim dimensions). Human behavior simulation is becoming essential. Agent evaluation needs realistic users before real users show up. Medical and classroom training need realistic patients and students. Social science needs synthetic participants at scale. But real people are not ideal assistants. Real patients panic or ignore good advice. Real students misunderstand. Real customers are vague, picky, impatient, or simply leave. Human behavior is messy, diverse, and often imperfect. Frontier LLMs are getting better at math, code, and long-horizon tasks. They are NOT getting better at simulating human behavior. If anything, they drift the other way: more assistant-ish, more homogeneous, fewer of the errors and quirks real humans show. This is no accident. The whole pipeline is built for helpfulness and task success, not behavioral realism. And you can't prompt your way out of that. So we rethink the recipe from scratch and release: 🧠 The OdysSim corpus: 21.4M real human interactions (~10B tokens) from 62 sources, every conversation retrofitted with social grounding (who is talking, and why) 📏 SOUL-Index: 23 human-behavior benchmarks unified into one suite across 5 axes 🤖 OSim-8B: open weights; tops more SOUL-Index benchmarks than any frontier model, acts more like a real user than any of them on τ-bench (nearly matching real humans in the reaction dimension), and writes far more human-like text along the way.
14
76
13
474
143,895
People are increasingly worried that AI tools make us overreliant. But how do we actually measure this? We introduce Offloading Score, a measure of reliance based on the fraction of cognitive effort offloaded to AI while completing a task. In a controlled user study, Offloading Score detects increased reliance under time pressure, while several common alternatives do not. (1/9)
7
77
14
229
94,835
We're going live in 30 minutes, and we'd love to have you join 🎬 Joined by @dorazhao9 and @Diyi_Yang, I'll be talking about our new paper and discussing promising future directions for more human-centered LLM design! Stream it here: youtube.com/live/2d49pMXiJOA…
Join us this Thursday May 28th 6-7pm PST for our first ever AM Podcast Live Stream! 🎉 We are hosting @cjziems, @dorazhao9, and @Diyi_Yang for a discussion on their new paper "Reflections and New Directions for Human-Centered Large Language Models"! RSVP 🔗⬇️
4
8
6,573
Caleb Ziems retweeted
Raise your hand if you have heard about "human-centered" AI or LLMs and not been sure what that means? 🙋‍♀️🙋‍♀️🙋‍♀️ Check out our paper for answers :)
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :) With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
1
4
48
6,659
Caleb Ziems retweeted
Tired of AI automating research and solving major open problems? Take a read at Diyi’s new masterpiece to gain more confidence on humans!
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :) With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
1
4
1
42
5,385
Caleb Ziems retweeted
GOAT ASSEMBLY
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :) With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
1
26
6,062
Caleb Ziems retweeted
Caleb Ziems (@cjziems) has a new paper out!
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :) With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
1
5
33
4,301
Caleb Ziems retweeted
This is like the avengers but for human AI interaction
The next frontier of AI is not only more capable model; it is an AI that *humans* can meaningfully live and work with :) With all students in my cs329x Human-Centered LLM class, we present 60+ pages of insights for developing Human-Centered LLMs (HCLLMs), from design & data sourcing to training, eval & deployment 🧵
3
3
66
9,559
Caleb Ziems retweeted
Can you boost your AI review scores by asking an LLM to rewrite your paper? Yes! We call it paper laundering Our @icmlconf spotlight paper argues current AI reviewers aren't ready to automate peer review, and outlines what a science of peer review automation should look like🧵👇
14
80
12
483
77,712
Caleb Ziems retweeted
What’s the point of a “helpful assistant” if you have to always tell it what to do next? In a new paper, we introduce a reasoning model that predicts what you’ll do next over long contexts (LongNAP 💤). We trained it on 1,800 hours of computer use from 20 users. 🧵
17
83
19
323
126,954
Caleb Ziems retweeted
🚨Postdoc opening: We are looking for a postdoc researcher with expertise in NLP, RL, and/or ML to develop AI-powered clinical support tools for mental health counseling in the Global South. Working with @EmmaBrunskill & @Diyi_Yang at Stanford. Apply by April 15, 2026 via tinyurl.com/ai4mentalhealth-… 🧵👇
13
65
4
279
48,899
Happy to share 🥤SODA Can we pre-train a transformer — like LLM pre-training — to build an audio backbone where every audio-text task is just next-token prediction? We study design choices & scaling laws for discrete audio models, training up to 4B params on 500B tokens. [1/5]
5
32
4
138
21,714
Caleb Ziems retweeted
Announcing 🌇HumanLM, a RL framework that trains LLMs to simulate human users’ responses, along with 🌆Humanual, a comprehensive user simulation benchmark humanlm.stanford.edu/ 🌄 One thing that’s fascinating about our society: human users shape the world and determine the value of almost everything 👨‍💼 Human reactions reflect how justifiable policies are 👩‍🎨 Human preferences determine the popularity of blogs/products/media 👩‍💻 Human feedback evaluates LLMs and makes the best LLM collaborators 🌅If we know how to simulate users **accurately**, we know how things are evaluated and what the future looks like, and we can improve things in a way that like or can collaborate well with. So, meet HumanLM, our effort to enable a more human-centric future by simulating users.
28
106
11
606
134,437
Caleb Ziems retweeted
Two amazing postdocs from our lab are on the academic job market this year. I've learned a lot from their wonderful research -- you should definitely reach out and hire them!
2
30
5
141
42,131
Caleb Ziems retweeted
To kick off the new year, I am super excited to launch The Augmented Mind Podcast (@augmind_fm) with @shannonzshen and @michaelryan207 to share technical human-centered AI work! 🎙️ Since I started my PhD working on human-agent collaboration, I've always noticed this missing channel: - There are many channels sharing AI work, but you seldom see human-centered AI work there. - There are papers out there, but many careful thoughts are buried and we just have too many papers these days. - There are some interviews, but they often focus more on high-level visions. Those careful designs or technical innovations that align AI with human needs, values, etc. remain unseen. In The AM Podcast, we plan to share compelling research, infrastructure, and systems through long-form monthly episodes. We want to show examples of how to develop AI that collaborates and augments rather than purely automates or replaces. In EP0, we share who we are, why we started the podcast, and what we're looking forward to. Our first episode will drop this week!
AI used to be a distant promise; now it permeates our lives. AI is getting better, but is it making us better? We are promised that AI will augment our minds, but how? We--@EchoShao8899, @shannonzshen, and @michaelryan207--are excited to launch the Augmented Mind Podcast (The AM Podcast), a podcast about technical human-centered AI work. We'll share compelling research, infrastructure, and systems through monthly episodes, featuring interviews with the pioneering minds behind them. We release EP0 today to share who we are, why we started this podcast, and what we're looking forward to. 0:00 - Prelude: the problems we care about 1:48 - Host introduction 2:03 - Why we started the AM Podcast 2:31 - Hot takes on human-centered AI 10:45 - Format of our podcast 11:28 - Unique technical challenges in human-centered AI 16:45 - Let the journey begin!
2
17
1
79
16,854