
Assistant Professor #VT_ECE. AI safety & alignment for agents that work with people, satisfy domain specs, and handle real-world constraints. PhD #EECS_Berkeley
Joined June 2021
- Tweets251
- Following193
- Followers391
- Likes741
Quick time update: Today's Agentic AI Frontier Seminar with Prof. Kaiqing Zhang starts at 12:30 PM ET / 9:30 AM PT (not 11 AM ET). apologies for the earlier typo, and see you on the same Zoom link shortly!
LLMs weren't trained for sequential decision-making—so when are LLM agents actually no-regret, and when do they fail?
Excited to host Prof. Kaiqing Zhang (UMD) @KaiqingZhang at the Agentic AI Frontier Seminar! He’ll dive into decision-making agents through the lens of regret:
• When off-the-shelf LLMs achieve vanishing regret (and where adversarial/variable losses break them)
• RMFT: Regret-minimization fine-tuning without optimal action labels
• RegPO: Step-level local regret credit assignment so good early actions aren't penalized for later mistakes—boosting web, household & robotics benchmarks
📅 Wed, Nov 18 | 11 AM ET / 8 AM PT
🔗 Zoom & info: agentic-ai-frontier-seminar.…
LLMs weren't trained for sequential decision-making—so when are LLM agents actually no-regret, and when do they fail?
Excited to host Prof. Kaiqing Zhang (UMD) @KaiqingZhang at the Agentic AI Frontier Seminar! He’ll dive into decision-making agents through the lens of regret:
• When off-the-shelf LLMs achieve vanishing regret (and where adversarial/variable losses break them)
• RMFT: Regret-minimization fine-tuning without optimal action labels
• RegPO: Step-level local regret credit assignment so good early actions aren't penalized for later mistakes—boosting web, household & robotics benchmarks
📅 Wed, Nov 18 | 11 AM ET / 8 AM PT
🔗 Zoom & info: agentic-ai-frontier-seminar.…
Happening in ~1hour!
Excited to host Prof. Csaba Szepesvári @CsabaSzepesvari (University of Alberta & DeepMind, co-inventor of UCT) for our next Agentic AI Frontier Seminar!
Topic: Allocating Computation Between Solving and Verification
🗓️ When: Tomorrow! Wednesday, Oct 21 | 11:00 AM ET (8:00 AM PT)
🔗 Join us: virginiatech.zoom.us/j/87872…
📅 Schedule: agentic-ai-frontier-seminar.…
When reasoning systems generate candidate solutions and check them, how should test-time compute be allocated? Prof. Szepesvári will explore a stochastic solver–verifier model to return correct solutions with high probability while minimizing expected computation.
Join us for a deep dive into the theoretical foundations of reasoning and verification!
#AgenticAI #ReinforcementLearning #Reasoning #MachineLearning #TestTimeCompute Organizers: @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
2
8
385
Excited to host Prof. Csaba Szepesvári @CsabaSzepesvari (University of Alberta & DeepMind, co-inventor of UCT) for our next Agentic AI Frontier Seminar!
Topic: Allocating Computation Between Solving and Verification
🗓️ When: Tomorrow! Wednesday, Oct 21 | 11:00 AM ET (8:00 AM PT)
🔗 Join us: virginiatech.zoom.us/j/87872…
📅 Schedule: agentic-ai-frontier-seminar.…
When reasoning systems generate candidate solutions and check them, how should test-time compute be allocated? Prof. Szepesvári will explore a stochastic solver–verifier model to return correct solutions with high probability while minimizing expected computation.
Join us for a deep dive into the theoretical foundations of reasoning and verification!
#AgenticAI #ReinforcementLearning #Reasoning #MachineLearning #TestTimeCompute Organizers: @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
🚨 Excited to host Prof. Zhuoran Yang @zhuoran_yang from Yale University for our next Agentic AI Frontier Seminar!
Topic: On the Interpretability and Self-Evolution of LLM Agents
🗓️ When: Wednesday, Oct 14 | 11:00 AM ET
🔗 Details & Schedule: agentic-ai-frontier-seminar.…
Prof. Yang will explore two critical frontiers for LLM agents: 🔍 Mechanistic Interpretability: Decoding the micro-mechanics of the agent loop (including circuit analysis on a 1-layer Transformer solving extreme Sudoku!) 🧬 Scalable Self-Evolution: Introducing INFUSER, a co-training framework where generators and solvers co-evolve, allowing an 8B model to outperform frozen 32B models.
Join us for a deep dive into the architecture and evolution of autonomous agents!
#AgenticAI #LLMs #MachineLearning #MechanisticInterpretability
Organizers: @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
7
735
🚨 Excited to host Prof. Sayan Mitra (Director of the Center for Autonomy, UIUC) @Mitrasayn for the Agentic AI Frontier Seminar!
Topic: Formal Verification of Visual Autonomy: Perception Contracts & Abstract Rendering
🗓️ When: Wed, Sep 30 | 8 AM PT | 11 AM ET
🔗 Join us: virginiatech.zoom.us/j/87872…
📅 Full Schedule: agentic-ai-frontier-seminar.…
How can we formally verify autonomous systems that rely on learned vision? Sayan will explore how to prove closed-loop safety and propagate uncertainty for AI agents acting in the physical world.
Key topics include: 🔹 Lyapunov perception contracts for vision pipeline error bounds 🔹 Abstract rendering for certifying downstream decisions 🔹 Real-world applications in automated landing & formation flight
Join us for a deep dive into formal assurance for visual autonomy! 👇
#FormalVerification #AutonomousSystems #AgenticAI #ComputerVision Organizers: @MingJin_AI @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang
4
216
Huge thanks to @p_song1 for an incredible guest lecture! The interactive Lean walkthrough perfectly bridged the gap between theory and practice for the class. Highly recommend checking out his slides below if you are interested in verifiable AI and formal methods. 🚀
Had a wonderful time guest lecturing at Virginia Tech’s Machine Learning course (jinming.tech/learn-ml-by-bui…). Thank you Prof. @MingJin_AI for the invitation!
The topic was Formal Reasoning in AI: From Next-Step Prediction to Verifiable Reasoning Agents. I’ve always been super excited about formal reasoning + AI, but putting these ideas into an educational setting was a new and surprisingly fun challenge: how do we connect LLM reasoning failures, neuro-symbolic systems, Lean, search, human-AI collaboration, adaptation, and verified software into one coherent story, and then let students build part of it themselves?
Really enjoyed the discussion and questions. Hope it was fun for the students as well!
Slides: peiyang-song.github.io/data/…
🚨 Excited to host Saman Zonouz (Associate Professor at Georgia Tech) @szonouz for our upcoming Agentic AI Frontier Seminar!
Topic: Security of Engineered Systems: Engineering Resilient Interdependent Infrastructures
🗓️ When: Wed, Sep 23 | 8 AM PT | 11 AM ET
🔗 Join us: virginiatech.zoom.us/j/87872…
📅 Full Schedule: agentic-ai-frontier-seminar.…
The failure of critical infrastructures like power grids, transportation, and manufacturing would have debilitating impacts on economic security and public safety. Saman will discuss real-world solutions for protecting these systems against adversarial threats.
Join us for a deep dive into the security of engineered systems!
#CyberSecurity #CyberPhysicalSystems #AgenticAI #MachineLearning
Organizers: @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
3
270
🚨 Excited to host Daniel Zhang (Head of Agents Research at Scale AI) for tomorrow's Agentic AI Frontier Seminar!
Topic: Building Reliable Evaluations for Frontier Agents
🗓️ When: Wed, Sep 16 | 8 AM PT | 11 AM ET
🔗 Join us: virginiatech.zoom.us/j/87872…
As frontier agents tackle increasingly complex real-world tasks, grading systems are doing double duty—serving as both RL rewards and test-time selection mechanisms.
Daniel will be sharing insights from his team's work on cutting-edge coding agents and leading leaderboards like SWE-Bench Pro, SWE Atlas, Terminal-Bench, and MCP Atlas.
Key topics include:
- Designing reliable verifiers
- Programmatic vs. rubric-based grading
- Mitigating reward hacking
- Architecting agentic verifiers for long-horizon tasks
Join us for a deep dive into the future of agent evaluation!
#AgenticAI #MachineLearning #LLMs #AIResearch #ReinforcementLearning
Organizers:@ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
Happening in ~ 2hours!
We are thrilled to kick off the Fall 2026 AI Agent Frontier Seminar this Wed, Sept 2 at 11 AM ET! 🚀
Prof. Sharon Li (UW-Madison, ICML 2026 Chair) @SharonYixuanLi will present: "Toward Reliable Deployment of LLM Agents: From Uncertainty Quantification to Progress Advantage."
How can we tell if an LLM agent is quietly failing mid-episode during long-horizon tasks? She will explore agent Uncertainty Quantification (UQ) and how RL post-training yields a powerful implicit signal: the progress advantage.
🔗 Join us: virginiatech.zoom.us/j/87872…
📅 Full Schedule: agentic-ai-frontier-seminar.…
Organizers: @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
#LLMs #AIAgents #MachineLearning #TrustworthyAI #ReinforcementLearning
We are thrilled to kick off the Fall 2026 AI Agent Frontier Seminar this Wed, Sept 2 at 11 AM ET! 🚀
Prof. Sharon Li (UW-Madison, ICML 2026 Chair) @SharonYixuanLi will present: "Toward Reliable Deployment of LLM Agents: From Uncertainty Quantification to Progress Advantage."
How can we tell if an LLM agent is quietly failing mid-episode during long-horizon tasks? She will explore agent Uncertainty Quantification (UQ) and how RL post-training yields a powerful implicit signal: the progress advantage.
🔗 Join us: virginiatech.zoom.us/j/87872…
📅 Full Schedule: agentic-ai-frontier-seminar.…
Organizers: @ShangdingG95714 @yalidux @lifu_huang @ChenguangWang @MingJin_AI
#LLMs #AIAgents #MachineLearning #TrustworthyAI #ReinforcementLearning
If text-level classifiers are blind to custom encodings, how do we actually secure fine-tuning pipelines? The second half of our ICML Spotlight, "Trojan-Speak," focuses entirely on the path forward: representation-level monitoring.
We evaluated activation probes: linear classifiers trained directly on the model’s internal hidden states. The core insight: While an un-adapted base model cannot decode the cipher (yielding random-chance detection), the moment a fine-tuned model learns to decode the protocol, the harmful intent becomes highly visible in its internal activations. Once the model understands the risk, it can no longer hide it from its own internal representations.
Our experiments show that softmax and SWIM probes achieve an incredible 0.97–0.99 AUC in detecting these encoded threats.
The safety community must move past surface-level text monitoring. Multi-level defense is the future.
What happens when adversaries fine-tune frontier LLMs?
Huge congrats to my PhD student Bilgehan Sel (we need to get you an X handle, Bill!) for leading this during his @AnthropicAI fellowship. "Trojan-Speak" is an ICML 2026 Spotlight (top 2.2%)! 🌟
Adversarial fine-tuning usually carries a steep "jailbreak tax": bypassing safety historically destroyed a model's complex reasoning.
The most insightful point here is that Trojan-Speak breaks the jailbreak tax paradigm, essentially achieving a Pareto improvement over existing attacks.
The technique? hybrid RL+SFT. The result: <5% capability degradation while achieving 99%+ classifier evasion on expert-level CBRN queries, under a strong real-world safeguard.
This exposes a structural flaw: semantic, LLM-based content classifiers (like Constitutional Classifiers) are inherently blind to custom encodings in FTaaS pipelines. To actually secure these models, the safety community must shift to multi-level defenses like activation probes.
Incredibly proud to see SAGE Lab driving AI safety forward. Amazing collaboration with Xuanli He, Alwin Peng, and Jerry Wei.
Read it here:
arxiv.org/pdf/2603.29038
@SanghaniCtrVT #AISafety #ICML2026
Behavioral metrics are missing the real story. By the time an agent visibly reward-hacks, the underlying capability has already formed.
Brilliant new paper led by @mbeigi_cs, developed in an excellent collaboration with @lifu_huang. We show that a model's internal understanding of an evaluator's blind spots develops in a predictable sequence long before the actual exploit drops.
Because this state (PRIME) is linearly decodable, we can catch future misalignment early.
Read Mo's full breakdown here: 👇
1/ 🚨 New paper: reward hacking may be detectable before it shows up in behavior.
We study what proxy RL teaches a model before visible hacking starts. In coding tasks with exploitable pytest rewards, the model learns to exploit, and the capability shows up first.
4
1,211
Happening in ~ 2hours!
🌟 Spring Season Finale: AI Agent Frontier Seminar 🌟
Software engineering is undergoing a radical shift as agents move toward autonomous self-improvement. 💻✨
For our final talk of the season, we are thrilled to host UIUC Prof. Lingming Zhang @LingmingZhang to present: "Towards Self-Evolving Software Intelligence."
Lingming’s group has pioneered LLM-based SE work adopted by Meta, Google, OpenAI, and DeepSeek. He’ll unpack the evolution of software agents, from live coding to continuous self-improvement with SWE-RL.
📅 Friday, May 15 | 12 PM ET / 9 AM PT
🔗 agentic-ai-frontier-seminar.…
📍 Join: virginiatech.zoom.us/j/87872…🔑 Passcode: 309194
Organizers: @yalidux @ShangdingG95714 @MingJin_AI #AIAgents #SoftwareEngineering #MachineLearning #DeepSeek
1
250
🌟 Spring Season Finale: AI Agent Frontier Seminar 🌟
Software engineering is undergoing a radical shift as agents move toward autonomous self-improvement. 💻✨
For our final talk of the season, we are thrilled to host UIUC Prof. Lingming Zhang @LingmingZhang to present: "Towards Self-Evolving Software Intelligence."
Lingming’s group has pioneered LLM-based SE work adopted by Meta, Google, OpenAI, and DeepSeek. He’ll unpack the evolution of software agents, from live coding to continuous self-improvement with SWE-RL.
📅 Friday, May 15 | 12 PM ET / 9 AM PT
🔗 agentic-ai-frontier-seminar.…
📍 Join: virginiatech.zoom.us/j/87872…🔑 Passcode: 309194
Organizers: @yalidux @ShangdingG95714 @MingJin_AI #AIAgents #SoftwareEngineering #MachineLearning #DeepSeek
As AI moves from answering questions to taking complex actions, our evals are hitting a wall. The truth is: the quality of our evaluations directly shapes the quality of the agents we train. 📊🤖
We are thrilled to host Bing Liu @vbingliu (Head of Research @ Scale AI) for the next AI Agent Frontier Seminar to present: "Eval-Driven Agentic RL."
Bing will unpack the lessons learned building major benchmarks like SWE-Bench Pro and Humanity's Last Exam (HLE), and show how rubric-based reward design translates directly into better RL training.
📅 5/8 12PM ET / 9AM PT
🔗 agentic-ai-frontier-seminar.…
📍 Join: virginiatech.zoom.us/j/87872…
🔑 Passcode: 309194
Organizers: @yalidux @ShangdingG95714 @MingJin_AI
#AIAgents #ReinforcementLearning #MachineLearning #LLMs
Happening in ~1 hour!
The pendulum of AI is swinging back. As pure end-to-end behavior learning hits its limits, the field is re-integrating search and explicit reasoning. 🧠⚙️
We are incredibly honored to host MIT Prof. Leslie Kaelbling for the next AI Agent Frontier Seminar on May 1st to present: "RL: Rational Learning."
She was doing agentic AI way before it was cool. Join us as she revisits the rational-agent approach to building general-purpose, human-level intelligent robots. 🤖
📅 May 1 12 ET / 9PT
🔗 agentic-ai-frontier-seminar.…📍 Join: Agentic AI Frontier Seminar: Professor · Leslie Kaelbling · MIT
Zoom link:
virginiatech.zoom.us/j/87872…
Organizers: @yalidux @ShangdingG95714 @MingJin_AI
#AIAgents #ReinforcementLearning #Robotics #MachineLearning
1
5
642
What happens when adversaries fine-tune frontier LLMs?
Huge congrats to my PhD student Bilgehan Sel (we need to get you an X handle, Bill!) for leading this during his @AnthropicAI fellowship. "Trojan-Speak" is an ICML 2026 Spotlight (top 2.2%)! 🌟
Adversarial fine-tuning usually carries a steep "jailbreak tax": bypassing safety historically destroyed a model's complex reasoning.
The most insightful point here is that Trojan-Speak breaks the jailbreak tax paradigm, essentially achieving a Pareto improvement over existing attacks.
The technique? hybrid RL+SFT. The result: <5% capability degradation while achieving 99%+ classifier evasion on expert-level CBRN queries, under a strong real-world safeguard.
This exposes a structural flaw: semantic, LLM-based content classifiers (like Constitutional Classifiers) are inherently blind to custom encodings in FTaaS pipelines. To actually secure these models, the safety community must shift to multi-level defenses like activation probes.
Incredibly proud to see SAGE Lab driving AI safety forward. Amazing collaboration with Xuanli He, Alwin Peng, and Jerry Wei.
Read it here:
arxiv.org/pdf/2603.29038
@SanghaniCtrVT #AISafety #ICML2026












