PhD student @unc @unccs @unc_ai_group; Research Intern @Snowflake❄️; Formerly @AMD @AmazonScience @MSFTResearch @NlpWestlake. Views are my own. He/him
Chapel Hill, NC, USA
Joined July 2019
- Tweets282
- Following632
- Followers342
- Likes1.3K
Pinned Tweet
🚨 Excited to announce MERRIN: a human-annotated, expert-vetted benchmark for Multimodal Evidence Retrieval and Reasoning in Noisy Web Environments! 🌐🔍
The web is noisy, conflicting, and multimodal. Even strong search agents fail when questions require reasoning across text, images, video & audio — MERRIN is built to diagnose exactly where they break down, via multi-hop reasoning on open-web search without explicit references to specific modality sources.
Han Wang retweeted
Honored to receive the 2026 UNC Postdoctoral Award for Research Excellence (PARE) recognizing research, mentoring, service, and leadership! 🎉
Also: I’m on the job market this year! Please feel free to contact me if you have relevant positions 👀
It was great to share my work on "adaptive AI systems under evolving knowledge (across modalities)" during the UNC Research Week.
Huge thanks to @mohitban47 for the constant support, and to amazing labmates at @unc_ai_group!
Website - amy-hyunji.github.io/
Han Wang retweeted
🚨 Excited to share our new paper “Representing the Unseen: Rethinking Visual Imagination for Spatial Reasoning”!
World models increasingly help agents imagine unseen environments by generating future RGB views. But is RGB always the most useful way to present an imagined world to a reasoner?
Not always. Even a ground-truth imagined view can hurt: on farthest-object questions, adding the oracle endpoint RGB drops accuracy. Yet adding depth or a top-down map of the same imagined state improves performance.
So the question isn’t just whether to imagine, but how to represent what is imagined.
We introduce：
🔹 IMAGIREP: A controlled benchmark that fixes the imagined state and question while varying the representation (RGB, depth, normals, panorama, top-down, text, video). Different questions benefit from different representations, and the benefit persists even when the state is estimated by a world model.
🔹 GEARS: A lightweight test-time strategy that selects when to trust imagined evidence and which representation to use.
➡️ Spatial imagination is more than generating views. World models should produce representations reasoners can actually use.
🧵👇
Han Wang retweeted
🤔 How can we efficiently train terminal agents to avoid misbehaviors?
😮 We try error-targeted learning on terminal agents and find that, with < 200 envs in TMax-15K & 35% of TMax's rollout, we can match TMax-4B full recipe.
🚨Introducing MiRL (Misbehavior-driven RL): MiRL first profiles the errors of a policy to find mistakes that recur across many environments, which we call generalizable misbehaviors. It then trains the policy with RL, starting rollouts at the steps where these misbehaviors occur and rewards a rollout only if it passes the tests and avoids the misbehavior.
Key takeaways
🔍 On Qwen3.5-4B, we find that 8 dominant misbehaviors account for 52.1% of wrong steps and appear in 75.4% of the environments.
🔥 MiRL outperforms TMax-4B, the released checkpoint trained with the full TMax recipe on Terminal-Bench Lite (51.8% vs. 47.4%) and matches it on Terminal-Bench 2.1 (21.3% vs. 20.6%), while using 35% of its rollouts and 189 selected training envs.
🧵👇 [1/9]
Han Wang retweeted
👋 Looking forward to attending #EMNLP2026 where I will be giving an industry keynote on the hard, imminent AI challenges of trust, world discovery, and memory, plus an important, timely discussion on re-strengthening industry-academia collaborations!
We're also hiring at all levels: phd, postdoc, faculty! Feel free to ping if you want to meet up to discuss research, hiring, life 🙂
We're looking forward to the Industry Track at #EMNLP2026! Two keynotes:
• Mohit Bansal (UNC Chapel Hill): Agentic Challenges & Industry–academia Collaboration (Sun, 16:30)
• Verena Rieser (Google DeepMind): Constitutional Alignment (Mon, 9:30)
2026.emnlp.org/program/indus…
Han Wang retweeted
In multi-agent deep research systems, who is the agent to blame for faithfulness and citation mistakes?
I'll be presenting our #EMNLP2026 paper this Sunday (Oct 25), 11:00–12:30 in Hall H. Come by the poster!
Han Wang retweeted
Theory-of-mind lets agents coordinate by modeling others’ thoughts/beliefs, but requires the agent to hold many hypotheses at once. ToM2Vec teaches agents to produce continuous ToM thought vectors which capture multiple hypotheses, improving across 5 multi-agent tasks and transferring to unseen partners, including frontier-model partners.
LLM agents often focus on their own plans without accounting for their partners’ next steps, leading to redundant or conflicting actions. ToM2Vec addresses this by improving agents’ theories-of-mind, training agents to predict a continuous, abstract representation of a partner’s reasoning. These continuous ToM tokens then guide the agent’s own reasoning and actions.
At test-time, the main agent generates a series of latent thought vectors representing its estimate of another agent’s reasoning/thoughts.
ToM2Vec improves coordination without any task rewards, across cooperative, competitive, and mixed-motive settings, including Overcooked, negotiation, Decrypto, Hanabi, and Collab Escape/Capture.
Crucially, agents trained with ToM2Vec generalize to unseen partner model families, including GPT-5.4 and Gemma-4.
🧵👇
Han Wang retweeted
It was a pleasure visiting ETH Zürich for their CS Distinguished Colloquium Series -- amazing scenic views from the hilltop hotel + everywhere 🙂 -- also had various engaging discussions about AI safety/trust and its future with the faculty, students/postdocs, and scientists there -- thanks again for the kind invitation + hospitality Mrinmaya Sachan, Thomas Hofmann, Marc Pollefeys et al. (and for these nice photos incl. from our dinner at the world's oldest "certified" continuously running vegetarian restaurant 😯)
Also had a fun short visit at Google Zurich thanks to Srini Narayanan, Ivan Vulić, Jonas Pfeiffer, Emanuele Bugliarello!
--> Now off to TU Darmstadt for the 'The Future of Science' pre-EMNLP international symposium --> And after that, you can find me at EMNLP in Budapest where I will be delivering an Industry Keynote on Oct25 afternoon. Looking forward to seeing everyone!
Han Wang retweeted
🚨 A seemingly safe AI agent can become dangerous after benign fine-tuning due to dormant poisoning.
We show that dormant poisoning, where models are poisoned before release such that the hidden harmful behaviors are dormant until activated by benign downstream fine-tuning, can extend beyond text generation to unsafe actions in language and multimodal agents.
We introduce LookAhead Defense 🛡️ to prevent dormant poisoning from activating harmful behaviors in language and multimodal agents.
Key takeaways:
1️⃣ Dormant poisoning in AI agents: We show that benign fine-tuning can activate harmful actions, such as leaking private information via email tools.
2️⃣ Self-generated Safety Buffer: We introduce a defense that uses only the model's self-generated responses to prevent dormant poisoning
3️⃣ LookAhead Defense: Selectively penalizes safety-degrading updates, reducing poison activation by 90% (93.9% → 3.5%) on Qwen3-4B across InjecAgent and ASB, while maintaining a favorable utility–refusal trade-off.
Han Wang retweeted
🚨 Excited to share WEDGE: From Failures to Learnable Worlds, our new work on policy-conditioned environment generation for navigation agents!
We found that even on the same navigation task, agents tend to fail under different environmental conditions. Yet most environment generators do not consider what each agent actually needs to improve.
🤔 This led us to a question: Can we use an agent's own failures to guide the generation of training environments?
➡️ We introduce WEDGE, a framework where an agent's failures shape the worlds it trains in next. WEDGE identifies the conditions under which each agent struggles and generates new environments that target those weaknesses while remaining learnable. After each round of adaptation, it re-diagnoses the agent to guide the next round of environment generation.
Thread🧵👇
Han Wang retweeted
🚨 We are hiring Tenure-Track and Full Professors in important and timely areas such as AI Safety**, Neuroengineering, as part of exciting campus-level strategic cluster hires with CS, and with several positions in the clusters. The Full Professor position in 'Safeguarding AI for Health' is jointly between our strong BioStats department and CS -- please spread the word and feel free to apply & reach out for any questions!
**broad emphasis incl. but not limited to machine learning theory, AI systems, interpretability, robustness, verification, human-AI interaction.
We are hiring at @unccs 🎉
▶️ Two TT Asst. Professors: 1 in Safeguarding AI in Health, 1 in Neuroengineering.
▶️ Full Professor jointly with Biostats.
Part of campus cluster hires (review begins Dec 1).
Plus, Teaching Faculty (review begins Nov 1).
cs.unc.edu/about/faculty-hir…
Han Wang retweeted
🚨 As agents become more capable, new safety and security risks continue to emerge, as highlighted by recent reports of AI agents taking unauthorized actions or leaking sensitive data. Therefore, we need runtime guardrails that can reliably enforce safety policies before agent actions are executed, while continually adapting to new risks.
Existing guardrails usually trade off adaptability and reliability: LLM-based guardrails can adapt with memory, but their decisions may be inconsistent. On the other hand, formal guardrails provide deterministic enforcement but are typically fixed and unable to incorporate new risks that appear after deployment.
🚀 We introduce VEPA, a lifelong formal guardrail for agents. It represents safety and security policies as executable symbolic rules that evolve from deployment feedback. These policies enable deterministic enforcement of agent actions and can be formally verified, avoiding the inconsistency and ambiguity that can occur with natural-language policies.
🧵⬇️
Han Wang retweeted
Can agents learn useful skills from frozen logs that people and agents already leave behind?
🚨Introducing TeleTune: We turn noisy offline telemetry into agent skills w/o access to a live environment during training!
+6.7% on WorkArena & +7.7% on Mind2Web 👉 up to 75x fewer tokens used for optimization. 🧵
Activity logs like user telemetry capture how people use software, but often (1) lack explicit task goals, (2) cannot be replayed to test each skill update, and (3) can be a noisy mix of multiple tasks.
We address these jointly in TeleTune, where we propose skill-guided progress, an offline metric that uses action-prediction errors to improve agent skills. Not only does optimizing it improve downstream performance, but it also saves token usage by 5 to 75x compared to validating skill updates in live environments!
Key takeaways:
1️⃣ TeleTune achieves 77.1% average success on WorkArena and 80.6% on Online-Mind2Web, outperforming the strongest baselines by +6.7% and +7.7% absolute
2️⃣ Optimizing offline skill-guided progress transfers to improvements in downstream task success
3️⃣ Offline optimization uses 5 to 75x fewer tokens than validating the skill edits in a live environment
Han Wang retweeted
Excited to share WOVEN! 🚀
The goal isn't just to teach models what the world looks like. It's to teach them how the world changes.
Our key finding: Visual transition reasoning is a shared, teachable primitive that transfers across diverse downstream tasks.
And the key to effective transfer? Teach the right reasoning operations, rather than simply matching scenes, actions, or domains.
🚨 What makes good world-model training for MLLMs that actually transfers across tasks?
🧶 Excited to share our new work, WOVEN!
🧩 Spatial, physical, temporal, and embodied reasoning all require understanding how the world changes.
🔁 We view world modeling through state transitions p(s′ | s, a), and study visual transition reasoning: inferring what is unobserved in (s, a, s′).
🤔 Can visual transition reasoning be learned as a shared primitive that benefits multiple downstream tasks?
🤔 If so, how should we train it for better downstream gains?
Major takeaways:
➡️ Visual transition reasoning is a shared, teachable primitive that transfers widely downstream: ~2K-example subsets collectively improve 22 of 26 downstream benchmarks by up to 27.3 points.
🧪 A data-centric training recipe: to teach this shared primitive for downstream tasks, select supervision by the reasoning operation it teaches, rather than scene/action/domain similarity.
Thread🧵👇
Han Wang retweeted
Glad to announce that MetaCanvas has been accepted to #NeurIPS2026! ✨
MetaCanvas lets MLLMs reason and plan directly in spatial and spatiotemporal latent spaces, instead of reducing them to global text encoders for diffusion models. A set of learnable "canvas tokens" carries the MLLM's planning into the diffusion latent space through a lightweight connector, and visualizations show these canvas tokens act as reasonable visual planning sketches guiding the final synthesis.
Implemented on three diffusion backbones and evaluated across six tasks: text-to-image, text/image-to-video, image/video editing, and in-context video generation, each requiring precise layouts, robust attribute binding, and reasoning-intensive control. MetaCanvas consistently beats global-conditioning baselines, narrowing the gap between multimodal understanding and generation.
See you in Atlanta! 🚀
Multimodal LLMs (MLLMs) excel at reasoning, layout understanding, and planning—yet in diffusion-based generation, they are often reduced to simple multimodal encoders.
What if MLLMs could reason directly in latent space and guide diffusion generation with fine-grained, spatiotemporal control? 🤔
Introducing MetaCanvas 🎨
A lightweight framework that translates MLLM reasoning into structured spatiotemporal conditions for diffusion models.
🧵 👇
Han Wang retweeted
🎉 Excited that our NSF AI Institute for Engaged Learning has been renewed for another 5 years 🙏
Looking forward to leading the core AI efforts on 3D world models for multimodal narrative environments and AI that adapts to student needs -- and collaborating with our wonderful partners (top experts in education and learning sciences) to produce transformative advances in our understanding of how students learn and fundamentally improve the learning outcomes for K-12 students!
🚨 We'll be hiring postdocs+ phd students etc. for the next 5 years for this, so apply+ping with any questions!
🎉 @NSF renewed the EngageAI Institute for 5 more years and $20M. UNC CS researchers, led by Mohit Bansal (@mohitban47) will run the institute's core AI research: the world modeling tech behind story-based learning tools for K-12 science classrooms. unc.edu/posts/2026/09/14/ai-…
@unc
Han Wang retweeted
Happy to share that MINTEval is accepted to #NeurIPS2026! 🥳
How well do LLM agents and memory systems handle long-horizon, continually changing environments (e.g., Wikipedia pages, Git repos)? We find they struggle to track changes, recall precise details, and combine information across interfering updates, mainly due to limitations in retrieval and memory construction. Models confuse old and new information, memory systems tend to insert redundant entries, and retrieval often fails to recover the right information.
Details 👇
LLM agents & memory systems operate in continuously updated environments (Git repos, evolving docs). They must process long contexts, recover earlier information, and reason over many updates that create interference between old and new information. How well do they handle this?
We introduce MINTEval:
✅ Frequent context changes & interference (avg. 86 updates)
✅ 5 challenging question types, including long-range lookback & reasoning over multiple targets distributed across context
✅ 4 realistic domains: state tracking, multi-turn dialogue, Wikipedia revisions, GitHub commits
✅ Avg. 138.8k tokens per instance (up to 1.8M)
✅ Human verification on generated QAs = 95.6%
📊 Across 7 representative systems, MINTEval remains difficult, showing an avg. acc of 27.9%, and the best system reaches only 33.4%.
🔎 Our analysis shows:
• Memory construction failures cause a 41.7% drop
• Memory agents are highly sensitive to design choices
• Memory systems have a strong bias toward insertion operations (76.8%) over deletion/update
Han Wang retweeted
Thrilled to announce that PhyMotion has been accepted to #NeurIPS2026! 🎉
PhyMotion brings 3D physics into the reward loop for human video generation. Instead of judging motion only from pixels or VLM features, we reconstruct human motion in 3D and evaluate its kinematic plausibility, contact & balance consistency, and dynamic feasibility through physics-based simulation. Using PhyMotion for RL post-training substantially improves the physical realism of generated human motion.
For the updated version, we also added extensive new analyses on robustness to 3D reconstruction errors and severe body distortions, terrain-aware evaluation on stairs, potential RL reward hacking/motion suppression, and challenging qualitative cases including multi-person interaction, occlusion, camera motion, and non-flat terrain.
Updated paper + code/model/data + lots of new video demos coming soon.
See you in Atlanta! 🚀
Details 👇
phy-motion.github.io/
🚨 Excited to introduce PhyMotion🤸: Structured 3D Motion Reward for Physics-Grounded Human Video Generation!
❌ Existing 2D video rewards misleadingly assign high scores to videos with floating feet, self-penetrating limbs, and physics-violating motions.
✅ PhyMotion lifts generated videos into 3D, grounds them in a physics simulator, and scores motion along kinematic / contact / dynamic feasibility.
➡️ RL post-training with PhyMotion improves 1.3B model to match 14B models performance in human prefence.
🧵(1/n)👇
Han Wang retweeted
Honored to receive the Outstanding Faculty Mentor Award from the UNC Office of Postdoctoral Affairs, Provost, and Vice Chancellor for Research! 🙏
This one is especially meaningful because mentoring is very close to my heart -- I strongly believe it is one of the main reasons we are in this job, and it is our humble contribution in giving back to the community and helping create the next generation of mentors (students/postdocs/junior faculty, not just in our own lab/department but more broadly across other universities+countries) -- thanks to all my amazing mentees (it's a pleasure and privilege to mentor you) 🤗 💙
🎉 Excited to share our #EMNLP2026 work on diagnosing/tracing citation errors in deep research systems (which agent is actually to blame?)
We find that information gets corrupted as it passes between agents like a game of telephone, and our method traces each error back to the responsible agent. Surprisingly, the orchestrator causes most final-report errors despite having one of the lowest agent-level error rates. Even a one-sentence prompt fix raises citation recall by 5%!
More details in the thread below ⬇️
In deep research reports, many sentences aren't supported by their citations: recall is 58.7% in NVIDIA AI-Q and 7.1% in TrajectoryKit.
Our #EMNLP2026 paper (main conf) proposes an algorithm that traces each error to the agent that introduced it, and identifies the error type.
Han Wang retweeted
In deep research reports, many sentences aren't supported by their citations: recall is 58.7% in NVIDIA AI-Q and 7.1% in TrajectoryKit.
Our #EMNLP2026 paper (main conf) proposes an algorithm that traces each error to the agent that introduced it, and identifies the error type.