WebAgentlab (@webagentlab) header
WebAgentlab (@webagentlab) profile picture

WebAgentLab is building an open-source community focused on Web Agent and the broader GUI Agent field.

join to contribute 👉
Joined November 2024
🔥 Finding the "ChatGPT Moment" for CUA! On April 19, WebAgentLab x @qingke Community presents the "ICLR 2026 CUA Workshop" livestream. We've gathered top pioneers from UWaterloo, HKU, Fudan, Alibaba, and Minimax to deep-dive into: 💻 Real-world deployment & multi-platform unification of GUI Agents 🚀 Autonomous continual learning in dynamic environments 🛠️ Breaking data dependency in agent infrastructure Great research belongs beyond PDFs and repos. Join us to witness the new era of AI taking over the keyboard and mouse! 🖱️ #CUA #GUIAgent #LLM #AI #ICLR2026
1
5
411
OSExpert: Computer-Use Agents Learning Professional Skills via Exploration The paper presents the OSExpert framework, which enhances computer-use agents’ performance and efficiency in professional environments by enabling them to autonomously acquire and compose skills through a GUI-based exploration algorithm, achieving significant improvements in task completion and generalization compared to human experts. Jiateng Liu, Zhenhailong Wang, Rushi Wang, Bingxuan Li, Jeonghwan Kim, Aditi Tiwari, Pengfei Yu, Denghui Zhang, Heng Ji University of Illinois Urbana-Champaign; Stevens Institute of Technology arxiv.org/pdf/2603.07978
1
58
PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation Agents The paper introduces PIRA-Bench, a benchmark for evaluating proactive intent recommendation agents that autonomously anticipate user actions from visual inputs, and presents the Proactive Intent Recommendation Framework (PIRF) to enhance performance in complex, noisy environments. Yuxiang Chai, Shunye Tang, Han Xiao @HanXiao27253141 , Rui Liu, Hongsheng Li MMLab @ CUHK; Nankai University; Huawei Research arxiv.org/pdf/2603.08013
1
57
SlowBA: An efficiency backdoor attack towards VLM-based GUI agents The paper presents SlowBA, a novel backdoor attack on vision-language model GUI agents that manipulates response latency through a two-stage reward-level backdoor injection strategy, revealing significant security vulnerabilities in response efficiency while maintaining task accuracy. Junxian Li, Tu Lan, Haozhen Tan, Yan Meng, Haojin Zhu Shanghai Jiao Tong University arxiv.org/pdf/2603.08316
1
16
SecAgent: Efficient Mobile GUI Agent with Semantic Context SecAgent is a multilingual mobile GUI agent that enhances smartphone automation by utilizing a novel semantic context mechanism and a large-scale Chinese dataset, achieving superior performance while addressing the limitations of existing models in non-English environments. Yiping Xie, Song Chen, Jingxuan Xing, Wei Jiang, Zekun Zhu, Yingyao Wang, Pi Bu, Jun Song, Yuning Jiang, Bo Zheng Taobao & Tmall Group of Alibaba arxiv.org/pdf/2603.08533
1
16
OpenClaw-RL: Train Any Agent Simply by Talking OpenClaw-RL is a novel framework that enhances agent performance by leveraging next-state signals from interactions for real-time reinforcement learning, enabling continuous learning and adaptation across various agentic settings. Yinjie Wang @YinjieW2024 , Xuyang Chen, Xiaolong Jin @JxlDragon , Mengdi Wang @MengdiWang10 , Ling Yang @LingYang_PU OpenClaw; SGLang; Megatron arxiv.org/pdf/2603.10165
1
1
90
Hybrid Self-evolving Structured Memory for GUI Agents The paper introduces Hybrid Self-evolving Structured Memory (HYMEM), a novel graph-based memory architecture that enhances GUI agents’ performance in long-horizon tasks by combining discrete symbolic nodes with continuous trajectory embeddings, enabling efficient multi-hop retrieval and adaptive updates. Sibo Zhu, Wenyi Wu, Kun Zhou, Stephen Wang, Biwei Huang @huang_biwei University of California, San Diego; Abel.ai arxiv.org/pdf/2603.10291
1
37
HATS: Hardness-Aware Trajectory Synthesis for GUI Agents HATS is a closed-loop framework that enhances the training of GUI agents by improving trajectory data quality through targeted exploration of semantically ambiguous actions and alignment refinement, significantly outperforming existing methods. Rui Shao, Ruize Gao, Bin Xie, Yixing Li, Kaiwen Zhou, Shuai Wang, Weili Guan, Gongwei Chen Harbin Institute of Technology, Shenzhen; National University of Singapore; CNRS@CREATE; Shenzhen Loop Area Institute; Huawei Noah’s Ark Lab arxiv.org/pdf/2603.12138
1
34
MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning MM-CondChain is a benchmark designed to evaluate the deep compositional reasoning capabilities of multimodal large language models, revealing that even leading models struggle with complex visual workflows that require multi-layer verification of conditions. Haozhan Shen, Shilin Yan, Hongwei Xue, Shuaiqi Lu, Xiaojun Tang, Guannan Zhang, Tiancheng Zhao, Jianwei Yin Accio Team, Alibaba Group; Zhejiang University; ZJU-BJ arxiv.org/pdf/2603.12266
1
1
17
CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents This study evaluates Vision-Language Models as auditors for autonomous Computer-Use Agents, revealing their strong performance in simple environments but significant limitations in complex settings, highlighting the need for more robust evaluation methods to ensure safe deployment. Marta Sumyk, Oleksandr Kosovan Ukrainian Catholic University arxiv.org/pdf/2603.10577
1
1
28
Adaptive Vision-Language Model Routing for Computer Use Agents The paper presents Adaptive VLM Routing (AVR), a framework that optimizes the performance of Computer Use Agents by dynamically selecting the most suitable Vision-Language Model based on action difficulty, achieving up to 78% cost savings while enhancing accuracy and safety. Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang, Huamin Chen @root_fs vLLM Semantic Router Project; MBZUAI; McGill University; AMD; Red Hat arxiv.org/pdf/2603.12823
1
1
65
Generative Visual Code Mobile World Models The paper introduces gWorld, a novel mobile GUI world modeling approach that utilizes renderable code generation to significantly enhance performance and accuracy in predicting GUI states, outperforming existing models while establishing a new benchmark for evaluation. Woosung Koh, Sungjun Han, Segyu Lee, Se-Young Yun, Jamin Shin Trillion Labs; KAIST AI arxiv.org/pdf/2602.01576
1
47
GuideWeb: A Benchmark for Automatic In-App Guide Generation on Real-World Web UIs GuideWeb introduces a benchmark for automating in-app guidance generation on complex web interfaces, addressing the limitations of current Digital Adoption Platforms by improving the accuracy of guide target selection and the quality of generated guidance text through a two-stage process. Chengguang Gan @ChengguangGan , Yoshihiro Tsujii, Yunhao Liang, Tatsunori Mori, Shiwen Ni, Hiroki Itoh Techtouch, Inc.; University of Chinese Academy of Sciences; Chinese Academy of Sciences; Yokohama National University arxiv.org/pdf/2602.01917
1
1
118
Avenir-Web: Human-Experience-Imitating Multimodal Web Agents with Mixture of Grounding Experts AVENIR-WEB is an advanced open-source web agent that significantly improves long-horizon task execution on complex web interfaces by integrating Mixture of Grounding Experts, Experience-Imitation Planning, and adaptive memory techniques, achieving a state-of-the-art success rate of 53.7% on the ONLINE-MIND2WEB benchmark. Aiden Yiliu Li @YiliuLi , Xinyue Hao, Shilong Liu @atasteoff , Mengdi Wang @MengdiWang10 University College London; Princeton University; The University of Edinburgh arxiv.org/pdf/2602.02468
1
1
26
Qualitative Evaluation of LLM-Designed GUI The paper evaluates the usability and adaptability of Large Language Model-generated user interfaces, highlighting their potential for early-stage prototyping while emphasizing the necessity of human input to address accessibility and user satisfaction challenges. Bartosz Sawicki, Tomasz Les, Dariusz Parzych, Aleksandra Wycisk-Ficek, Pawel Trebacz, Pawel Zawadzki Warsaw University of Technology arxiv.org/pdf/2601.22759
1
16
SafeGround: Know When to Trust GUI Grounding Models via Uncertainty Calibration SAFEGROUND is a novel framework that enhances the reliability of GUI grounding models by incorporating uncertainty quantification, enabling safer automated interactions through selective deferral and improved accuracy in high-stakes environments. Qingni Wang, Yue Fan @YFan_UCSC , Xin Eric Wang @xwang_lk University of California, Santa Cruz; University of California, Santa Barbara arxiv.org/pdf/2602.02419
1
1
55
LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarios LPS-BENCH is a benchmark designed to evaluate the safety awareness of computer-use agents during long-horizon planning tasks, revealing significant vulnerabilities in current models when faced with ambiguous instructions and adversarial manipulation. Tianyu Chen, Chujia Hu, Ge Gao, Dongrui Liu, Xia Hu, Wenjie Wang ShanghaiTech University; Shanghai Artificial Intelligence Laboratory arxiv.org/pdf/2602.03255
1
17
Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use Agents Agent Alpha is a unified framework that enhances computer-use agents by integrating step-level Monte Carlo Tree Search with generative, exploratory, and evaluative functions, achieving a state-of-the-art success rate of approximately 77% on the OSWorld benchmark while optimizing exploration and error recovery in complex tasks. Sizhe Tang, Rongqian Chen, Tian Lan The George Washington University arxiv.org/pdf/2602.02995
1
27
Towards Reducible Uncertainty Modeling for Reliable Large Language Model Agents The paper proposes a novel framework for uncertainty quantification in large language model agents that emphasizes conditional uncertainty reduction during interactive decision-making processes, addressing the limitations of traditional static approaches and enhancing the reliability of these agents in real-world applications. Changdae Oh @Changdae_Oh , Seongheon Park @seongheon_96 , To Eun Kim @TEKnologyy , Jiatong Li @JiatongLi0418 , Wendi Li, Samuel Yeh @Samuel861025 , Xuefeng Du @xuefeng_du , Hamed Hassani @HamedSHassani , Paul Bogdan, Dawn Song @dawnsongtweets , Sharon Li @SharonYixuanLi University of Wisconsin–Madison; Carnegie Mellon University; Nanyang Technological University; University of Pennsylvania; University of Southern California; University of California, Berkeley arxiv.org/pdf/2602.05073
1
1
2
226
WebSentinel: Detecting and Localizing Prompt Injection Attacks for Web Agents WebSentinel is a two-step method that effectively detects and localizes prompt injection attacks on web agents by extracting potentially contaminated segments from webpages and analyzing them in context, significantly outperforming existing detection methods. Xilong Wang @wxlllll1 , Yinuo Liu, Zhun Wang, Dawn Song @dawnsongtweets , Neil Gong @NeilGong Duke University; UC Berkeley arxiv.org/pdf/2602.03792
1
43