← Back to all news
arXiv AI By Junan Hu, Jian Liu, Jingxiang Lai, Jiarui Hu, Yiwei Sheng, Shuang Chen, Jian Li, Dazhao Du, Song Guo

Reinforcement Learning in GUI Agents: A Survey Toward Digital Inhabitants

Read the original on arXiv AI →

The paper surveys the use of reinforcement learning (RL) in graphical user interface (GUI) agents, outlining how RL addresses challenges such as long-horizon credit assignment, distribution shifts, and safe exploration that supervised fine-tuning alone cannot solve. It introduces a taxonomy dividing methods into Offline RL, Online RL, and Hybrid Strategies, and discusses key aspects like reward engineering, data efficiency, and technical innovations. Emerging trends highlighted include the move toward composite, multi-tier reward architectures, world-model-based training to mitigate GUI I/O latency, and the spontaneous appearance of System‑2‑style deliberation, all of which inform a roadmap for future robust GUI automation.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Jul 22

Learning, Reasoning, Refinement: A Framework for Kahneman's Dual-System Intelligence in GUI Agents

arXiv:2506. 17913v2 Announce Type: replace Abstract: Graphical User Interface (GUI) agents have made significant progress in automating digital tasks through the utilization of computer vision and language models.

By Jinjie Wei, Jiyao Liu, Lihao Liu, Ming Hu, Junzhi Ning, Mingcheng Li, Weijie Yin, Junjun He, Xiao Liang, Chao Feng, Dingkang Yang
llmsagentscomputer-visionbenchmarks
More like this →
arXiv AI
Oct 16

MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments

MAI-Gym is an open, all-in-one platform that unifies GUI agent evaluation, data collection, and online reinforcement learning training across mobile, computer, and web environments. It supports ten established benchmarks, scales to 20,000 parallel environment instances, and enables automated synthesis of diverse GUI applications and tasks with execution-validated verifiers. Using this platform, the authors trained a 9B-parameter agent, MAI-Gym-9B, achieving state‑of‑the‑art performance on MobileWorld and OSWorld-Verified, and demonstrated that training on a mocked train‑ticketing app improves real‑world task success from 28.8% to 48.1% on a physical phone.

By Panrong Tong, Quyu Kong, Pengxiang Li, Tianyu Xia, Hanzhang Zhou, Lei Wang, Chenglin Cai, Long Li, Long Chen, Jianan Zhang, Zhengwen Zeng, Xu Zhang, Gongjie Zhang, Yue Wang, Steven Hoi
agentsreinforcement-learningbenchmarks
More like this →
arXiv AI
Sep 2

UI-Venus-2 Technical Report

UI‑Venus‑2 is a general‑purpose foundation GUI agent that operates across mobile, web, and desktop environments using a unified closed‑loop reasoning‑action framework. The report details how the system expands environment coverage to over 170 multilingual mobile apps and native desktop OSes, scales task generation through a deep‑research pipeline, and enhances verification with trace‑level and sample‑level evaluators that use visual keypoints and multi‑model voting. Safety‑aware mechanisms are also incorporated to control consequential actions, positioning UI‑Venus‑2 as an efficient, open‑source tool for more generalizable, verifiable, and self‑reflective agents in real‑world applications.

By Venus Team, Zhuohan Cai, Haoxing Chen, Jiaxuan Chen, Weizhi Chen, Changlong Gao, Zhangxuan Gu, Yuan Guo, Yusong Hu, Jianrong Jiang, Jianguo Li, Runze Li, Jinzhen Lin, Zhenyu Ma, Changhua Meng, Han Peng, Xinyu Qiu, Shuheng Shen, Zhongyi Shui, Weiqiang Wang, Ming Wen, Zhuoer Xu, Hang Yan, Kaiwen Yang, Ruilin Yao, Nanjun Yu, Zhengwen Zeng, Lianrui Zhang, Yunzhu Zhang, Zhe Zhao, Beitong Zhou
agentsreinforcement-learningmultimodalbenchmarkssafety
More like this →