MobileWorldSafety is a benchmark that evaluates the safety of large language model–powered GUI agents on Android by exposing them to 142 real-world risk tasks involving environmental injection attacks. The benchmark uses a two‑stage verification pipeline—rule‑based checks for clear cases and an LLM judge for ambiguous ones—to distinguish safety failures from capability failures. Experiments on six agents show high vulnerability, with attack success rates between 40.4% and 66.9%, highlighting that current agents often fail to remain safe when faced with adversarial content presented as normal mobile context.
By Sujin Chen, Lijun Li, Tianyi Du, Jing Shao
arXiv:2608. 07775v1 Announce Type: new Abstract: Mobile agents have achieved promising results on clean online benchmarks such as AndroidWorld, yet their performance often degrades sharply in real-world deployment due to environmental variations and imperfect interface conditions.
By Xiaoou Liu, Longchao Da, Hanyang Chen, Yuan Ling, Hua Wei
arXiv:2609.39547v1 Announce Type: new
Abstract: GUI agents built on large language and vision-language models still struggle on unseen applications and complex multi-step tasks, as completing real GU...
By Bo Han, Qianyi Wang, Shuai Liu, Xiong Zifan, Changqiao Wu, Yuanfa Li, Pengzhi Gao, Wei Liu, Jian Luan, Heng Qu, Yunpeng Song, Zhongmin Cai
arXiv:2608.22847v1 Announce Type: new
Abstract: Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottl...
By Long Zhang, Yuhan Chen, Chaoran Zhang, Wanxia Cao, Kun Huang, Pengzhi Gao, Wei Liu, Jian Luan, Chenliang Li, Lixin Zou
Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottlenecked by two fundamental issues: current data s...
arXiv:2609.22724v1 Announce Type: cross
Abstract: Mobile agents powered by foundation models now automate complex, multi-step workflows on real devices, but their trajectories can violate app-specifi...
By Changyue Jiang, Jiayi Wang, Xin Wen, Jiarun Dai, Geng Hong, Xudong Pan