Jev-Mobile introduces a new approach for mobile GUI agents by separating high‑frequency lightweight execution from low‑frequency vision‑language model (VLM) planning. The VLM sets local goals, the accessibility tree provides a structured action space, and Jev, a fast typed decision model, repeatedly selects actions within this space, allowing multiple GUI actions per VLM decision. On the AndroidWorld task suite, Jev-Mobile achieves 79% task success, reduces mean end‑to‑end execution time by 32.7%, and cuts model API cost by 73.4% compared to a step‑wise VLM baseline.
By Linghua Zhang
Hybrid Computer Use (HCU) is a unified interface that enables general agents to interact with computers through three complementary channels: programmatic code and command-line tools, semantic accessibility (AX) features, and visual graphical user interfaces (GUI). The design emphasizes efficient AX interaction, batch execution, and AX diffs to reduce round trips, while a supplementary skill guides agents in selecting and coordinating these channels. On the OSWorld 2.0 benchmark, HCU outperforms other systems in both efficiency and task success, demonstrating the benefits of flexible, multi‑channel computer‑use interfaces.
By Haoqing Wang, Xiaoyi Zhang, Sicheng Fan, Zhexin Zhang, Xinjie Zhang, Jiahao Li, Bin Li, Yan Lu
arXiv:2607. 13027v1 Announce Type: cross Abstract: Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action.
By Hongru Cai, Yongqi Li, Ran Wei, Wenjie Li
GUI-CC is a benchmark designed to assess the contextual consistency of GUI world models when used as agent environments, rather than just one‑step next‑screen predictors. It includes two tracks: an offline reference‑action track that rolls models along real mobile GUI trajectories, and an online agent‑loop track where fixed probing agents interact with model‑generated UIs. The benchmark evaluates transition fidelity, plausibility, contextual consistency, and task progress across 500 offline trajectory tasks and 200 online tasks spanning 30 mobile apps.
By Lin Fu, Zheyuan Yang, Tianhui Zhang, Jinbiao Wei, Guo Gan, Boxu Liu, Yilun Zhao, Yu Rong
arXiv:2512. 12634v4 Announce Type: replace Abstract: Mobile GUI Agents, AI agents capable of interacting with mobile applications on behalf of users, have the potential to transform human computer interaction.
By Youngmin Im, Byeongung Jo, Jaeyoung Wi, Seungwoo Baek, Tae Hoon Min, Joo Hyung Lee, Sangeun Oh, Insik Shin, Sunjae Lee
arXiv:2610.01215v1 Announce Type: new
Abstract: GUI agents require high-quality interaction trajectories to learn how software environments respond to actions, maintain state, and support multi-step...
By Cheng Yang, Yifan Wu, Yutao Huang, Zhaohua Zhang, Beiduo Chen, Muxi Chen, Chenchen Zhao, Hexuan Deng, Haolin Yang, Geyuan Zhu, Sa Zhu, Jianhuan Zhuo, Qiuyong Xiao, Jianhao Ruan, Yiran Peng, Jiayi Zhang, Tian Ye, Xinlei Yu, Tianwen Jiang, Jihong Zhang, Yuyu Luo
GUI agents require high-quality interaction trajectories to learn how software environments respond to actions, maintain state, and support multi-step workflows. However, the diversity of available tr...
Mobile GUI agents can execute tasks from natural-language instructions, but their evaluation remains difficult to make both realistic and reproducible. Existing benchmarks typically trade off these go...
arXiv:2608. 08939v1 Announce Type: new Abstract: The rise of autonomous AI agents represents a major paradigm shift in how users interact with mobile devices.
By Rahul Deivasigamani, Sayeda Faatin Alvi, Derqui Andrea, Kaushal Punjabi, Stjepan Picek
APPSim-Bench is a new benchmark for mobile GUI agents that uses controllable simulated apps to balance realism and reproducibility. It includes 557 tasks across 17 popular Chinese and English apps, with a coding-agent-assisted and human-verified workflow that ensures deterministic evaluation. Evaluation of 19 agents shows that autonomous mobile execution is still far from perfect, with the best model completing only 50.27% of tasks and many failures in longer workflows and numerical reasoning.
By Jintian Feng, Long Chen, Xiao Yu, Jiayi Dai, Chenglong Liu, Haoru Wang, Zizhen Xue, Yuxuan Shi, Ziyang Wang, Yichen Gong
The paper introduces GMA, a new benchmark for evaluating general mobile assistants in realistic, challenging scenarios. GMA expands on existing benchmarks by offering seven open‑source applications across diverse domains and 300 tasks organized into four difficulty tiers, ranging from simple actions to complex multi‑step workflows. The authors evaluate eight state‑of‑the‑art models, showing that performance drops sharply with task complexity, and conduct ablation studies on harness design—such as context retention and state tracking—to demonstrate how these choices can improve outcomes, especially for demanding workflows.
By Yiqi Zhu, Feiyu Gao, Jiaxing Fan, Jiahui Zeng, Minggang Wu, Chenliang Li, Haiyang Xu, Peng Li, Ming Yan, Yang Liu
JarvisGUI is a new benchmark that tests GUI agents on cross-device workflows involving Android, Windows, and Ubuntu, requiring transfer of intermediate results and coordination across heterogeneous platforms. It formulates tasks as input-output transformations under a lightweight type system, enabling automatic composition of multi-step, cross-device workflows and dynamic evaluation within a unified framework. The benchmark reveals that state-of-the-art open-source GUI agents struggle with state-transfer awareness, cross-platform contextual reasoning, and long-horizon dependency management, exposing a critical capability gap invisible to existing benchmarks.
By Zixiang Chen, Yuheng Lu, Zihao Cheng, Zeming Liu, Jizeng Bai, Ziye Huang, Zhiyin Lin, Zihan Li, Yuhang Guo, Yunhong Wang, Haifeng Wang