MobileWorldBench introduces a benchmark for evaluating vision‑language models as world models in mobile GUI agents, shifting from pixel‑based predictions to natural‑language state transitions. The authors release MobileWorld, a 1.4 million‑sample dataset that enhances VLM world‑modeling performance. They also present a framework that integrates these semantic world models into mobile agent planning, showing improved task success rates.
arXiv:2601.04035v2 Announce Type: replace Abstract: Mobile GUI agents have shown strong potential in real-world automation and practical applications. However, most existing agents remain reactive, m...
arXiv:2606. 12817v2 Announce Type: replace Abstract: Understanding the digital world on mobile devices is shifting from static UI perception to dynamic action comprehension.
arXiv:2608.29483v1 Announce Type: cross Abstract: Modern Vision-Language Models (VLMs) perform well above the human baseline in image geolocalization, a task critically important in disaster response...
arXiv:2608. 05891v1 Announce Type: new Abstract: Mobile GUI agents can operate apps through pixel perception and touch actions, making them a promising interface for collecting and improving long-horizon mobile interaction policies.
GUI-CC is a benchmark designed to assess the contextual consistency of GUI world models when used as agent environments, rather than just one‑step next‑screen predictors. It includes two tracks: an offline reference‑action track that rolls models along real mobile GUI trajectories, and an online agent‑loop track where fixed probing agents interact with model‑generated UIs. The benchmark evaluates transition fidelity, plausibility, contextual consistency, and task progress across 500 offline trajectory tasks and 200 online tasks spanning 30 mobile apps.
Jev-Mobile introduces a new approach for mobile GUI agents by separating high‑frequency lightweight execution from low‑frequency vision‑language model (VLM) planning. The VLM sets local goals, the accessibility tree provides a structured action space, and Jev, a fast typed decision model, repeatedly selects actions within this space, allowing multiple GUI actions per VLM decision. On the AndroidWorld task suite, Jev-Mobile achieves 79% task success, reduces mean end‑to‑end execution time by 32.7%, and cuts model API cost by 73.4% compared to a step‑wise VLM baseline.
arXiv:2606. 29705v1 Announce Type: new Abstract: Data, as the fundamental substrate of modern intelligence, has greatly driven the development of current foundation models.
Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottlenecked by two fundamental issues: current data s...
arXiv:2511. 07332v2 Announce Type: replace-cross Abstract: Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen elements.
arXiv:2608.22847v1 Announce Type: new Abstract: Vision-Language Models (VLMs) based GUI agents stand to benefit significantly from online reinforcement learning (RL). However, their training is bottl...
arXiv:2606. 12817v1 Announce Type: new Abstract: Understanding the digital world on mobile devices is shifting from static UI perception to dynamic action comprehension.
arXiv:2607. 03461v1 Announce Type: cross Abstract: World models aim to capture environment dynamics in ways that support perception, reasoning, and action, and have recently become a central direction in Vision-Language-Action-World (VLAW) modeling.