arXiv AI
Aug 7
StepReflect: Structured UI Transition Reflection for Mobile GUI Agents
arXiv:2608. 05587v1 Announce Type: new Abstract: Autonomous mobile GUI agents require accurate action reflection for reliable long-horizon execution.
Xiaomi-GUI-0 Technical Report
arXiv:2606. 31410v1 Announce Type: new Abstract: Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation.
Related stories
Hugging Face Trending Papers
Aug 6
StepReflect: Structured UI Transition Reflection for Mobile GUI Agents
Autonomous mobile GUI agents require accurate action reflection for reliable long-horizon execution. Existing approaches rely on open-ended multimodal reasoning after each action, which is costly and poorly matched to the structured nature of GUI state transitions.
arXiv AI
Jun 19
ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis
arXiv:2605. 25160v2 Announce Type: replace Abstract: GUI agents powered by large language models are advancing rapidly, creating urgent needs for evaluation and training based on realistic environments.
arXiv AI
Aug 11
UI-MOPD: Multi-Platform On-Policy Distillation for Unified GUI Agents
arXiv:2607. 04425v2 Announce Type: replace-cross Abstract: Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction.
arXiv AI
Jun 15
GUITrans2Act: Understanding User Operational Behaviors from Mobile GUI Interactions with Vision-Language Models
arXiv:2606. 12817v2 Announce Type: replace Abstract: Understanding the digital world on mobile devices is shifting from static UI perception to dynamic action comprehension.
arXiv AI
Jul 15
PalmClaw: A Native On-Device Agent Framework for Mobile Phones
arXiv:2607. 13027v1 Announce Type: cross Abstract: Large Language Model (LLM) agents have moved beyond generating responses to executing multi-step tasks by calling tools, observing the results, and iteratively deciding the next action.
arXiv AI
1d ago
UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations
arXiv:2608. 15930v1 Announce Type: new Abstract: Foundation GUI agents can automate complex digital tasks, but deployment is hindered by scarce and biased training data, ambiguous prompts, and unreliable execution.
arXiv AI
Aug 7
AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents
arXiv:2608. 05891v1 Announce Type: new Abstract: Mobile GUI agents can operate apps through pixel perception and touch actions, making them a promising interface for collecting and improving long-horizon mobile interaction policies.
arXiv AI
Jul 20
SeerGuard: A Safety Framework for Mobile GUI Agents via World Model Prediction
arXiv:2607. 15550v1 Announce Type: new Abstract: Mobile graphical user interface (GUI) agents have demonstrated remarkable capabilities in automating complex tasks, yet they introduce critical safety risks where a single erroneous action can lead to irreversible consequences.
arXiv AI
Jun 12
Reasoning for Mobile User Experience with Multimodal LLMs: Task, Benchmark, and Approach
arXiv:2606. 13192v1 Announce Type: new Abstract: User experience (UX) centered on usability, perceived consistency, and functional clarity is fundamental to real-world user interfaces (UI).
arXiv AI
Jun 12
Teach-and-Repeat: Accurately Extracting Operational Knowledge from Mobile Screen Demonstrations to Empower GUI Agents
arXiv:2606. 12817v1 Announce Type: new Abstract: Understanding the digital world on mobile devices is shifting from static UI perception to dynamic action comprehension.
arXiv AI
Jun 2
MobiBench: Multi-Branch, Modular Benchmark for Mobile GUI Agents
arXiv:2512. 12634v4 Announce Type: replace Abstract: Mobile GUI Agents, AI agents capable of interacting with mobile applications on behalf of users, have the potential to transform human computer interaction.