As GUI agents are increasingly deployed for mobile task automation, their reliability depends not only on final task outcomes, but also on whether they reach these outcomes through stable interaction...
arXiv:2606. 31410v1 Announce Type: new Abstract: Graphical user interface (GUI) agents build on vision-language models to complete user tasks end-to-end in real applications through interface actions such as tapping, swiping, text entry, and navigation.
ComponentBench is a new benchmark that evaluates computer‑use agents at the component level on modern web UIs. It contains 97 canonical UI components and 2,910 programmatically verified tasks, along with cleaned human reference trajectories for measuring task success and interaction efficiency. The benchmark also offers a scalable pipeline for auditing structural difficulty and synthesizing failure analyses across tasks and component families.
arXiv:2610.00948v1 Announce Type: cross Abstract: The executable harness surrounding a GUI model determines how observations are assembled, actions are executed, and verification, recovery, and termi...
arXiv:2608. 02643v1 Announce Type: cross Abstract: Computer-use agents (CUAs) operate real desktop and web interfaces through screenshots, mouse and keyboard actions, and stateful UI feedback, yet their failures remain difficult to diagnose and repair.
GUI-CC is a benchmark designed to assess the contextual consistency of GUI world models when used as agent environments, rather than just one‑step next‑screen predictors. It includes two tracks: an offline reference‑action track that rolls models along real mobile GUI trajectories, and an online agent‑loop track where fixed probing agents interact with model‑generated UIs. The benchmark evaluates transition fidelity, plausibility, contextual consistency, and task progress across 500 offline trajectory tasks and 200 online tasks spanning 30 mobile apps.
arXiv:2607. 26041v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks.
ARIA is a multi‑agent large‑language‑model framework that autonomously runs end‑to‑end visual tests on Android infotainment systems. From simple scenario sentences, it executes interactions, generates reproducible scripts, and produces detailed reports with visual evidence. In evaluation on a manufacturer’s device, ARIA achieved a 93.3% completion rate, correctly identified all known defects, and demonstrated lower false‑positive rates compared to a single‑agent baseline.
arXiv:2609.24890v1 Announce Type: cross Abstract: Evaluation of Computer-Use Agents (CUAs) is often limited to the final deliverables they create (at the end of hundreds of steps) and assessed with f...
arXiv:2509.23263v3 Announce Type: replace Abstract: Long-horizon GUI automation remains challenging due to error accumulation over extended interaction sequences. Process Reward Models (PRMs) provide...
Evaluation of Computer-Use Agents (CUAs) is often limited to the final deliverables they create (at the end of hundreds of steps) and assessed with functional verifiers, as seen in OSWorld. However, s...
arXiv:2609.22000v1 Announce Type: new Abstract: Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Re...