Hugging Face Trending Papers
How GUI Agents Fail: GraceBench for Execution Stability Diagnosis
Read the original on Hugging Face Trending Papers →The Flow has not summarised this story yet — read it at Hugging Face Trending Papers.
Related stories
arXiv AI
Sep 2
GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments
GUI-CC is a benchmark designed to assess the contextual consistency of GUI world models when used as agent environments, rather than just one‑step next‑screen predictors. It includes two tracks: an offline reference‑action track that rolls models along real mobile GUI trajectories, and an online agent‑loop track where fixed probing agents interact with model‑generated UIs. The benchmark evaluates transition fidelity, plausibility, contextual consistency, and task progress across 500 offline trajectory tasks and 200 online tasks spanning 30 mobile apps.
arXiv AI
Jul 29
Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?
arXiv:2607. 26041v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks.
arXiv AI
Sep 30
GUITAR: Structured Failure Diagnosis of GUI Agents via State Transitions
arXiv:2609.34113v2 Announce Type: replace Abstract: Understanding where and why Graphical User Interface (GUI) agents fail is essential for building more reliable systems, yet current evaluation reli...
arXiv AI
Oct 2
GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolution
arXiv:2610.00948v1 Announce Type: cross Abstract: The executable harness surrounding a GUI model determines how observations are assembled, actions are executed, and verification, recovery, and termi...
arXiv AI
Aug 24
Automated Trajectory Evaluation for Mobile Agents via Step-Level Consequence Reasoning and Aggregation
The paper introduces CRATE, a two‑stage vision‑language model framework that evaluates mobile agents by reasoning about each step’s consequences and aggregating this evidence to assess task completion. It also presents CRATE‑S, an extension that evaluates operational safety. Experiments show CRATE and CRATE‑S outperform existing benchmarks, achieving high F1‑scores on AndroidWorld and MobileRisk datasets.
arXiv AI
Jun 3
AgentLens: Revealing The Lucky Pass Problem in SWE-Agent Evaluation
arXiv:2605. 12925v3 Announce Type: replace-cross Abstract: Evaluation of software engineering (SWE) agents is dominated by a binary signal: whether the final patch passes the tests.