← Back to all news
arXiv AI By Xiaoyi Chen, Yifei Gao, Yang Xu, Zhenlong Wang, Jian Yan, Jiming An, Xingxing Song, Liqun Deng, Xiaojun Meng, Jiansheng Wei, Yi Zhang, Jitao Sang

GUITestScape: Towards Open-set Evaluation on Exploratory GUI Testing

Read the original on arXiv AI →

GUITestScape introduces an interactive benchmark for exploratory GUI testing, covering 61 Android apps and 508 defects that include both interaction and display types. It also presents GUIJudge, an open‑set evaluator that breaks down an agent’s testing trajectory into independently diagnosable capabilities, enabling process‑aware assessment beyond predefined annotations. Experiments show GUIJudge outperforms baselines, revealing detection as the main bottleneck and demonstrating that adding GUIJudge’s verifiers to existing agents improves detection without retraining.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv AI
Sep 10

Improving Proficiency and Efficiency of Android GUI Agents via Self-Generating Tool Actions

The paper introduces DroidTool, a framework that enables Android GUI agents to automatically generate and integrate tool actions—Python functions that manipulate application states—into their hybrid action space. DroidTool follows an agentic workflow with stages of proposal, implementation, test generation, execution, and repair, and it constructs relational tests to ensure comprehensive verification. Experiments on benchmarks such as AndroidWorld, B-MoCA, and MobileSafetyBench show that agents using DroidTool’s self-generated tools outperform GUI-only agents by about 4.47% in performance while reducing interactions by roughly 20.05%.

By Juyong Lee, Woogyeol Jin, Kimin Lee
agentsbenchmarks
More like this →
arXiv AI
Sep 7

ARIA - An Agentic Framework for Autonomous Testing of Infotainment Systems

ARIA is a multi‑agent large‑language‑model framework that autonomously runs end‑to‑end visual tests on Android infotainment systems. From simple scenario sentences, it executes interactions, generates reproducible scripts, and produces detailed reports with visual evidence. In evaluation on a manufacturer’s device, ARIA achieved a 93.3% completion rate, correctly identified all known defects, and demonstrated lower false‑positive rates compared to a single‑agent baseline.

By Ant\'onio Azevedo, Bruno Lima, Jo\~ao Pascoal Faria
llmsagentssafety
More like this →
arXiv Computation and Language
Sep 21

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

arXiv:2609.22000v1 Announce Type: new Abstract: Computer-use agents (CUAs) have advanced along two separate lines: graphical interaction and software development through code and the command line. Re...

By Shuai Bai, Jiayong Deng, Yikun Fu, Chang Gao, Xuhao Hu, Mianqiu Huang, Yizhen Jiang, Yuheng Jing, Dehui Kong, Keliang Li, Ning Li, Wanli Li, Dayiheng Liu, Dunjie Lu, Changwei Luo, Que Shen, Zheyuan Wang, Zijian Wang, Jie Wu, Gao Wu, Zhihui Xie, Rui Xie, Haiyang Xu, An Yang, Jiakang Yuan, Yanming Zhang, Jiajun Zhang, Xi Zhang, Zhenru Zhang, Zhuo Zhen, Mingkang Zhu, Bowen Zhou
llmsagentsbenchmarks
More like this →