← Back to all news
arXiv AI By Lin Fu, Zheyuan Yang, Tianhui Zhang, Jinbiao Wei, Guo Gan, Boxu Liu, Yilun Zhao, Yu Rong

GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent Environments

Read the original on arXiv AI →

GUI-CC is a benchmark designed to assess the contextual consistency of GUI world models when used as agent environments, rather than just one‑step next‑screen predictors. It includes two tracks: an offline reference‑action track that rolls models along real mobile GUI trajectories, and an online agent‑loop track where fixed probing agents interact with model‑generated UIs. The benchmark evaluates transition fidelity, plausibility, contextual consistency, and task progress across 500 offline trajectory tasks and 200 online tasks spanning 30 mobile apps.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Computer Vision
Aug 31

Benchmarking General Mobile Assistants in Challenging Real-World Scenarios

The paper introduces GMA, a new benchmark for evaluating general mobile assistants in realistic, challenging scenarios. GMA expands on existing benchmarks by offering seven open‑source applications across diverse domains and 300 tasks organized into four difficulty tiers, ranging from simple actions to complex multi‑step workflows. The authors evaluate eight state‑of‑the‑art models, showing that performance drops sharply with task complexity, and conduct ablation studies on harness design—such as context retention and state tracking—to demonstrate how these choices can improve outcomes, especially for demanding workflows.

By Yiqi Zhu, Feiyu Gao, Jiaxing Fan, Jiahui Zeng, Minggang Wu, Chenliang Li, Haiyang Xu, Peng Li, Ming Yan, Yang Liu
agentsmultimodalbenchmarks
More like this →