arXiv AI August 19, 2026 By Michael Solodko, Justin Wagle

ScreenSearch: Uncertainty-Aware OS Exploration

ScreenSearch is a system for exploring desktop GUI states under partial observability. It combines structural screen retrieval, deduplication, and an ambiguity-aware PUCT graph-bandit to expand the reachable frontier while reducing state ambiguity. Across 11 applications, it collected over 1 M screenshots and 30 K deduplicated states, demonstrating that both ambiguity reduction and frontier exploration are essential for effective corpus building.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

arXiv Machine Learning

2d ago

Learning Reliable GUI Agents under Imperfect Priors

arXiv:2609.39547v1 Announce Type: new Abstract: GUI agents built on large language and vision-language models still struggle on unseen applications and complex multi-step tasks, as completing real GU...

By Bo Han, Qianyi Wang, Shuai Liu, Xiong Zifan, Changqiao Wu, Yuanfa Li, Pengzhi Gao, Wei Liu, Jian Luan, Heng Qu, Yunpeng Song, Zhongmin Cai

llms rag agents multimodal benchmarks

arXiv Computer Vision

Sep 10

TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents

arXiv:2609.10297v1 Announce Type: new Abstract: GUI agents accumulate high-resolution screenshots as the trajectory unfolds, increasing inference latency and memory usage. Training-free visual token...

By Yuhao Wang, Mu Qiao, Xindong Zhang, Yunzhi Zhuge, Lei Zhang, Huchuan Lu

agents efficiency benchmarks

arXiv AI

Aug 14

Beyond Retrieval: Query-Conditioned Reuse of Long-Horizon Agent Trajectories

arXiv:2608. 12847v1 Announce Type: new Abstract: Retrieval can identify a past trajectory that may matter, yet it does not specify how an acting agent should use that trajectory after users, entities, constraints, or environment state have changed.

By Yifei Li, Heng Wang, Lingling Zhang, Muye Huang, Xinyu Zhang, Jiashuai Liu, Hang Yan, Rongman Xu

rag agents

arXiv Computer Vision

Aug 28

OS-Marathon: Benchmarking Computer-Use Agents on Vast-Horizon, Repetitive Tasks

OS-Marathon is a new benchmark that tests computer‑use agents on vast‑horizon, repetitive tasks, covering 100 tasks across five scenarios and ten domains. The study shows that current state‑of‑the‑art agents perform poorly on these tasks, and that simply decomposing workflows into subtasks does not solve the problem. Introducing a cost‑friendly personalization method called GraphDemo, which adapts agents from a single human demonstration, improves performance, highlighting the value of human guidance for these challenging tasks.

By Jing Wu, Wenjie Ai, Daphne Barretto, Yiye Chen, Qingyu Chen, Yuhang He, Pranit Chawla, Nicholas Gyd\'e, Yanan Jian, Vibhav Vineet

agents benchmarks

arXiv AI

Jul 29

Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?

arXiv:2607. 26041v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks.

By Abhishek Pillai, Samir Kumar Nayak, Yuan Chen

agents benchmarks

arXiv AI

Jul 16

Experience Memory Graph: One-Shot Error Correction for Agents

arXiv:2607. 13884v1 Announce Type: new Abstract: Large Language Model (LLM) agents have shown remarkable capabilities in autonomous decision-making by generating sequential trajectories of states, actions, and observations.

By Wenjun Wang, Yuchen Fang, Fengrui Liu, Zibo Liang, Kai Zheng

llms agents benchmarks

One email a morning, machine-written

Related stories

Learning Reliable GUI Agents under Imperfect Priors

TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI Agents

Beyond Retrieval: Query-Conditioned Reuse of Long-Horizon Agent Trajectories

OS-Marathon: Benchmarking Computer-Use Agents on Vast-Horizon, Repetitive Tasks

Desktop-Delta Bench: Do Computer-Use Models Understand Desktop GUI Transitions?

Experience Memory Graph: One-Shot Error Correction for Agents