Hugging Face Blog

Holo1: New family of GUI automation VLMs powering GUI agent Surfer-H

arXiv AI
Aug 6

Terminal Agents Suffice for Enterprise Automation

arXiv:2604. 00073v3 Announce Type: replace-cross Abstract: There has been growing interest in building agents that can interact with digital platforms to execute meaningful enterprise tasks autonomously.

By Patrice Bechard, Orlando Marquez Ayala, Emily Chen, Jordan Skelton, Sagar Davasam, Srinivas Sunkara, Vikas Yadav, Sai Rajeswar
arXiv AI
Sep 28

VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automation

VLAA-GUI is a modular framework for autonomous GUI agents that addresses early stopping and repetitive loops by integrating three core components: a Completeness Verifier, a Loop Breaker, and an on-demand Search Agent. The framework also includes optional Coding and Grounding Agents for specialized tasks. Evaluations on five backbones across Linux and Windows benchmarks show strong performance, with some models surpassing human results and the Loop Breaker significantly reducing wasted steps.

By Qijun Han, Haoqin Tu, Zijun Wang, Haoyue Dai, Yiyang Zhou, Nancy Lau, Alvaro A. Cardenas, Yuhui Xu, Ran Xu, Caiming Xiong, Zeyu Zheng, Huaxiu Yao, Yuyin Zhou, Cihang Xie
arXiv AI
Sep 3

OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human Demonstrations

OmegaUse‑SOP is a human‑in‑the‑loop system that transforms expert demonstrations of professional computer use into reusable skills for graphical user interface (GUI) agents. It refines demonstrations through four modules—Observe, Reason, Configure, and Execute—to record multimodal traces, abstract them into semantic steps, incorporate domain rules, and execute the resulting skills with verification. In a collaboration with a power‑sector client, OmegaUse‑SOP was tested on photovoltaic simulation workflows in PVsyst 7.2, showing improved reliability for GUI agents on professional SOP tasks.

By Yixiong Xiao, Lang An, Hucheng Yang, Pinxue Ma, Yongquan Chen, Jingjia Cao, Yusai Zhao, Ting Wang, Ting Liu, Siqi Bao, Jingbo Zhou, Hua Wu