Hugging Face Blog

Smol2Operator: Post-Training GUI Agents for Computer Use

arXiv AI
Sep 24

Learn How to Act from Your Own Interactions: On-Policy Self-Distillation for GUI Agents

The paper introduces GUI‑SD‑v2, an on‑policy self‑distillation framework that extends previous methods from GUI grounding to multi‑turn GUI interaction. It employs a two‑stage training process that first improves privilege following by jointly optimizing rollouts with and without privileged guidance, then selectively distills step‑specific reasoning and memory guidance via a privilege‑conditioned self‑teacher. Experiments on AndroidWorld and MobileWorld benchmarks demonstrate that GUI‑SD‑v2 outperforms existing OPSD baselines and state‑of‑the‑art methods in Pass@1 and Pass@3 success rates.

By Yan Zhang, Daiqing Wu, Huawen Shen, Liang Li, Gang Cao, Zhi Gong, Wei Dai, Xiaode Zhang, Can Ma, Yu Zhou
arXiv AI
Aug 18

UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations

arXiv:2608. 15930v1 Announce Type: new Abstract: Foundation GUI agents can automate complex digital tasks, but deployment is hindered by scarce and biased training data, ambiguous prompts, and unreliable execution.

By Zihan Ding, Longxu Dou, Qi Gao, Xiangwu Guo, Shengchao Hu, Zilong Huang, Zihang Jiang, Lei Ke, Mengcheng Lan, Weixian Lei, Hanxuan Li, Honglin Li, Xiyun Li, Zaitang Li, Leowei Liang, Xin Luo, Haozhe Ma, Jiayi Mao, Zhoujie Pan, Can Qin, Tianyuan Qu, Weiqi Wang, Wenkai Wang, Yonglin Wang, Yuxin Wang, Chenxu Wu, Yingchen Yu, Chenyu Zhang, Yuhao Zheng