The paper introduces MAGE, a framework that uses user manuals to enhance computer use agents (CUAs) for new software. MAGE retrieves relevant manual content during execution, generates procedural guidance, and adapts the agent policy through interaction. In training, it converts manuals into executable task-plan pairs to guide rollouts, reducing the need for human‑annotated demonstrations and improving task success rates across six applications.
By Chulhoon Yoon, Kyungri Park, Jinyeong Yun, Daeyun Jeong, Sukmin Yun, Woohwan Jung
arXiv:2611.02102v3 Announce Type: replace
Abstract: Scaling computer-use agents from individual actions to complex software work requires experience that connects decisions to complete outcomes. We p...
By Chaoyun Zhang, Bingxuan Li, Pinxue Guo, Yangfan Gao, Zikun Cai, Tianci Xie, Hui Li, Yuqi Huang, Yifei Cao, Bo Mao, Tian Tang, Qichang Zheng, Tongtong Liu, Xiaoxing Wang, Jinxiu Liu, Lizhi Lin, Zhou Shao, Maosen Zhou, Miao Zheng, Shuo Zhang, Yangji He, Kepeng Lei, Ning Zhao, Wei Shen, Zhenjiang Jin, Xinyue Zhang, Peng Sun, Zhenhua Han, Rui Zheng, Hang Yan
CUA-Universe is a scalable environment-to-data pipeline that transforms real desktop software into hybrid GUI+CLI environments, enabling agents to coordinate visual inspection with command-line operations. It includes App-Forge for reproducible VMs and CLI surfaces, Task-Weave for generating diverse hybrid tasks, and Path-Steer for efficient rollouts and trajectory harvesting. Training on this data improves agent success and efficiency across multiple benchmarks, demonstrating the value of hybrid interaction.
By Haoting Shi, Wenhao Wang, Weicheng Fang, Yaozhong Liang, Tian Jin, Pengxiang Zhao, Guangyi Liu, Siheng Chen, Yanfeng Wang
arXiv:2607. 04425v2 Announce Type: replace-cross Abstract: Recent advances in multimodal foundation models and agent systems have driven GUI agents from single-platform task execution toward cross-platform interaction.
By Niu Lian, Tongbo Chen, Zhehao Yu, Chengzhen Duan, Fazhan Liu, Hui Liu, Pei Fu, Jian Luan, Heng Qu, Shu-Tao Xia, Jinpeng Wang
arXiv:2607. 11185v1 Announce Type: new Abstract: Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution.
By Bowen Lv, Xiao Liu, Yanyu Ren, Hanyu Lai, Bohao Jing, Hanchen Zhang, Yanxiao Zhao, Shuntian Yao, Jie Tang, Yuxiao Dong
Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical user interface (GUI) interactions, which are often...
arXiv:2609.38008v1 Announce Type: new
Abstract: Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical us...
By Tongbo Chen, Junbo Niu, Zhengxi Lu, Niu Lian, Fei Tang, Yuchen Yan, Yike Hong, Yong Du, Yizhou Liu, Bofan Chen, Yongliang Shen
ComputerSD is an online self‑distillation method for computer‑use agents that leverages real‑time feedback from executed GUI transitions. It uses a fine‑tuned GUI analyzer to generate guidance and a step‑level value score after each action, combining token‑level OPSD with trajectory‑level GRPO in an asynchronous training framework. On the OSWorld‑Verified benchmark, ComputerSD improves performance over outcome‑only GRPO by 1.9 and 4.1 percentage points on Qwen3‑VL‑8B‑Thinking and EvoCUA‑8B backbones, and shows strong generalizability in out‑of‑distribution tests.
By Yong Du, Tongbo Chen, Zhengxi Lu, Yizhou Liu, Bofan Chen, Tao Jiang, Wenhao Xu, Yongliang Shen
The paper introduces EvoSkill-GUI, a training‑free framework that enables GUI agents to evolve their skills during deployment. Each skill is packaged with metadata, executable plans, and recovery rules, and the system follows a reflect‑revise‑reuse loop where the agent instantly revises skills based on execution feedback. Experiments on MobileWorld, AndroidWorld, and OSWorld show consistent performance gains up to +16.2% without any additional training.
By Bofan Chen, Boxuan Zhang, Fei Tang, Zhengxi Lu, Yong Du, Tongbo Chen, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
LiteGUI is a new framework for training lightweight GUI agents that tackles the long‑horizon, multi‑path nature of GUI interactions. It introduces a two‑stage post‑training pipeline: Guided On‑Policy Distillation, which uses human‑verified multi‑solution action annotations to provide privileged teacher guidance at each state, and Multi‑Solution Dual‑Level GRPO, which combines action‑level supervision with history‑conditioned planning‑quality supervision while handling multiple valid actions. The authors also present a scalable data generation pipeline and the Lite‑Dataset, enabling extensive evaluation across GUI benchmarks and demonstrating substantial performance gains over existing training paradigms.
By Yubin Wu, Zicheng Cai, Liping Ning, Hua Wang, Zhi Chen, Yaohua Tang, Hao Chen
Online training enables computer-use agents (CUAs) to improve through interaction with executable environments. However, existing methods primarily rely on sparse outcome rewards, which provide no sup...
CUA-Master introduces a nested double-loop recursive improvement framework to scale computer-use agents. The outer flywheel continuously integrates open-world data and model weaknesses to guide data construction and optimization, while the inner flywheel mines instructional videos for seed tasks and propagates them into reusable primitives for diverse task synthesis. The system also employs a feedback-guided snapshot-rollback retry mechanism for reliable trajectory collection, achieving competitive performance on multiple benchmarks and demonstrating continuous gains across iterations.