OpenGCUA is an open‑stack project that aims to make generalist computer‑use agents (GCUs) fully accessible to the research community. The first release focuses on code execution, allowing agents to use intermediate results and programming skills directly in computer interaction, and introduces harness‑to‑model decision transfer to shift observation and task‑state decisions from fixed rules to the model. On the OSWorld‑v2 benchmark, the full configuration boosts task scores for all eight evaluated models—including GPT‑6 Astra and Claude Opus 5—by an average of +19.6 partial‑score points and cuts steps per task by 53 % compared to batched GUI‑only execution, while also providing a real‑device harness and cross‑device SDK.
whyItMatters":"By releasing the full stack, OpenGCUA enables systematic study and broader experimentation with generalist computer‑use agents, addressing the current limitation of closed implementations and demonstrating significant performance gains across diverse models."
By Hanzhang Zhou, Long Li, Quyu Kong, Lei Wang, Panrong Tong, Long Chen, Tianyu Xia, Gongjie Zhang, Xu Zhang, Chen Liu, Jianan Zhang, Chenglin Cai, Zhengwen Zeng, Steven Hoi, Yue Wang
OS-Helix is an end‑to‑end multimodal agent that unifies graphical user interface (GUI) actions and code execution into a single action space. It is trained on high‑quality hybrid data generated through application‑artifact composition and a decoupled instruction/evaluator pipeline, and further refined with online reinforcement learning using a step‑balanced GSPO algorithm. The resulting models achieve significant performance gains over hybrid baselines on the OSWorld‑Verified benchmark and demonstrate cross‑platform transfer to WindowsAgentArena.
By Bowen Yang, Kaiming Jin, Zichen Ding, Zhenyu Wu, Zehao Li, Zhoumianze Liu, Kanzhi Cheng, Fangzhi Xu, Yanan Sun, Qiushi Sun, Yu Qiao
arXiv:2605. 25160v2 Announce Type: replace Abstract: GUI agents powered by large language models are advancing rapidly, creating urgent needs for evaluation and training based on realistic environments.
By Guohong Liu, Jialei Ye, Pengzhi Gao, Wei Liu, Jian Luan, Yunxin Liu, Yuanchun Li
arXiv:2608. 02352v1 Announce Type: new Abstract: Native computer use offers a general interface for agents to operate almost any software available to people, but requires long-horizon state tracking, large-scale interactive experience, and learning from sparse yet verifiable outcomes.
By Dunjie Lu, Shuai Bai, Tianyi Bai, Sicheng Fan, Chang Gao, Jian Guan, Feng Hu, Mianqiu Huang, Xingyang Huang, Yizhen Jiang, Yuheng Jing, Dehui Kong, Ning Li, Dayiheng Liu, Shixuan Liu, Zheng Liu, Que Shen, Bowen Wang, Junli Wang, Chencan Wu, Rui Xie, Tianbao Xie, Zhihui Xie, Haiyang Xu, An Yang, Tao Yu, Wenzhen Yuan, Xi Zhang, Zhenru Zhang, Mingkang Zhu, Zhaoqing Zhu, Yizhong Cao, Kai Dang, Binyuan Hui, Kaixin Li, Junyang Lin, Haiquan Wang, Zekun Wang, Yiheng Xu, Fan Yan, Mengqi Yuan, Danyang Zhang, Jiajun Zhang, Zhipeng Zhang, Fan Zhou, Fan Zhou
ToolCUA is an end‑to‑end agent that learns to optimally orchestrate GUI actions and tool calls for Computer Use Agents. It introduces a staged training pipeline that first scales interleaved GUI‑tool trajectories from static GUI data, then bootstraps decision making with single‑turn reinforcement learning, and finally refines performance via online agentic RL guided by a tool‑efficient path reward. On the OSWorld‑MCP benchmark, ToolCUA achieves 46.85% accuracy, a 66% relative improvement over the baseline and a 3.9% gain over GUI‑only models, setting a new state of the art for comparable‑scale models.
By Xuhao Hu, Xi Zhang, Haiyang Xu, Kyle Qiao, Jingyi Yang, Xuanjing Huang, Jing Shao, Ming Yan, Jieping Ye
arXiv:2610.33907v1 Announce Type: new
Abstract: Computer-using agents are increasingly capable of performing complex tasks in the digital world. However, high-quality interaction trajectories remain...
By Zhoumianze Liu, Zichen Ding, Bowen Yang, Zehao Li, Zhenyu Wu, Yibo Zhao, Kanzhi Chen, Yian Wang, Yanan Sun, Qiushi Sun, Yu Qiao