arXiv:2607. 11185v1 Announce Type: new Abstract: Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution.
arXiv:2610.33907v1 Announce Type: new Abstract: Computer-using agents are increasingly capable of performing complex tasks in the digital world. However, high-quality interaction trajectories remain...
Mage‑VL‑Cowork is a 4‑billion‑parameter hybrid computer‑use agent that integrates visual GUI interaction with code and terminal execution in a single policy. It is trained via a progressive five‑stage curriculum—mid‑training, supervised fine‑tuning, and reinforcement learning—starting from a multimodally aligned checkpoint. The training pipeline, Mage‑CUA‑Forge, automates task generation and skill conditioning, producing nearly 300,000 desktop and web tasks while cutting per‑task costs by 82.6%, and the resulting model achieves 66.0% on OSWorld‑Verified, setting a new state‑of‑the‑art for its size.
CUA-Master introduces a nested double-loop recursive improvement framework to scale computer-use agents. The outer flywheel continuously integrates open-world data and model weaknesses to guide data construction and optimization, while the inner flywheel mines instructional videos for seed tasks and propagates them into reusable primitives for diverse task synthesis. The system also employs a feedback-guided snapshot-rollback retry mechanism for reliable trajectory collection, achieving competitive performance on multiple benchmarks and demonstrating continuous gains across iterations.
OpenClaw has emerged as a leading agent framework for complex task automation, yet it faces insufficient cross-platform GUI interaction support and a well-built self-evolution mechanism. These flaws limit its adaptation to diverse device ecosystems and prevent performance improvements through continuous learning from execution experience.
arXiv:2610.16319v1 Announce Type: new Abstract: Real-world hybrid web tasks need both information gathering and page interaction, yet search-based and operation-based agents are developed and evaluat...
arXiv:2608. 15930v1 Announce Type: new Abstract: Foundation GUI agents can automate complex digital tasks, but deployment is hindered by scarce and biased training data, ambiguous prompts, and unreliable execution.
AnyAct introduces a universal action layer that consolidates diverse tool capabilities into a self‑evolving action space for AI agents operating in open‑world environments. It tackles the scale dilemma, tool non‑stationarity, and heterogeneous feedback by using hierarchical progressive retrieval and test‑time reliability evolution, while a heterogeneous observation grounding module unifies multi‑modal feedback. Evaluations on LiveMCPBench and the newly created OSMCP benchmark show state‑of‑the‑art performance, with significant gains in task success rate and reduced execution steps, especially for models with limited native capabilities.
arXiv:2606. 17321v1 Announce Type: new Abstract: Training computer-use agents (CUAs) -- models that interact with graphical desktops through screenshots and keyboard/mouse actions -- requires large-scale, diverse trajectory data collected in full desktop environments.
arXiv:2511. 20297v2 Announce Type: replace Abstract: Large Language Model (LLM)-based agents are increasingly capable of complex, multi-step tasks such as GUI automation, tool use, and data manipulation, yet they cannot learn from experience: each new session rediscovers solutions from scratch.
The paper introduces MAGE, a framework that uses user manuals to enhance computer use agents (CUAs) for new software. MAGE retrieves relevant manual content during execution, generates procedural guidance, and adapts the agent policy through interaction. In training, it converts manuals into executable task-plan pairs to guide rollouts, reducing the need for human‑annotated demonstrations and improving task success rates across six applications.
Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical user interface (GUI) interactions, which are often...