arXiv:2606. 17321v1 Announce Type: new Abstract: Training computer-use agents (CUAs) -- models that interact with graphical desktops through screenshots and keyboard/mouse actions -- requires large-scale, diverse trajectory data collected in full desktop environments.
arXiv:2609.38008v1 Announce Type: new Abstract: Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical us...
arXiv:2607. 14186v2 Announce Type: replace-cross Abstract: Synthesizing training data to scale agent capabilities in LLM post-training is bottlenecked by substrate-bound task synthesis: tasks are generated from fixed tools, repositories, or skill graphs, so expanding coverage requires manual substrate engineering, transferring to a new domain demands bespoke infrastructure, and the resulting distributions inherit substrate biases rather than reflecting real-world demand.
arXiv:2607. 26041v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly act through desktop GUIs to complete long-horizon tasks.
Computer use agents (CUAs) have demonstrated strong capabilities in completing digital tasks. However, existing CUAs either rely solely on graphical user interface (GUI) interactions, which are often...
arXiv:2608. 15930v1 Announce Type: new Abstract: Foundation GUI agents can automate complex digital tasks, but deployment is hindered by scarce and biased training data, ambiguous prompts, and unreliable execution.
Terminal-Universe is a framework that transforms large collections of terminal‑based agent trajectories into reusable, executable environments. By replaying recorded file operations and completing missing files, it reconstructs the original workspace and task, then synthesizes new tasks and multi‑round interactions. The resulting 37.3k task‑sufficient environments enable significant performance gains when fine‑tuning language models on terminal‑based benchmarks.
arXiv:2606. 04970v1 Announce Type: cross Abstract: We envision a proactive multi-modal assistant system which gives users real-time step-by-step guidance on a procedural task, autonomously deciding \textit{when} to interrupt, and \textit{how} to coach.
arXiv:2511. 07332v2 Announce Type: replace-cross Abstract: Building reliable computer-use agents requires grounding: accurately connecting natural language instructions to the correct on-screen elements.
CUA-Universe is a scalable environment-to-data pipeline that transforms real desktop software into hybrid GUI+CLI environments, enabling agents to coordinate visual inspection with command-line operations. It includes App-Forge for reproducible VMs and CLI surfaces, Task-Weave for generating diverse hybrid tasks, and Path-Steer for efficient rollouts and trajectory harvesting. Training on this data improves agent success and efficiency across multiple benchmarks, demonstrating the value of hybrid interaction.
arXiv:2608. 15089v1 Announce Type: new Abstract: Long-horizon agents can fail even when their underlying models can solve the constituent steps.
Terminal-Universe is a framework that converts large collections of terminal-based agent trajectories into reusable, executable environments. By replaying recorded file operations and filling missing files with a completion agent, it reconstructs the original workspace and generates new tasks, scaling them in breadth (cross-workspace queries) and depth (multi-round interactions). The resulting 37.3k task-sufficient environments enable significant performance gains when fine‑tuning language models on terminal‑centric benchmarks.
arXiv:2608. 05466v1 Announce Type: new Abstract: High-quality long-horizon training data for terminal agents is expensive to produce, often costing hundreds to thousands of dollars per task, because each task must keep the instruction, environment, reference solution, and verifier mutually consistent.