arXiv AI
cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use Agents
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents
arXiv:2606. 28480v1 Announce Type: cross Abstract: As large language models and harness frameworks continue to advance, agents operating in terminals are increasingly capable of performing a broader range of general computer-use tasks beyond coding.
arXiv Machine Learning
Jun 2
Multi-Agent Computer Use
arXiv:2606. 01533v1 Announce Type: cross Abstract: Computer use agents (CUAs) today are primarily deployed as single serial agents.
arXiv AI
Sep 7
CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI Agents
CUA-Universe is a scalable environment-to-data pipeline that transforms real desktop software into hybrid GUI+CLI environments, enabling agents to coordinate visual inspection with command-line operations. It includes App-Forge for reproducible VMs and CLI surfaces, Task-Weave for generating diverse hybrid tasks, and Path-Steer for efficient rollouts and trajectory harvesting. Training on this data improves agent success and efficiency across multiple benchmarks, demonstrating the value of hybrid interaction.
arXiv AI
Jul 14
SCALECUA: Scaling Computer Use Agents with Verifiable Task Synthesis and Efficient Online RL
arXiv:2607. 11185v1 Announce Type: new Abstract: Computer use agents (CUAs) are emerging as a powerful interface for automating complex digital workflows through visual perception and GUI execution.
arXiv AI
Jun 9
WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
arXiv:2606. 09426v1 Announce Type: new Abstract: Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools.
arXiv AI
Sep 2
UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.