arXiv AI
TerminalWorld: Benchmarking Agents on Real-World Terminal Tasks
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jun 30
TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents
arXiv:2606. 28480v1 Announce Type: cross Abstract: As large language models and harness frameworks continue to advance, agents operating in terminals are increasingly capable of performing a broader range of general computer-use tasks beyond coding.
arXiv AI
Jun 30
OSWorld2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
arXiv:2606. 29537v1 Announce Type: new Abstract: Existing computer-use benchmarks fail to capture the realism, complexity, and long-horizon demands of real-world computer use, limiting their ability to reveal the limitations of frontier agents.
arXiv AI
Jul 15
OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
arXiv:2606. 29537v2 Announce Type: replace Abstract: Existing computer-use benchmarks fail to capture the realism, complexity, and long-horizon demands of real-world computer use, limiting their ability to reveal the limitations of frontier agents.
arXiv AI
Jul 13
Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
arXiv:2607. 08964v1 Announce Type: new Abstract: AI agents have become capable of autonomously completing short, well-specified tasks.
arXiv Computation and Language
Oct 12
Scaling Agentic Data for Long-Horizon Terminal Intelligence
The paper introduces TerminalHorizon, an automated engine that synthesizes long‑horizon trajectories for terminal agents by reconstructing environments from real‑world professional work and progressively expanding tasks to require further exploration and improvement. Using this engine, the authors create 1.5K task environments and fine‑tune Qwen3.5‑27B, producing TerminalHorizon‑27B, which shows significant performance gains across 12 benchmarks in coding, agentic production tasks, and reasoning. Case studies demonstrate the model’s ability to experiment beyond initial solutions, refine methods based on feedback, and preserve effective improvements in final deliverables.
arXiv AI
Sep 4
Terminal-Universe: Turning Agent Trajectories into Scalable Terminal Environments
Terminal-Universe is a framework that transforms large collections of terminal‑based agent trajectories into reusable, executable environments. By replaying recorded file operations and completing missing files, it reconstructs the original workspace and task, then synthesizes new tasks and multi‑round interactions. The resulting 37.3k task‑sufficient environments enable significant performance gains when fine‑tuning language models on terminal‑based benchmarks.