arXiv AI
Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 13
Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading
arXiv:2607. 08964v1 Announce Type: new Abstract: AI agents have become capable of autonomously completing short, well-specified tasks.
arXiv Computation and Language
1d ago
Interlock: From Local Correctness to Task Completion in Long-Horizon Agents
Interlock is a framework that transforms long‑horizon tasks into typed completion graphs, linking evidence, decisions, and required effects through grounding, coupling, and realization edges. It compiles these graphs into a resettable workspace, a public request, and an evaluation rubric, and uses source‑coverage, public‑input, and agent‑as‑judge reviews to ensure that all mandatory requirements are met. Fine‑tuning Qwen3.6 backbones on Interlock‑generated trajectories improves pass rates on AutomationBench, JobBench, and Harvey LAB, showing that the synthesized experience benefits external task environments.
arXiv AI
Aug 18
StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling
arXiv:2608. 15089v1 Announce Type: new Abstract: Long-horizon agents can fail even when their underlying models can solve the constituent steps.
arXiv AI
1d ago
SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition
arXiv:2610.14966v1 Announce Type: new Abstract: Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evalua...
arXiv AI
Oct 5
Self-Supervised Scaling of Terminal Environments for Scientific Domains
The paper introduces a self‑supervised framework called software‑in‑the‑loop reconstruction (SWR) that automatically generates reference outputs and verification targets for terminal agents by leveraging existing scientific software workflows. SWR executes multiple input configurations, partitions cases into public observations and hidden evaluations, and uses a hierarchical verifier to assess agent‑generated programs against hidden workflow outputs. The authors demonstrate the approach on 500 workflows across six domains, achieving significant performance gains on the Terminal‑Bench 2 benchmark with the Qwen3.8‑Max and Qwen3.8‑27B models.
arXiv Computation and Language
4d ago
Scaling Agentic Data for Long-Horizon Terminal Intelligence
The paper introduces TerminalHorizon, an automated engine that synthesizes long‑horizon trajectories for terminal agents by reconstructing environments from real‑world professional work and progressively expanding tasks to require further exploration and improvement. Using this engine, the authors create 1.5K task environments and fine‑tune Qwen3.5‑27B, producing TerminalHorizon‑27B, which shows significant performance gains across 12 benchmarks in coding, agentic production tasks, and reasoning. Case studies demonstrate the model’s ability to experiment beyond initial solutions, refine methods based on feedback, and preserve effective improvements in final deliverables.