arXiv:2607. 13034v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly automate multi-step engineering and informatics workflows, yet they rarely ask how much effort a task actually requires.
By Junjie Yin, Xinyu Feng
AgentHorizon is a benchmark comprising 1,373 computer‑use tasks with instruction‑trajectory pairs collected from 166 hours of human‑recorded activity across three operating systems. The benchmark uses a paired design that swaps instructions to create negative tasks, testing judges’ ability to distinguish successful trajectories from those that completed a similar but incompatible request. Eleven judges were evaluated, with GPT‑5.5 achieving the highest balanced accuracy of 80.9% on the full trajectory subset, while tool‑use benefits some models but harms others, revealing large variability in judges’ performance.
By Xing Han L\`u, Dheeraj Vattikonda, Sina Hajimiri, Fatemeh Pesaran Zadeh, Parishad BehnamGhader, Ghazwa Darwiche, Amirhossein Kazemnejad, Christopher Pal, Alexandre Drouin, Siva Reddy
arXiv:2607. 09711v1 Announce Type: new Abstract: Existing agent benchmarks primarily test task completion, tool use, or skill utility, but do not isolate whether a runtime can convert evidence from its own runs into reusable skills that improve fresh executions after authoring overhead.
By Zhiyuan Peng, Xin Yin, Chenhao Ying, Zhe Cui, Zixiang Ding, Zhenhua Liu, Jiang Wu, Yuan Luo
arXiv:2604. 05336v2 Announce Type: replace Abstract: Models often fail to complete agentic tasks because they lack core capabilities required by the target environment.
By Hangoo Kang, Tarun Suresh, Jon Saad-Falcon, Azalia Mirhoseini
The paper introduces Skillware, an open architectural framework that separates an agent’s intent from its execution by providing modular, versioned software capability bundles. These bundles include typed JSON contracts, deterministic Python code, task directives, datasets, and unit tests, allowing agents to perform tasks reliably without relying on fragile prompt engineering. Experiments across 23 models and 368 benchmarks show that small local models equipped with Skillware outperform larger frontier models, reducing token usage by up to 84.8%, cutting inference costs by up to 97.3%, and eliminating parameter errors.
By Vladimiros Peilivanidis (ARPA Hellenic Logical Systems)
RSI-Opt introduces Recursive Self-Improvement Optimizer Agents that evolve their own training policies by accumulating knowledge from previous runs, including prompts, playbooks, optimizer code, and decision cadences. A meta‑agent rewrites and archives these policies, enabling reusable procedures and code that adapt to new tasks. Across multiple generations and tasks, RSI‑Opt improves optimizer performance, achieving higher z‑scores on evolution suites, better cross‑model transfer, and significant error reductions on PDE benchmarks compared to fixed‑prompt baselines.
By Binghang Lu, Yige Sun, Guang Lin