Bench2Robust is a framework that augments existing tool‑using LLM benchmarks with scenario‑controlled solvability, adding episodes that require retrying the original path, switching to a fallback tool, or recognizing that no viable path remains. It separates recovery‑policy quality from randomness in failure persistence, and across 7 models and 10 task slices, 69 of 70 model‑subset pairs degrade under tool failures by up to 46.7 percentage points. The authors introduce Bayesian Tool Memory (BTM) for structured runtime recovery context and use reinforcement learning (RL) for learned recovery behavior, achieving up to a 20.7‑point improvement over the base model without sacrificing clean performance.
By Chaoran Chen, Vy Nguyen, Ziji Zhang, Abhinav Gullapalli, Ziyi Wang, Yuxuan Lu, Dakuo Wang, Jing Huang, Zhou Yu, Jin Lai
LoopArena is a benchmark that evaluates how effectively a model (the Controller) can guide a separate coding agent (the Worker) through long-running tasks by monitoring progress, assigning work, and deciding when to stop. It consists of three settings: Type I with 160 low‑cost control questions, Type II with 80 task slices, and Type III with 80 full tasks. In full tasks, the best observed strict success rate is 36.25%, and Controllers can reduce inference cost by an average of 60.6% while maintaining a strong correlation in performance ranking.
By Yi Wang, Haopeng Zhang, Chengxiang Huang, Rui Dai, Kaikui Liu, Piotr Koniusz, Xiangxiang Chu
SkillScriptBench is a 350‑task benchmark that evaluates how well large language models can self‑evolve executable agent skill packages by distinguishing between documentation repair, script repair, and preservation. The benchmark draws from 100 real GitHub skill packages, creating 150 repair tasks with injected script faults and a controlled track of 200 tasks that test clean, documentation‑faulty, script‑faulty, and combined‑faulty states. Experiments show that AST‑Guided Skill Revision—using abstract syntax trees to target relevant code locations—improves repair success by up to 27.7% and yields more consistent outcomes across repeated runs.
whyItMatters":"The benchmark provides a systematic way to measure and improve the reliability of automated skill package maintenance, which is essential for deploying robust LLM agents in real‑world applications."
By Yuxuan Liu, Haoran Li, Yuhao Zhang, Jiahe Guo, Hongyu Luo, Wenbin Hu, Huihao Jing, Kawai Chung, Junle Chen, Changxuan Fan, Qing Zong, Lingyun Xie, Yangqiu Song
The paper presents a DataBrain‑specific prototype that converts natural‑language analytic requests into inspectable dashboards. It combines a Codex plugin and an Agent Runtime that use schema‑constrained model decisions, typed tools, persistent state, and deterministic Hooks for approval, audit, checkpointing, recovery, and failure handling. The pipeline resolves entities, discovers metadata, enforces read‑only SQL, composes dashboards, and applies static checks, dynamic preflight, and browser inspection, with the model proposing actions while deterministic software controls execution and records state transitions.
By Yiou Wu, Zezhi Tang, Ningwei Bai, Liuhaichen Yang
arXiv:2610.17727v2 Announce Type: cross
Abstract: Agent skills turn natural-language instructions into actions, creating a security challenge: legitimate automation and malicious behaviour can look d...
By Jerrold Huang, Junwei Quan, Saiyue Lyu, Zihan Guo, Yingqiang Ge
arXiv:2610.17197v1 Announce Type: cross
Abstract: In the age of "Software 3.0", when a complete program can be rewritten in minutes by an agent, the traditional approach of relying on code as the sol...
By Thomas Bartz-Beielstein
arXiv:2610.14737v1 Announce Type: cross
Abstract: Training language agents on long-horizon interactive tasks is challenging because sparse terminal rewards provide weak credit signals for intermediat...
By Ruolin Chen, Yang Li, Jihang Wang, Zhuoya Zhao, Yuchen Liu, Jiaming Xu, Qian Zhang
arXiv:2610.16146v1 Announce Type: cross
Abstract: We present Unravel, a training-free framework that turns a single image into an interactive 3D scene, where every object is an independent and movabl...
By Hui Shan, Keyang Luo, Binjia Zhou, Ming Li, Kai Li, Xin Wang, Zhenyang Li, Zesong Yang, Weikai Chen, Runze Zhang, Xin Wang, Yanwei Fu, Zhen Chen, Xiangru Huang
arXiv:2610.17643v1 Announce Type: new
Abstract: An enterprise agent that answers under a written policy must choose an action before it says anything: answer, ask for a missing fact, ask for confirma...
By Hyojung Han, Ryangkyung Kang, Youngeun Kim, Seung-Hun Jeon
arXiv:2610.16946v1 Announce Type: cross
Abstract: Descriptions of how people use AI are becoming training data for the next generation of models. We ask whether what a model reads about its users cha...
By Harsh Kumar, George Eilender, Ashton Anderson
arXiv:2610.15492v1 Announce Type: new
Abstract: Skill internalization aims to convert externally specified procedural skills into persistent model capabilities, allowing language-model agents to exec...
By Haomiao Qiu, Yining Tang, Hongyi Wan, Changfeng Sun, Ziyue Qiao, Miao Zhang
arXiv:2610.19133v1 Announce Type: new
Abstract: Indirect prompt injection uses untrusted retrieved content, tool outputs, or agent messages to override an LLM's intended behavior. We show that defens...
By Kshitij Mishra, Salem Lahlou, Nils Lukas
arXiv:2610.17666v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards (RLVR) for multi-turn agents relies on sparse terminal rewards that say little about individual steps....
By Xuyan Ye, Xiyan Fu, Shengda Fan, Jiawei Cui, Yankai Lin, Wei Lu
arXiv:2610.14709v1 Announce Type: cross
Abstract: Large language models are increasingly deployed in applications that act on users' long-term behavior histories, from personal assistants to recommen...
By Yang Xiao, Yusong Sun, Haoyi Wu, Wenyang Hui, Weiqiang Hang, Luqing Wang, Zejian Wang, Zirui Tu, Jingsheng Mao, Shaobo Hu, Yang Luo, Jiale Guo, Jin Zhang, Yudan Liu, Zhaokai Luo, Ren Huan, Yao Hu, Chengyue Jiang, Wenjie Li, Chuan Mu
arXiv:2610.14802v1 Announce Type: cross
Abstract: End-to-end driving policies can achieve strong task performance while offering limited control over their driving style. Driving style describes how...
By Guantian Zheng, Lidong Yu, Tianyu Gao, Jiaxing Chen, Jiashu Li, Chen Lv
arXiv:2610.16224v1 Announce Type: cross
Abstract: Text-to-image models have made remarkable progress in visual fidelity and instruction following, yet complex compositional generation remains challen...
By Zixu Wang, Xiaoran Zhao, Yahui Liu, Feng-ao Wang, Junhao Xiao, You Shu, Bing Liu, Yiting Liu, Songlei Jian, Zhiliang Tian, Dongsheng Li
arXiv:2610.14005v1 Announce Type: new
Abstract: Under the accelerated pace of AI development, any benchmark whose difficulty is calibrated with humans or current AI systems becomes circular, or satur...
By Jose Hernandez-Orallo
arXiv:2610.14576v1 Announce Type: new
Abstract: As frontier LLMs gain more capabilities, the cost to run them also goes up. At the same time, small language models (SLMs) also gain more capabilities...
By Keegan Wang
arXiv:2610.14828v1 Announce Type: new
Abstract: Office workflows are a demanding testbed for self-improving agents because their deliverables must be correct, natively editable, and mutually consiste...
By Zhou Liu, Xingyan Chen, ZhaoYang Han, Zewei Pan, Zeli Su, Ningxin Pan, Xue Hu, Kaixiong Ting, Wentao Zhang
arXiv:2610.15228v1 Announce Type: new
Abstract: Words are increasingly becoming an interface for action: large language models (LLMs) can now invoke tools, operate devices, and make decisions that af...
By Xinnan Dai, Kaiqi Yang, Yaxin Li, Cheng Luo, Wanli Li, Mingqing Xiao, Xinyang Jiang, Daoan Zhang, Kaitao Song, Fengze Liu, Zefan Cai, Caihua Shan