arXiv:2610.34238v1 Announce Type: cross
Abstract: In this paper, we introduce $\Delta\tau$-Bench, an executable agent benchmark that models the structural evolution of large tool ecosystems. In $\Del...
By Shuofei Qiao, Xuanguang Pan, Yijun Chen, Wendi Feng, Bin Wu, Runnan Fang, Yuqi Zhu, Shumin Deng, Emine Yilmaz
arXiv:2609.24663v1 Announce Type: new
Abstract: Self-evolving agents convert interaction feedback into persistent artifacts, such as memories or skills, which in turn guide subsequent decisions. As t...
By Hongqiang Lin, Chao Liu, Xiaofan Bai, Xuan Jin, Yuhong Li, Nenggan Zheng, Xipeng Cao
Self-evolving agents convert interaction feedback into persistent artifacts, such as memories or skills, which in turn guide subsequent decisions. As these artifacts are iteratively updated throughout...
DynaRev is a new benchmark comprising 181 tasks designed to evaluate web agents when users revise their goals during interaction. It introduces semantic milestones to trigger additive, unconditional, and runtime-conditioned revisions, and assesses agents on correctness, responsiveness, and efficiency via trajectory‑grounded evaluation. Experiments with five models show that agents often only partially satisfy revised tasks, with performance varying by task source, revision type, and model, and highlight the benefits of instruction rewriting versus browser resets.
By Shiqi Zhang, Guochen Zhou, Zesheng Wei, Hai Zhao, Zhisong Zhang
The paper investigates how coding agents can self‑evolve their harnesses—software, configuration, and instructions—by reviewing completed sessions and proposing changes. Using Claude Code on 30 SWE‑bench issues, the authors analyzed 125 proposals from the
etro skill, implemented 24, and evaluated 12 changes on subsequent tasks. The results show mixed effectiveness: some changes eliminated targeted problems, others reduced or had no impact, and some added unrelated work, indicating that self‑evolution requires mechanisms to assess and refine changes.
By Christoph Treude, Sebastian Baltes
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.
By Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo, Jingyi Yang, Yi Liu, Tingfeng Hui, Xinyu Yuan, Li Sun, Sen Su, Jing Shao