The paper introduces POrchestra, a proactive agentic orchestration framework that lets sub‑agents request harness adjustments during execution, enhancing adaptability to dynamic environments. It also proposes On‑Policy Orchestration Distillation (OPOD), which uses hindsight feedback from sub‑agents’ requests to provide targeted improvement signals for training the orchestration policy. Experiments on three long‑horizon benchmarks show that POrchestra boosts task success by about 9.1% over the best baseline, and with OPOD the improvement rises to 10.0%, while also improving cost‑performance trade‑offs.
By Xueyang Feng, Chaozhuo Li, Xiaohe Bo, Jiakai Tang, Chenxi Li, Qiwei Ye, Xu Chen
The paper introduces Conservative Meta-Agent Search (CMAS), a method for adapting agent architectures in non‑stationary environments. CMAS retrieves archived designs, proposes typed graph edits, and evaluates them through paired shadow replay with variance‑penalized scoring and a simulated canary stage before promotion. Using the MetaInvest‑Bench simulator, CMAS reduces surrogate dynamic‑oracle regret and improves held‑out utility across 96 financial streams, while maintaining low false‑promotion rates and modest inference cost.
By Ali Atiah Alzahrani
The paper introduces ICRL4AHT, a large-scale benchmark for evaluating In-Context Reinforcement Learning (ICRL) in Ad-Hoc Teamwork (AHT) scenarios using Overcooked-V2. It provides a diverse teammate suite, a reproducible pipeline, and evaluates history-conditioned ICRL algorithms such as Algorithm Distillation and Decision-Pretrained Transformer. The results show that these methods often perform worse than random baselines and do not improve with longer horizons, underscoring the difficulty of strategic inference under partial observability in AHT.
By Yuheng Jing, Kai Li, Ziwen Zhang, Jiajun Zhang, Zeyao Ma, Jiaxi Yang, Lei Zhang, Zhe Wu, Jinmin He, Junliang Xing, Jian Cheng
arXiv:2608. 00881v1 Announce Type: new Abstract: Large language model agents increasingly act through stateful tools, yet model generation and environment execution remain serialized at every step.
By Hao Mark Chen, Jinnan Guo, Wayne Luk, Hongxiang Fan
arXiv:2609. 05019v1 Announce Type: new Abstract: Agents tend to optimize, select, or constrain execution structures before decisive runtime outcomes are observed.
By Tianxing Wang, Mingming Zhao, Shuai Huang, Huiyang Xu, Chaoyue Niu, Shengzhong Liu, Fan Wu
arXiv:2609.06059v1 Announce Type: new
Abstract: As large language models evolve from question-answering systems into general-purpose agents, evaluation must move beyond static answer correctness to a...
By Yu Liu, Zhilin Liu, Zhiwei Yang, Shaojie Zhang, Zheyuan Deng, Tingwei Huang, Zhenbo Luo, Lei Jiang, Yanbing Liu, Pei Fu
arXiv:2610.16288v2 Announce Type: new
Abstract: Frontier LLM agents perform complex interactive tasks, but their inference costs motivate transferring their experience to smaller open-source agents....
By Yuetian Du, Meng Wang, Yucheng Wang, Zhangzhe Chen, Wei Wang, Boyu Yang, Weiqi Zhai, Bing Zhao, Ming Kong, Zhan Zhou, Jie Liu, Qiang Zhu
arXiv:2606. 08340v1 Announce Type: new Abstract: As language models are increasingly deployed as autonomous agents, they must coordinate with others over long horizons in open-ended interactive tasks.
By Kale-ab Abebe Tessera, Andras Szecsenyi, Cameron Barker, Alexander Rutherford, Davide Paglieri, Aidan Scannell, Henry Gouk, Elliot J. Crowley, Tim Rockt\"aschel, Amos Storkey
arXiv:2608. 15089v1 Announce Type: new Abstract: Long-horizon agents can fail even when their underlying models can solve the constituent steps.
By Ziheng Qin, Yaxin Lu, Zhangyang Atlas Wang, Kai Wang
UniACE is a unified framework that standardizes the evaluation of large language model (LLM) agents by representing each benchmark as an instruction–tool–environment triplet and running models through a shared, task‑agnostic harness in isolated runtimes. It preserves native success criteria, offers an offline mode for dynamic‑resource tasks, and standardizes efficiency metrics, execution records, and failure attribution. Applying UniACE to 7 benchmarks across 24 domains and 15 models revealed significant score shifts, ranking reversals, and sensitivity to evidence representation, highlighting the impact of evaluation configuration on reported agent performance.
By Pengyu Zhu, Lijun Li, Yaxing Lyu, Qianxin Luo, Jingyi Yang, Yi Liu, Tingfeng Hui, Xinyu Yuan, Li Sun, Sen Su, Jing Shao
arXiv:2604. 00830v3 Announce Type: replace-cross Abstract: Test-Time Learning (TTL) enables language agents to iteratively refine their performance through repeated interactions with the environment at inference time.
By Zhanzhi Lou, Hui Chen, Yibo Li, Qian Wang, Bryan Hooi
Opera is a verbal critic framework designed for long‑horizon coding agents, treating each correction as a persistent note that is followed until the problem is resolved. It uses periodic and event‑driven triggers to diagnose issues, audits feedback against visible evidence, and tracks subsequent actions to distinguish compliance from true resolution. In tests, Opera improves the resolve rate of non‑critic agents by up to 12.4, 15.0, and 8.9 percentage points on three benchmarks and provides on‑policy training data that boosts a model’s performance without requiring a critic at inference time.
By Kai Mei, Zhiyuan Hu, Yutong Dai, Juntao Tan, Yifan Zhang, Dingjie Song, Dimitris N. Metaxas, Silvio Savarese, Ran Xu, Zeyuan Chen