AHEAD is a step‑aware framework that augments reinforcement learning for multi‑turn LLM agents by matching different supervision sources to different step types. The teacher receives environment feedback on all steps and LLM‑generated corrective hints only on error steps, providing finer‑grained guidance than uniform trajectory‑level rewards. Across ALFWorld, WebShop, and Search‑based QA, AHEAD improves task success by 13.3 points on ALFWorld and 11.0 on WebShop at 7B, reaches target success rates faster, and solves tasks within tighter interaction budgets compared to outcome‑only RL and prior self‑distillation baselines.
By Xiaolong Jin, Dingmin Wang, Vijay Lingam, Varun Kumar
arXiv:2610.16523v1 Announce Type: new
Abstract: Training LLM agents to interact with users over multiple turns is challenging because the learning signal depends not only on the agent, but also on th...
By Min Cai, Dan Zhang, Gao Xing, Libin Ma, Zhaohui Xu, Xi Ye
arXiv:2606. 02355v1 Announce Type: new Abstract: Long-horizon LLM agents can benefit from reusable skills, yet existing skill-based methods often rely on external skill generators during training or persistent skill retrieval at inference, increasing engineering complexity, context length, and deployment latency.
By Zhongyu He, Yuanfan Li, Fei Huang, Tianyu Chen, Siyuan Chen, Xingyang Li, Meng Hsuan Yu, Xiangrong Liu, Leyi Wei, Lu Pan, Ke Zeng, Xunliang Cai
The paper introduces CoPE, a method that trains language agents by adding multi‑step action targets to the policy backbone. By supervising each action forecasted from preceding histories, CoPE enables acting and forecasting to co‑evolve without extra inference cost. Across five benchmarks, CoPE outperforms existing RL baselines, achieving up to twice the sample efficiency on ScienceWorld and improving overall performance on realistic applications.
By Haofei Yu, Yixing Chen, Yining Zhao, Xuyan Ye, Yuxiang Liu, Zhigang Hua, Yan Xie, Shuang Yang, Jiaxuan You
arXiv:2607. 14004v1 Announce Type: new Abstract: Most reported gains from agent-optimization methods are one-shot: an agent is optimized against a fixed benchmark and the resulting improvement is reported as if it were a stable property of the method.
By Wenxiao Wang, Priyatham Kattakinda, Soheil Feizi
arXiv:2608. 07371v1 Announce Type: new Abstract: Recent agentic reinforcement learning methods use hindsight to complement sparse outcome rewards.
By Haoyu Zheng, Yun Zhu, Qing Wang, Wenqiao Zhang
arXiv:2503. 13077v2 Announce Type: replace Abstract: Multi-agent reinforcement learning has shown promise in learning cooperative behaviors in team-based environments.
By Amir Baghi, Jens Sj\"olund, Joakim Bergdahl, Linus Gissl\'en, Alessandro Sestini
arXiv:2607. 23263v1 Announce Type: new Abstract: Deciding whether a trajectory actually fulfills its instruction governs how we measure computer-use agents on long-horizon graphical-user-interface tasks and how we train them with reinforcement learning.
By Yang Wan, Zhenhao Zhang, Jierui Wang, Linchao Zhu
arXiv:2606. 27814v4 Announce Type: replace Abstract: Training small language-model agents for long-horizon interactive tasks requires both fast imitation and reward-driven improvement.
By Qitai Tan, Zefang Zong, Mo Li, Yipeng Shi, Yang Li, Peng Chen
AREX-2 is a new approach that enhances the self‑improving ability of large language model agents by combining reflection—producing better solutions—and long‑horizon execution—maintaining effectiveness over many iterations. The method synthesizes improvement trajectories from machine‑learning and algorithmic programming tasks, providing verifiable feedback and sustained iteration. Trained on this data, an agent based on Qwen3.8‑27B achieves strong performance across multiple benchmarks and continues to improve as more iterative rounds are allowed.
By Hongjin Qian, Chaofan Li, Kun Luo, Wenqing Wei, Jianlyu Chen, Shuqi Lu, Yuyang Hu, Hongwang Xiao, Hui Wang, Chaozhuo Li, Qiwei Ye, Zhicheng Dou, Defu Lian, Zheng Liu
arXiv:2608. 02391v1 Announce Type: cross Abstract: Tool-using large language model (LLM) agents produce long, multi-turn trajectories, making gradient-based post-training memory-intensive.
By Zhiyuan Wang, Shengcai Liu, Jiahao Wu, Ning Lu, Hui Ouyang, Shaofeng Zhang, Haoze Lv, Ke Tang
HINT-SD introduces a targeted self‑distillation framework for long‑horizon language‑model agents that uses full‑trajectory hindsight to identify failure‑relevant actions and applies feedback‑conditioned distillation only to those action spans. This selective approach reduces the need for per‑turn feedback, improving training efficiency and effectiveness. Experiments on BFCL v3 and AppWorld demonstrate that HINT‑SD outperforms dense per‑turn feedback baselines by up to 13.60 percentage points on average while cutting training time per step by 2.26×.
By Woongyeong Yeo, Yumin Choi, Taekyung Ki, Sung Ju Hwang