AHEAD is a step‑aware framework that augments reinforcement learning for multi‑turn LLM agents by matching different supervision sources to different step types. The teacher receives environment feedback on all steps and LLM‑generated corrective hints only on error steps, providing finer‑grained guidance than uniform trajectory‑level rewards. Across ALFWorld, WebShop, and Search‑based QA, AHEAD improves task success by 13.3 points on ALFWorld and 11.0 on WebShop at 7B, reaches target success rates faster, and solves tasks within tighter interaction budgets compared to outcome‑only RL and prior self‑distillation baselines.
By Xiaolong Jin, Dingmin Wang, Vijay Lingam, Varun Kumar
arXiv:2606. 11559v1 Announce Type: new Abstract: Reinforcement learning typically improves multi-turn agent capabilities through the terminal outcome of the trajectories, which makes it difficult to determine credit assignments for each intermediate turns.
By Haoran Liu, Yuwei Zhang, Xiyao Li, Bohan Lyu, Jingbo Shang
arXiv:2607. 17558v1 Announce Type: new Abstract: On-policy self-distillation (OPSD) offers a promising approach for training large language models without relying on a separate teacher model.
By Fan Yang, Rui Meng, Yuxin Wen
arXiv:2608. 09555v1 Announce Type: new Abstract: External natural-language skills provide large language model (LLM) agents with reusable and editable guidance for solving complex tasks.
By Tianjun Pan, Yuan Li, Hongda Wang, Linbo Jin, Mengfei Song, Lei Gao, Qiming Shi, Shaokang Fu, Jiarong Zhao, Chengyu Wang, Chengfu Huo
arXiv:2607. 28076v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) is effective for training large language model agents.
By Binbin Zheng, Zijun Xie, Guanqun Zhao, Enlei Gong, Xing Ma, Xiaoliang Fu, Zeyu Chen
Iterative self-distillation enables LLM agents to learn from successive deployments, offering a path toward recursive self-improvement (RSI). Yet our experiments with existing methods reveal a collaps...