PlanPO introduces a group planning-aware policy optimization method for multi-turn agentic large language models, addressing the issue of advantage collapse caused by treating all successful trajectories equally. By incorporating coarse-to-fine advantage signals that reflect differences in trajectory and turn lengths, PlanPO encourages agents to learn generalizable planning and generation behaviors. Experiments show a 27.2% average improvement over GRPO on benchmarks such as ALFWorld, WebShop, and SciWorld, with minimal extra training cost.
By Dayang Liang, Liyuan He, Xuan Feng, Shuxin Li, Bo An, Yunlong Liu
arXiv:2609.17088v1 Announce Type: new
Abstract: Recent advancements in large language models have significantly enhanced the capabilities of agents in modeling long-term conversations. Despite these...
By Cai Ke, Jiangyue Yan, Han Zhang, Xin Liu, Zike Yuan, Yue Yu, Hui Wang, Ruifeng Xu
arXiv:2602. 11351v2 Announce Type: replace Abstract: Proactive large language model (LLM) agents aim to actively plan, query, and interact over multiple turns, enabling efficient task completion beyond passive instruction following and making them essential for real-world, user-centric applications.
By Yihang Yao, Zhepeng Cen, Haohong Lin, Shiqi Liu, Zuxin Liu, Jiacheng Zhu, Zhang-Wei Hong, Laixi Shi, Ding Zhao
arXiv:2607. 20734v1 Announce Type: new Abstract: As LLMs become more capable, they are increasingly deployed as collaborative agents, taking on user-delegated tasks through iterative interaction.
By Jihoon Tack, Philippe Laban, Jennifer Neville
arXiv:2607. 27973v1 Announce Type: new Abstract: Recently, Reinforcement Learning (RL) has emerged as a crucial paradigm for the post-training of Large Language Model (LLM) agents.
By Cong Li, Peixi Peng, Yisen Zhao, Xinyu Hu, Shudong Liu, Zhan Su, Zhuojian Li
arXiv:2607.26178v2 Announce Type: replace
Abstract: Turn-taking is a central component of full-duplex interaction. Which turn-taking behaviors are appropriate varies with the scenario, yet current mo...
By Takyoung Kim, Kang-wook Kim, Sang Hoon Woo, Julia Hirschberg, Gunhee Kim, Dilek Hakkani-T\"ur
arXiv:2607. 04713v1 Announce Type: cross Abstract: Reinforcement learning holds significant potential for training large language models (LLMs) to handle multi-turn interactive tasks.
By Qiang Liu, Taian Guo, Ruizhi Qiao, Xing Sun
The paper introduces LM2LM, a method that trains large language models (LLMs) to propose improvements for other LLMs using reinforcement learning (RL). It employs a frozen research world model (RWM) to predict the gains of proposals, providing a cheaper surrogate reward, and periodically validates high‑scoring proposals to keep feedback reliable. Experiments across eight LM development environments show that LM2LM enables a Qwen3.8‑27B model to outperform GPT‑6‑sol on held‑out tasks while reducing data‑generation cost by 33%, and it generalizes to unseen model families.
By Minhua Lin, Zijun Wang, Zewen Liu, Zhaotian Weng, Bing He, Zhan Shi, Benoit Dumoulin, Xiang Zhang, Suhang Wang, Hanqing Lu
arXiv:2503. 00539v2 Announce Type: replace-cross Abstract: Reinforcement learning from human feedback (RLHF) has evolved to be one of the main methods for fine-tuning large language models (LLMs).
By Debmalya Mandal, Paulius Sasnauskas, Goran Radanovic
The paper introduces Reverse‑Turn Policy Optimization (RTPO), a method that restructures multi‑turn agentic reinforcement learning rollouts into sparse reverse trees and updates policies in temporal reverse order. This approach addresses three key instability sources—context mismatch, weak turn‑level credit assignment, and asynchronous policy drift—by aligning each decision with its downstream continuation. Theoretical analysis shows RTPO eliminates context mismatch and drift, reduces credit bias, and converges to recursive optimality, while experiments demonstrate performance gains of 21.50% over trajectory‑level and 10.76% over turn‑level baselines on multi‑turn agentic RL benchmarks.
By Yugu Li, Jimmy Cao, Jianglin Qiao, Siyi Hu
arXiv:2606. 00135v1 Announce Type: cross Abstract: Tool-calling is a central component of modern large language model (LLM) agents, equipping them with skills beyond their parametric knowledge.
By Tong Liu, Cheng Qian, Matej Cief, Yuan He, Daniele Dan, Nikolaos Aletras, Gabriella Kazai
arXiv:2606. 27136v1 Announce Type: new Abstract: For LLM agents in multi-step interactive environments, a key challenge is to make effective use of accumulated interaction experience.
By Shicheng Ye, Chao Yu