arXiv:2610.20497v1 Announce Type: new
Abstract: Reinforcement learning with verifiable rewards (RLVR) trains language models from checked task outcomes, but assigning a terminal reward to individual...
By Wei Chen, Yubing Wu, Zhou Xu, Haoyue Zhang, Delu Zeng, Jiahao Wang, Junmei Yang, Pan tiewen, Jingfeng Liu, Zhiheng Zhou, Bo Li, Min Chen
arXiv:2608. 07371v1 Announce Type: new Abstract: Recent agentic reinforcement learning methods use hindsight to complement sparse outcome rewards.
By Haoyu Zheng, Yun Zhu, Qing Wang, Wenqiao Zhang
The paper introduces CoPE, a method that trains language agents by adding multi‑step action targets to the policy backbone. By supervising each action forecasted from preceding histories, CoPE enables acting and forecasting to co‑evolve without extra inference cost. Across five benchmarks, CoPE outperforms existing RL baselines, achieving up to twice the sample efficiency on ScienceWorld and improving overall performance on realistic applications.
By Haofei Yu, Yixing Chen, Yining Zhao, Xuyan Ye, Yuxiang Liu, Zhigang Hua, Yan Xie, Shuang Yang, Jiaxuan You
arXiv:2610.15840v1 Announce Type: cross
Abstract: Reinforcement learning (RL) for large language model (LLM) agents remains challenging in long-horizon tasks with sparse outcome rewards, where assign...
By Kaijie Zhu, Weixiang Yan, Omar Attia, Abbas Kazerouni, Alexander Toshev, Andrew Szot
The paper introduces ActObs, a supervised fine‑tuning method that, unlike standard approaches, also predicts environment observations in agent trajectories. While both ActObs and action‑only training perform similarly after initial fine‑tuning, ActObs diverges during subsequent reinforcement learning, yielding higher pass@k scores on several benchmarks and better cross‑domain task performance. The authors attribute this advantage to ActObs’s joint supervision, which preserves observation gradients and prevents the policy from over‑specializing on actions alone.
By Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan, Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah
arXiv:2607. 14171v1 Announce Type: new Abstract: Reinforcement learning has emerged as the dominant paradigm for training large language model (LLM) agents that interact with executable sandboxes.
By Bowei He, Yankai Chen, Xiaokun Zhang, Xue Liu
arXiv:2607. 09042v1 Announce Type: new Abstract: Reinforcement learning (RL) is increasingly used to post-train vision-language-action (VLA) models, but every update consumes robot rollouts that are slow and costly to collect, making sample efficiency a central concern.
By Iris Xu, Sunshine Jiang, John Marangola, Nitish Dashora, Richard Li, Thomas Liu, Zexue He, Yuheng Zhi, Alex Pentland, Pulkit Agrawal, Zhang-Wei Hong
arXiv:2607. 15610v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories for advantage estimation.
By Xintong Li, Sha Li, Yuwei Zhang, Changlong Yu, Rongmei Lin, Hongye Jin, Shuyi Guan, Xin Liu, Linwei Li, Qingyu Yin, Jingbo Shang
arXiv:2610.17666v1 Announce Type: cross
Abstract: Reinforcement learning with verifiable rewards (RLVR) for multi-turn agents relies on sparse terminal rewards that say little about individual steps....
By Xuyan Ye, Xiyan Fu, Shengda Fan, Jiawei Cui, Yankai Lin, Wei Lu
AttnSR introduces a step-level reward augmentation technique for long-horizon language model agents, deriving reward allocations from policy attention during training replay without extra reward models or rollouts. By tracing backward through the policy attention graph, it estimates the relevance of earlier steps to the final outcome, using the terminal step relative to a baseline to reinforce or penalize selected steps. Experiments on Qwen2.5-7B-Instruct show AttnSR consistently outperforms PPO on ALFWorld, raising held‑out success from 91.2% to 97.1%, and it also improves ScienceWorld while reducing environment interactions by 29.9%.
By Mengsong Wu, Tianchen Zhou, Zhaochen Su, Zhenhua Liu, Zhe Cui, Aimin Zhou
arXiv:2608. 01418v1 Announce Type: cross Abstract: Autoregressive rollout generation is a major computational cost in reinforcement learning for large language models.
By Wenhao Zhang, Yibo Xie, Rui Wang, Jiahua Yang, Lei Jiang, Zibo Yang, Yawei Wang, Jiali Xu, jasperawang, Haoyang Long, Huan Xiong, alantzhao
TreeHCA introduces a method that couples hindsight-guided exploration with hindsight credit assignment for multi-turn large language model agents. By using changes in reference-answer likelihood to revisit intermediate histories and sample alternative continuations, it builds a tree of reasoning and tool interactions, and applies an inverse-hindsight importance correction to propagate terminal outcomes into turn-level advantages. Experiments on seven question-answering benchmarks with three Qwen backbones show that TreeHCA outperforms all baselines in aggregate exact match.
By Debajoy Mukherjee, Jeremy Carleton, Sushil Vemuri, Srinivas Shakkottai, Dileep Kalathil