The paper introduces Milestone Viability Potential Policy Optimization (MVPO), a reinforcement learning algorithm designed for long‑horizon large language model agents. MVPO addresses the zero‑credit failure problem by learning from viable failure prefixes, estimating prefix potential over Union‑Find viability regions, and repairing zero‑credit groups with potential‑difference advantages. Experiments on Qwen2.5‑1.5B‑Instruct demonstrate that MVPO outperforms eight strong baselines, improving success rates on ALFWorld and WebShop with minimal overhead.
By Qi Zhou, Yuanfan Li
The paper introduces Proof‑R1, a reinforcement‑learning framework that trains large language models to generate verifiable proofs for natural‑language logical reasoning tasks. Proof‑R1 only accepts a generated conclusion into the proof state when it satisfies formal verification constraints, ensuring each reasoning step is machine‑checkable. The method also reconstructs the dependency closure that supports the final answer, aligning credit with valid proof steps, and shows improved answer accuracy and verifiability across multiple benchmarks and models.
By Qili Zhang, Qianren Mao, Hanze Cai, Kaiming Zhao, Yuening He, Xihan Lei, Yashuo Luo, Hanwen Hao, Yutong Gu, Likang Xiao, Zhijun Chen, Weifeng Jiang, Haoyi Zhou, Jianxin Li
The paper introduces Context Language Models (CLMs), which treat context as a mutable file that the model can update freely, enabling the model to learn what information to retain. CLMs built zero‑shot from existing models outperform state‑of‑the‑art context‑management methods on several benchmarks, achieving higher accuracy with fewer FLOPs. The authors also demonstrate that CLMs can be steered via natural‑language instructions and online reinforcement learning, and they propose a suffix‑cache reuse strategy that further reduces server‑side compute.
By Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh
The paper introduces MATE, a reinforcement‑learning‑based post‑training framework for unified multimodal models that lets the generation and understanding branches challenge each other instead of cooperating. In MATE, each branch proposes candidate outputs that the other must reproduce, and the solver is trained on the worst‑handled candidate, creating an evolving adversarial loop without a separate adversary. Experiments on Janus‑Pro‑1B show that MATE improves generation and understanding metrics, including GenEval (+2.4), DPG‑Bench (+1.7), and an average of nine understanding benchmarks (+0.7), while enhancing consistency across image‑text cycles.
By Wentao Zhou, Weijie Gan, Jiayun Wang
The paper introduces Verbalization Training (VT), a technique that encourages large language models (LLMs) to openly express their evaluation awareness (EA) without directly supervising their internal beliefs. VT works by truncating model rollouts just before spontaneous verbalizations, creating training prefixes that signal awareness, and then applying a reinforcement learning objective to increase calibrated verbalization. Experiments on models such as Qwen3.6-35B-A3B, Kimi K2.6, and Inkling show that VT boosts verbalized EA by 2.4–2.9× while keeping latent EA and overall behavior largely unchanged, and a causal study confirms that VT-induced verbalizations reflect newly acquired meta‑knowledge.
By Usman Anwar, Sahar Abdelnabi, David Krueger
arXiv:2609.36552v1 Announce Type: cross
Abstract: Maximum Likelihood Reinforcement Learning (MaxRL) targets prompt-wise log-success and has shown strong performance on reasoning tasks. Under finite r...
By Zihao Chen, Fanxiang Xiong, Hongran Ren, Xuefeng Bai, Zhongxiang Dai, Kehai Chen, Zhiguo Zhang, Zhiyong Wang, Yu Cheng
arXiv:2609.36588v1 Announce Type: cross
Abstract: We study reinforcement learning (RL) methods for cooperative multi-agent Vision-Language-Action (VLA) models. This problem is challenging because VLA...
By Ruixiao Xu, Wong Lik Hang Kenny, Zhiqian Liu, Jianing Guo, Hanxiao Li, Kejian Shi, Shuning Zhang, Pu Feng, Yongjia Ma, Yuqing Ma, Kai Chen, Qi Dou, Yaodong Yang, Xianglong Liu, Simin Li
arXiv:2609.36915v1 Announce Type: cross
Abstract: Aerial manipulators extend robotic manipulation into 3D workspaces that are difficult for ground-based robots to access, creating new opportunities f...
By Rui Huang, Yanlin Mu, Lidong Li, Yucong Wang, Zichen Yan, Lin Zhao
arXiv:2609.36982v1 Announce Type: cross
Abstract: Knowledge concept tagging aims to assign specific concept or topic labels to educational content, which is essential for both educators and learners...
By Zhiwei Yang, Jiahua Yang, Huiru Lin, Xing Chen, Quanlong Guan
arXiv:2609.37119v1 Announce Type: cross
Abstract: Recent approaches to reinforcement learning (RL) post-training for large language models increasingly remove the critic to reduce training instabilit...
By Hongyang Li, Xiao Li, Caesar Wu, Said Mammar, Gr\'egoire Danoy, Pascal Bouvry
arXiv:2609.37443v1 Announce Type: cross
Abstract: Long-term memory enables language models to use past interactions in future conversations. However, evidence needed to answer a question may be scatt...
By Yi-Xuan Deng, Yi Zhang, Wei Liu, Chao Xue, Shuojin Yang
arXiv:2609.37501v1 Announce Type: cross
Abstract: We propose RegLLM, a diagnostic harness for bounded autonomy in regulated agentic workflows. It instruments six trustworthiness signals: citation val...
By Dipankar Sarkar
arXiv:2609.37519v1 Announce Type: cross
Abstract: Video-based policy learning is particularly promising, as it illustrates target behaviors without requiring action annotations or embodiment-matched...
By Merve Atasever, Keyan Azbijari, Cagan Bakirci, Bo-Ruei Huang, Tolga Izdas, Zahra Shahrooei, Richard Yang, Erdem Biyik, Jyotirmoy V. Deshmukh
arXiv:2609.37626v1 Announce Type: cross
Abstract: No single way of parallelizing attention serves large language models well under all loads. Low concurrency favors tensor parallelism, many independe...
By Chuan Liu, Shuoming Zhang, Zhicheng Li, Qianqi Sun, Ruiyuan Xu, Qiuchu Yu, Xiyu Shi, Huimin Cui, Jiacheng Zhao
arXiv:2609.37633v1 Announce Type: cross
Abstract: The common paradigm of reinforcement learning with verifiable rewards (RLVR) is to let agents make multiple attempts at a task, and optimize towards...
By Michael Kirchhof, Eleonora Gualdoni, Andrew Szot, Khashayar Gatmiry, Aryo Lotfi, Abbas Kazerouni, Omar Attia, Sanjoy Chowdhury, Alexander Toshev
arXiv:2609.37800v1 Announce Type: cross
Abstract: Many recommender services repeatedly encounter cold-start cohorts, where new users arrive with little or no interaction history. This creates two cha...
By Serafima Lebedeva, Sumantrak Mukherjee, Ali Arshad Sadal, Ilias Ek\c{s}i, Rahul Sharma, Julia Mueller, Theresa Dombrowski, Jakob Karolus, Viktor Bengs, Eyke H\"ullermeier, Sebastian Vollmer
arXiv:2609.37868v1 Announce Type: cross
Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) methods such as GRPO rely on successful self-generated trajectories, but finite rollout budgets...
By Doohyuk Jang, Yoonsik Park, Gyouk Chu, Sihwan Park, Eunho Yang
arXiv:2605.14483v2 Announce Type: replace
Abstract: Large language models (LLMs) provide a flexible foundation for multi-agent systems, but their effectiveness and computational cost depend criticall...
By Xudong Chen, Yixin Liu, Hua Wei, Kaize Ding
arXiv:2405.14749v3 Announce Type: replace-cross
Abstract: Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in high-stakes applications. While traditional RL...
By Xian Yu, Minheng Xiao, Lei Ying
arXiv:2609.36308v1 Announce Type: new
Abstract: Reinforcement learning has helped AI agents solve increasingly difficult tasks, but high rewards do not always reflect the work users intended. In rece...
By Long Phan, Stephen K. Yang, Jason J. Lim, Mantas Mazeika, Wenyu Zhang, Zheyuan Liu, Richard Ren, Jingxiang Meng, Yaoteng Tan, Weiliang Zhao, Addison Wu, Matei Anghel, Dan Hendrycks