The paper introduces GAC, a noise‑aware adaptive mixing controller that dynamically balances supervised fine‑tuning (SFT) and reinforcement learning (RL) gradients during hybrid post‑training of large language models. By deriving a closed‑form mixing weight that minimizes an MSE upper bound on the mixed gradient, GAC incorporates EMA smoothing, a schedule prior, and capped updates, all estimated online from training tensors. Experiments show that GAC outperforms rule‑based controllers, improving performance on diverse benchmarks by up to 3.8 percentage points, reducing KL‑drift by 28%, and cutting large mixing‑weight changes by over 70% with negligible overhead, with gains scaling from 1.5 B to 14 B parameters.
By Yuelin Hu, Wei Liu, Zhenbo Yu, Zhengxue Cheng, Li Song
The paper introduces Single‑Rollout Pass@k Policy Optimization (SR‑PPO), a reinforcement learning method that uses only one rollout per prompt to train language‑model agents. SR‑PPO combines Monte Carlo token‑level advantage estimation with Pass@k gradient reweighting and an online critic for baseline and weight estimation. Experiments show that SR‑PPO achieves stable learning and performance comparable to traditional Group Relative Policy Optimization (GRPO) on reasoning and logic benchmarks, even when off‑policy drift degrades standard Pass@1 PPO.
By Fengdi Che, Yang Liu, Lei Yu, Meng Cao, Tong Che, Rupam Mahmood, Dale Schuurmans
The paper introduces MusicConstraintBench, a benchmark of 2180 items testing symbolic music generation under multiple explicit constraints. It shows that current language models, such as Llama-3.1-70B, perform poorly on multi-constraint tasks, achieving only 4.4% success on four-constraint items. The authors propose MusicRLVR, a reinforcement learning approach that uses verifier rewards without human annotation, which significantly improves performance—raising Qwen3-4B-Instruct-2507’s success from 16% to 80.7% on mixed constraints after 300 updates.
By Haoyue Liu, Xiaoying Tang
The paper introduces a self‑distillation method that leverages answer‑only data to improve large language models (LLMs) after training. By training an inverse reasoning LLM to generate latent chain‑of‑thought traces conditioned on the answer, the authors can perform offline distillation that significantly boosts performance on hard prompts for tasks like Countdown and GSM8K. Additionally, the inverse model serves as an accurate verifier, enhancing best‑of‑n selection accuracy.
By Eric O. Korman
PMO-Dock introduces a standardized benchmark for docking‑based molecular optimization, comprising 25 tasks that span hit generation, lead optimization, and a specificity task that rewards strong on‑target binding while penalizing off‑target interactions. The protocol separates model development from final evaluation, using validation tasks for hyperparameter tuning and held‑out test tasks evaluated under a strict oracle budget to mimic realistic drug‑discovery constraints. Four diverse methods—Saturn, GenMol, Genetic‑guided GFlowNet, and ChemLlama—were benchmarked, revealing no single method dominates across all tasks and highlighting limited hyperparameter transferability, which underscores the need for method‑dependent selection strategies.
By Gor Simonyan, Tatevik Abrahamyan, Narek Abelyan, Tigran Fahradyan, Hrant Khachatrian
The paper introduces Lag, a mean‑field optimal index policy for online assignment problems with reusable resources, such as peer review, call routing, ridesharing, emergency dispatch, and crowdsourcing. By modeling the problem as a contextual restless multi‑armed bandit and deriving an occupancy‑measure linear program, the authors obtain Lag from the dual and prove it is asymptotically optimal in a scaling regime where server copies and arrivals grow together. Experiments on peer‑review and call‑center data show Lag achieves 83–99% of the LP upper bound and outperforms greedy assignment by up to 36% in synthetic settings.
By Keerthana Gurushankar
StaRPO is a stability‑augmented reinforcement learning framework that enhances large language models’ reasoning by incorporating two lightweight stability metrics—Autocorrelation Function (ACF) for local step‑to‑step coherence and Path Efficiency (PE) for global goal‑directedness—into the optimization objective. By combining these stability rewards with task rewards, StaRPO provides process‑aware feedback that reduces logical inconsistencies, structural erraticism, and redundancy in generated responses. Experiments on four reasoning benchmarks demonstrate that StaRPO consistently outperforms baseline methods, improving both final‑answer accuracy and logical stability.
By Jinghan Zhang, Fengran Mo, Tharindu Cyril Weerasooriya, Ruimin Dai, Xiaoyan Han, Yanjie Fu, Dakuo Wang, Kunpeng Liu
arXiv:2610. 19080v1 Announce Type: new Abstract: Reinforcement learning (RL) has substantially improved the reasoning of autoregressive and discrete-diffusion language models.
By Xu Han, Jiajing Hu, Li-Ping Liu
The paper challenges the common practice of selecting latent world models based on one‑step prediction loss, arguing that this metric poorly predicts planning performance. Using a frozen‑decoder protocol, the authors decompose one‑step error into encoder‑inherent and predictor‑added components and evaluate goal‑conditioned planning across three continuous‑control environments. Their experiments show that improvements in one‑step fidelity do not correlate with planning success, with correlation coefficients near zero and inconsistent exchange rates across tasks.
By Joshua Cheung, Zhang He
The paper proposes a method called GracefulRL to train language models to fail gracefully. By reshaping the reinforcement learning optimization landscape, the models learn to preserve successful behavior while executing a safe fallback policy when actions are unreliable, all from outcome-level reward signals without extra annotations or inference-time checks. Experiments across various tasks and scales show that models maintain task-solving capability while selectively falling back when success is unlikely.
By Daman Arora, Xuecheng Liu, Himanshu Gaurav Singh, Fahim Tajwar, Abitha Thankaraj, Ruslan Salakhutdinov, Haiwen Feng, Yuda Song, Andrea Zanette
The paper investigates the role of first‑moment momentum in reinforcement learning with verifiable rewards (RLVR). Using a bandit formulation with bias‑corrected exponential moving average (EMA) momentum, the authors prove that, when policy gradients are exact, increasing the EMA coefficient β within a safe range accelerates optimization. In practice, they show that maintaining a larger β preserves a non‑vanishing expected gain even as the learning rate shrinks, and experiments on Qwen models confirm that higher β values improve validation accuracy by up to about 2 percentage points.
By Kaifei Wang, Zikang Shan, Liwei Wang, Yinyu Ye, Han Zhong
Selective Growing Q‑Networks (SGQN) extend value‑based Q‑learning for continuous control by replacing uniform action‑grid growth with an adaptive grid that adds bins near high‑Q regions and prunes low‑Q bins, while maintaining decoupled Q‑learning and epsilon‑greedy action selection. On eight DeepMind Control Suite tasks, SGQN achieves the highest mean final return on four tasks, matches the best method within 1.1% on three more, and uses on average 7.5 of 9 available bins per dimension. Experiments show that the adaptive grid’s growth and pruning have limited impact on performance, whereas starting from a coarse grid and using epsilon‑greedy selection are critical for success.
By Parham Nooranbakht, Seyed Mahdi Basiri Azad, Joschka B\"odecker
arXiv:2610.19108v1 Announce Type: cross
Abstract: Multimodal Large Language Models (MLLMs) have been widely applied in embodied agents and robotics for vision tasks. However, MLLMs suffer from limita...
By Mahdieh Mallahnezhad, Masoud Jafaripour, Mohamadali Tofigh, Sheila Schoepp, Osmar R. Zaiane
arXiv:2603.19714v2 Announce Type: replace
Abstract: Looped language models (LoopLMs) perform iterative latent computation to refine internal representations, offering an alternative to explicit chain...
By Guo Tang, Shixin Jiang, Heng Chang, Zihan Zhang, Nuo Chen, Yuhan Li, Huiming Fan, Jia Li, Ming Liu, Bing Qin
arXiv:2606.17029v2 Announce Type: replace
Abstract: Deep research agents synthesize long-form reports by searching and reasoning over retrieved evidence. Reinforcement learning with rubric-based rewa...
By Minghang Zhu, Chuyang Wei, Junhao Xu, Yilin Cheng, Zhumin Chen, Jiyan He
arXiv:2604.13993v2 Announce Type: replace-cross
Abstract: Physical reasoning over images requires integrating perception, domain knowledge, and multi-step symbolic inference, yet Vision-Language Mode...
By Derek Lilienthal, Manisha Mukherjee, Sameera Horawalavithana
arXiv:2610.20584v1 Announce Type: new
Abstract: Computer-use agents have demonstrated strong capabilities in interpreting screens and executing keyboard and mouse actions. However, games, with their...
By Mingyu Ouyang, Yiran Wang, Siyuan Hu, Xingyilang Yin, Huiqiang Sun, Guangzhi Wang, Junfu Pu, Cheng Cheng, Honghao Chen, Ying Shan, Mike Zheng Shou, Hwee Tou Ng
arXiv:2601.08010v4 Announce Type: replace
Abstract: Vision-language models achieve strong performance across a wide range of multimodal understanding and reasoning tasks, yet their multi-step reasoni...
By Chaoyu Li, Fei Tao, Pooyan Fazli
SAM3RL introduces a reinforcement‑learning based memory controller for the Segment Anything Model 3 (SAM 3) visual object tracker, deciding which past frames to store in a memory bank to improve tracking performance. This data‑driven approach replaces hand‑crafted memory update rules and achieves state‑of‑the‑art results on 10 of 12 tracking and video object segmentation benchmarks, with no runtime overhead. The same method applied to SAM 2 (SAM2RL) also outperforms hand‑crafted rules on 9 of 12 benchmarks, demonstrating its generality across model generations.
By Tom\'a\v{s} \v{C}\'i\v{z}ek, Matej Straka, Illia Volkov, Mat\v{e}j Kripner, Ervin Maci\'c, Jiri Matas, Klara Janouskova, Martin Schmid
VASPO introduces a new integration of self-distillation and PPO by leveraging privileged value estimation to guide language model training. The method uses a self-teacher and a critic that both evaluate the same privileged information on student-generated prefixes, allowing the critic to compute token-level PPO advantages from predicted continuation returns while the teacher modulates positive advantages via bounded probability ratios. Experiments on six mathematical reasoning benchmarks show that VASPO improves mean accuracy over standard PPO by 2.59 and 1.81 percentage points on Qwen3-4B and OLMo3-7B, respectively, and achieves the highest mean accuracy among evaluated methods on Knights and Knaves.
By Fei Wu, Zhenrong Zhang, Jianshu Zhang, Jun Du