The paper introduces Rollout-Time Intervention (RTI), a method that uses reward-side evidence during rollouts to transform unsuccessful trajectories into successful ones, thereby providing learning signals for tasks beyond a policy’s intrinsic exploration limits. RTI filters out trajectories with excessive distributional shifts, allowing these off-policy interventions to guide policy optimization effectively. Experiments on five benchmarks show that RTI consistently outperforms existing RL baselines, enabling post-trained policies to approach the performance of stronger large language models and surpass their original capability boundaries.
By Chen Xi, Ding Ruixue, Wang Zhigaoyuan, Zhang Fanrui, Zhang Qiang, Chen Boli, Wang Shihang, Wang Tianfu, Ren Ailin, Wang Pengyong, Xie Pengjun, Xiong Hui
The paper introduces Behavioral Self-Rewarding (BSR), a method that lets reinforcement learning agents with verifiable rewards judge and compare their own debugging trajectories to generate additional bounded auxiliary rewards. BSR aggregates pairwise preferences among trajectories that share the same verifier outcome, adding reward contrast while maintaining the original pass/fail ordering. Experiments on SWE-bench Verified and SWE-bench-Live with Qwen3-14B and Qwen3-32B show consistent performance gains, demonstrating that a policy’s own comparative judgments can serve as a useful learning signal.
By Yiran Wu, Andrew Zhao, Yurui Chang, Qingyun Wu, Xuan Feng
DP-MIND is a memory‑guided framework that enables differentially private inference for large language models on million‑token contexts. It locally perturbs user queries and processes them in chunks, using a compact memory that retains task‑relevant information across the sequence. The memory construction is optimized via reinforcement learning on perturbed inputs, and a local denoising module recovers corrupted answer spans, resulting in improved scalability and utility over existing DP and long‑context inference baselines.
By Zhihao Liu, Yitian Cheng, Shuang Liu, Rui Wen, Yuke Hu, Zhan Qin
AdaC-GRPO introduces an adaptive curriculum learning framework for mastering long-horizon visual Sudoku puzzles, a multimodal task that challenges large language models with multi-step deductive reasoning. The authors present FG-Sudoku, a fine-grained benchmark that categorizes puzzles by grid size and required deductive steps, revealing that frontier multimodal large language models (MLLMs) struggle with complex long-range dependencies. AdaC-GRPO dynamically constructs training batches by tracking the model’s Zone of Proximal Development through online rollout accuracy, enabling Qwen2.5-VL-7B to close a significant portion of the accuracy gap to Gemini-2.5-Pro and demonstrating generalizability across other models.
By Sida Li, Yuanxin Liu, Linli Yao, Shicheng Li, Xu Sun, Haoli Bai, Xiaohui Li
The paper introduces Humanize MLE Agents (HMA), a workflow that alternates two agents across sessions on a shared workspace, limiting each session to at most k experiments and resetting context between sessions. This approach, modeled as a context‑reset semi‑Markov decision process, outperforms single‑agent baselines, achieving a 78.2% any‑medal rate on 75 Kaggle MLE‑bench tasks—8.0 points higher than the best open‑source baseline and using only a quarter of its time budget. Across six heterogeneous agent pairs, HMA improves the any‑medal rate by an average of 6.4 points, and a supplementary kernel‑optimization case reaches 878 cycles, within 1.6% of the community leaderboard leader.
By Changye Li, Zijian Zhang, Muheng Li, Yiduo Pang, Tingjun Wu, Menghan Li, Yitong Liu, Hongkai Guo, Mengchi Chen, Chenyu He, Kaisheng Ma, Wei Xu, Song Han, Sihao Liu, Ligeng Zhu
RATED (Reliability‑Aware Tool Retrieval via Execution Dynamics) is a retrieval framework that blends robust semantic matching with dynamic reliability adaptation for AI agents using external tools. It uses a frozen dual‑encoder with contrastive fine‑tuning for semantic relevance and a lightweight contextual bandit module that re‑ranks tool candidates based on real‑time execution feedback, allowing rapid adaptation to runtime failures without costly retraining. The EcoAgent benchmark, comprising 5,172 real‑world tools and over 210,000 query‑tool pairs, shows that RATED improves mean reciprocal rank by 52.2% over the ToolLLM API retriever and cuts fault selection from 86.2% to 2.9% during severe API outages, with negligible latency overhead.
By Weiliang Chen, Song Liao
The paper presents a reinforcement learning-driven neural architecture search (RL‑NAS) framework that discovers compact audio‑visual speech enhancement models. By preserving temporal alignment and using a reward function that balances SI‑SNR improvement with model size, the search evaluates only 35 candidates and finds a 2.9‑million‑parameter model that achieves a 14.5 dB SI‑SNR improvement—just 0.7 dB below the 22‑million‑parameter baseline. On commercial mobile CPUs, the new architecture reduces the real‑time factor by 4.3‑fold, enabling practical on‑device deployment while maintaining high enhancement performance.
By Khubaib Ahmed, Ahsan Adeel, Nasir Saleem, Kia Dashtipour, Ahsan Ulhaq, Amir Hussain
The paper introduces Generalized Proximal Policy Optimization (GPPO), a unified framework that extends PPO by incorporating $f$-divergence trust-region theory and novel estimator topologies for large language model reinforcement learning. GPPO demonstrates that different divergences produce distinct policy update dynamics and that estimator design significantly impacts performance. Experiments on eight math and coding benchmarks show GPPO surpasses standard GRPO by up to 50% in math and 14% in coding, achieving state‑of‑the‑art results.
By Zhicheng Cai, Xinyuan Guo, Hanlin Wu, Weinan Dai, Yuyan Zhou, Wei-Ying Ma, Ya-Qin Zhang, Hao Zhou
The paper introduces Evolving Creatives with Clicks (ECC), a method that refines e-commerce advertising images by selectively editing attributes that negatively impact click‑through rate (CTR) while preserving beneficial ones. ECC identifies 28 CTR‑correlated attributes using ANOVA and Random Forest regression, then instruction‑tunes a prompt model to generate attribute‑specific editing suggestions, further refined with real click preferences. Consistent Preference Optimization (CPO) is added to enforce consistency across semantically equivalent suggestions, reducing overfitting and improving generalization, leading to significant CTR gains over state‑of‑the‑art models.
By Xu Zhang, Haohan Wang, Zheng Zhang, Yingjie Xia, Wei Feng, Honghe Zhu, Jingjing Lv, Changping Peng, Xuheng Qin, Zehua Zhang, Zheng Luo, Qicheng Li
The paper proposes adding a social component—marginal cost—to autonomous vehicle (AV) routing rewards to account for the impact of AVs on human-driven vehicles (HDVs). Using multi-agent reinforcement learning on real-world networks, the authors show that this social reward reduces HDV travel time increases and improves overall system performance compared to purely selfish AV routing. A scalable approach limits counterfactual simulations to the most influential AVs, maintaining performance while cutting computational cost.
By Anastasia Psarou, Mikolaj Rams, Blazej Torbus, Rafal Kucharski
The paper introduces Explanatory World Models (EWM), an attribution framework that leverages explanations from a frontier language model to identify earlier events that plausibly contributed to a delayed outcome. EWM redistributes reward recursively while preserving the undiscounted episode return, and is evaluated on long‑horizon RL benchmarks, a public sequential‑recommendation task, and an industry‑scale generative recommender, consistently outperforming baseline methods, especially when rewards are highly delayed. The study demonstrates that structured explanations can provide reusable credit signals for learning from delayed outcomes.
By Oriol Corcoll, Qiyao Wei, Ciar\'{a}n Mark Gilligan-Lee
The paper introduces Loopless Variance-Reduced Policy Gradient (LVR-PG), a framework that unifies first‑order estimators combining recursive updates with defensive importance sampling, such as PAGE-PG and STORM-PG. It proves that LVR-PG achieves a ∫O(ε⁻²) last‑iterate global convergence rate under weak gradient domination for general parametric policies, using only first‑order updates and minimal assumptions. By extending lower bounds to continuous parametric policies, the authors show an Ω(ε⁻²) sample complexity limit, establishing that LVR-PG is minimax‑optimal up to logarithmic factors and horizon terms.
By Leonardo Cesani, Matteo Papini, Marcello Restelli
SafeDiSTL is a constrained multi‑agent reinforcement learning framework that decomposes joint constraints using Signal Temporal Logic robustness to assign credit to individual agents. By measuring episode‑level violations through running‑maximum cost increments and computing action sensitivities, the method distributes each cost increment among agents at the same decision time, accounting for delayed effects. Experiments on cooperative payload transport, connected navigation, and networked traffic control show that SafeDiSTL improves safe‑success rates and throughput compared to constrained baselines while preserving unconstrained policy performance.
By Hongrui Zheng, Aditya Singh, Rahul Mangharam
COBALT is a contextual bandit learning method that reuses offline multi-turn code generation trajectories to train large language models efficiently. It treats multi-turn code generation as a one-step recoverable Markov decision process, collecting partial trajectories from a reference LLM and training the target model to complete each partial trajectory in a single step. Experiments on LiveCodeBench show that COBALT improves Pass@1 scores for R1-Distill 8B and Qwen3 8B by up to 9.0 and 6.2 points, matches the performance of two online RL baselines, and enhances robustness against misleading execution feedback.
By Ziru Chen, Dongdong Chen, Ruinan Jin, Yingbin Liang, Yujia Xie, Huan Sun
The paper introduces CORE‑Seg, an end‑to‑end framework that combines reasoning and segmentation for complex lesion detection in medical images. It leverages a Semantic‑Guided Prompt Adapter and a progressive training strategy from SFT to GRPO, using an adaptive dual‑granularity reward mechanism to address reward sparsity. CORE‑Seg achieves state‑of‑the‑art performance, with a mean Dice score of 37.06%—14.89% higher than the next best baseline—and reduces the failure rate to 18.42%.
By Yuxin Xie, Yuming Chen, Yishan Yang, Yi Zhou, Tao Zhou, Zhen Zhao, Jiacheng Liu, Huazhu Fu
Sampling-Based Safe Reinforcement Learning (SBSRL) is a model‑based RL algorithm that enforces safety by jointly constraining a finite set of dynamics samples, approximating a worst‑case optimization over uncertain dynamics. It introduces an exploration strategy that limits epistemic uncertainty, removing the need for explicit exploration bonuses. The authors provide high‑probability safety guarantees and a finite‑time sample complexity bound, and demonstrate safe, efficient exploration in both simulation and real robotic hardware, with extensions to deep‑ensemble implementations for high‑dimensional continuous control.
By Luca Vignola, Bruce D. Lee, Manish Prajapat, Manuel Wendl, Melanie Zeilinger, Andreas Krause, Yarden As
The paper introduces fixed‑point flow matching (FFM), a data‑free method for training and fine‑tuning flow policies in reinforcement learning. FFM formulates the actor update as a fixed‑point iteration using a flow‑matching regression objective, where the critic’s action gradients act as targets. Experiments show that FFM can effectively fine‑tune pretrained flow policies in offline‑to‑online scenarios and train flow policies from scratch in online RL, establishing it as a principled approach for learning expressive flow‑based policies without training data.
By Denis Blessing, Onur Celik, Gerhard Neumann
The paper introduces Self‑Sparse Speculative Rollouts (S$^3$R), a method that uses sparse attention to propose tokens and then verifies them with full attention, thereby reducing KV‑cache traffic while preserving the dense‑policy distribution for reinforcement learning rollouts. S$^3$R eliminates the need for a separate drafter by sharing policy parameters between drafting and verification, and a KV‑aware scheduler limits cache thrashing to maintain parallelism. Experiments on long‑context mathematical reasoning with the Qwen3 model show that S$^3$R achieves 1.35–1.39× higher end‑to‑end training throughput with comparable task performance.
By Yuhang He, Hongyu Ge, Zihan Lin, Siyi Liu, Yongqi Zhang
MORAD is an online reinforcement learning framework that trains a single shared policy for backbone‑conditioned RNA diffusion models, enabling structure‑conditioned RNA inverse design across diverse target backbones. It integrates backbone‑conditioned group sampling, group‑relative reward normalization, and a forward‑process diffusion objective, jointly optimizing six measurements—including tertiary geometry, base pairing, ensemble behavior, and sequence composition—by mapping each to a bounded desirability score and combining them with a weighted geometric mean. Compared to a tertiary‑structure‑only reward, MORAD improves 10 of 11 evaluation metrics, achieving significant gains in GDT‑TS, pairing MCC, and normalized ensemble defect, and surpasses existing methods such as RiboDiffusion, gRNAde, and RDesign on tertiary structural accuracy, secondary‑structure pairing, and thermodynamic quality.
By Jixin Tang, Ji Guo, Jun Zhao
The paper introduces Persistent Candidate LinUCB (PC‑LinUCB), a variant of the LinUCB algorithm for linear bandits that reduces per‑round score evaluations from ≥×K to ≥×(k+q) by maintaining a small set of active actions and sampling challengers from the remaining actions. It provides high‑probability regret bounds that include a standard learning term and an additive discovery cost of order ≥×((K-k)×log(K-k))/q, which does not grow with the horizon. Experiments demonstrate that PC‑LinUCB accelerates LinUCB while keeping regret comparable, and the persistent candidate approach can be applied beyond LinUCB.
By Bingkui Tong, Daria Korovaitceva, Marina Sheshukova, Sergey Samsonov, Junpei Komiyama