The paper introduces SCOUT, an offline multi-agent reinforcement learning framework that combines a generative behavioral prior with a decomposed value function for test-time action refinement. SCOUT uses optimal unified transport and Stein variational gradient descent to steer behavioral samples toward high-value regions, with the number of transport steps providing adaptive scaling. The authors prove a single-term KL bound on the joint soft-value gap under the individual-global-max principle and demonstrate that SCOUT outperforms existing methods on both discrete and continuous offline MARL benchmarks, including offline-to-online settings.
By Dongsu Lee, Haoran Xu, Amy Zhang
The paper introduces Follow the Winners (FTW), a critic‑free policy‑learning algorithm that adapts the cross‑entropy method for reinforcement fine‑tuning of agentic large language models. FTW replaces group rollouts with an ordinal filter on replay‑buffer samples, achieving polynomial concentration in the order statistic of returns and offering a bounded risk‑seeking offset that balances variance reduction. Experiments on Sokoban and Search‑R1 show that FTW matches the performance of GRPO and PPO while reducing reliance on value models or repeated rollouts.
By Joery Ari\"en de Vries, Neil David Lawrence, Zhenwen Dai
The paper introduces checkpoint handoff, an evaluation protocol that separates an agent’s ability to reach useful states from its ability to complete tasks in reinforcement learning. By using one checkpoint as a reacher up to a handoff point and another as a solver from the same replayed history, the authors can measure Reach (how often states within a fixed number of actions from success are achieved) and Solve (how often the task is completed from those states). Experiments on TravelPlanner and ALFWorld show that switching the solver from supervised fine‑tuning to RL yields larger gains when RL is used as the reacher, indicating that RL more effectively finds solvable states.
By Xuan Liu, Jingbin Qian
The paper introduces Residual Quantization (RQ) as a representation layer for contextual bandits, converting continuous contexts into discrete centroid assignments across multiple levels. This approach allows additive bandit algorithms to capture nonlinear reward structures while maintaining strictly bounded memory and efficient online updates. Experiments on 13 datasets show that RQ variants outperform non-RQ counterparts on 11 datasets and match advanced baselines like XGBoost and neural methods using up to 1000 times less memory.
By Ami Tavory, Noam Touitou, Tal Sarig, Frank Cheng, Ido Guy
RASPER is a reward‑aligned summarizer that tailors the extraction of information from unstructured discharge notes to improve downstream clinical predictions. It uses a tunable LLM summarizer trained with reinforcement learning, where the reward comes from the loss of a downstream predictor, and incorporates patient‑specific context via a longitudinal encoder that soft‑prompts the summarizer with structured codes. The approach consistently outperforms strong baselines on readmission prediction and medication recommendation tasks in the MIMIC‑III and MIMIC‑IV datasets.
By Arya Hadizadeh Moghaddam, Mohsen Nayebi Kerdabadi, Chen Chen, Dongjie Wang, Zijun Yao
The paper introduces MIRA, a hierarchical architecture that separates meta-reasoning from execution for long-horizon research agents. An outer-loop meta-reasoner selects the next investigation based on a persistent research record, while an inner-loop executor carries out the chosen work order. The authors demonstrate that MIRA improves inference and compute allocation in theorem proving and neural-architecture research, and that a generative actor-critic variant (MIRA-AC) can be trained efficiently without directly optimizing long execution traces.
By Ankur Samanta, Yonathan Efroni, Paul Sajda, Kaveh Hassani, Anirudh Goyal
The paper introduces a two‑stage training framework for rubric‑based reinforcement learning (RL) that first uses rubric‑privileged on‑policy distillation (RP‑OPD) to provide dense token‑level supervision, then applies RL to optimize the rubric reward. In RP‑OPD, a student model learns to match a rubric‑aware teacher’s next‑token distributions on student‑generated prefixes. Experiments on HealthBench, ResearchQA, and RubricHub Science show that this two‑stage approach yields the highest scores and reduces reward hacking compared to a standard SFT + RL baseline.
By Xinpeng Wang, Wei Shi, Yu-Chia Chen, Maria Zontak, Yun He, Richard Yuanzhe Pang
The paper proposes a framework for agent calibration that ensures consistent capabilities when agents are deployed, migrated, or scaled across different models, harnesses, infrastructures, and user contexts. It defines standards for basic capabilities, technical environments, and user contexts, diagnoses gaps, applies revisions, and rechecks within fixed budgets, using diagnostic source behavior and mandatory tests to qualify agents. The framework includes semantic checkpoints, repair localization, and reusable configuration artifacts, and outlines an automatic calibration tool that relies on limited authorized user trajectories and tests, though empirical validation remains pending.
By Yaxiao Liu (PwC China AI Center), Pengbo Liu (PwC China AI Center), Yiwen Liu (PwC China AI Center), Yihua Guan (PwC China AI Center), Jiaxing Song (Tsinghua University)
The study investigates whether mixture‑of‑experts (MoE) architectures improve deep reinforcement‑learning policies for cryptocurrency order execution. Using 5‑minute BTC/USDT limit‑order‑book data, the authors compare vanilla Double Deep Q‑Learning (DDQL) with K‑means‑partitioned MoE experts (K=2,4,8) and dense networks matched in size to the expert budgets. Across 100 training seeds, none of the MoE variants reduce mean implementation shortfall relative to DDQL, and all perform worse than simple benchmarks (TWAP and immediate liquidation). The authors find that training‑specification issues—particularly exploration schedules—drive most failures, while MoE partitioning offers no intrinsic benefit and can mask underlying problems. They also report that higher K increases tail risk and that performance variability depends heavily on seed selection.
By Alexander Ardaiz, Varun Budati, Ali Habibnia
Power‑SMC is a training‑free sampling method that targets the sequence‑level power distribution used to sharpen large language model outputs for reasoning tasks. It maintains multiple candidate sequences, assigns importance weights based on how well each matches the power distribution, and periodically prunes low‑scoring candidates to keep inference latency close to standard decoding. Experiments on MATH500, GSM8K, GPQA, and HumanEval show that Power‑SMC matches or surpasses Metropolis–Hastings sampling in accuracy, preserves output diversity, and can accelerate inference speed by up to 17.6×.
By Seyedarmin Azizi, Erfan Baghaei Potraghloo, Minoo Ahmadi, Souvik Kundu, Massoud Pedram
The paper introduces GAW-PO, a gradient‑aligned token reweighting technique for Direct Preference Optimization (DPO). It assigns weaker penalties to rejected tokens whose gradients align with preferred behavior, while maintaining stronger penalties for conflicting tokens. Across 11 benchmarks in mathematics, reasoning, coding, and question answering, GAW‑PO outperforms standard DPO and other baselines, and remains robust when the DPO regularization parameter is reduced.
By Andreea Dutulescu, Stefan Ruseti, Mihai Masala, Traian Rebedea, Mihai Dascalu
The paper introduces DyCPO, a co‑evolutionary framework that jointly optimizes token selection and adaptive counterfactual intervention for video reasoning. It builds a multi‑role dependence metric to balance visual exploration with answer‑relevance mining, suppressing filler tokens and spurious visual noise. By deriving counterfactual signals from the model’s own successful and failed rollouts, DyCPO enables self‑diagnostic analysis and co‑evolution of the optimization objective with the policy, leading to consistent performance gains on complex video reasoning benchmarks.
By Yudong Han, Yong Wang, Zaiquan Yang, Liang Lin, Chongyang Tao, Xiangxiang Chu, Liyuan Pan
The article proposes a Quantum Reinforcement Learning (QRL) algorithm to reduce energy consumption in 5G and beyond networks by optimizing base station (BS) operations while maintaining quality of service. QRL uses quantum principles such as superposition and entanglement via parameterized quantum circuits, achieving faster convergence than traditional Deep Reinforcement Learning (DRL) and Q‑Learning. Simulations show that QRL can significantly lower energy usage even with highly dynamic user equipment (UE) associations.
By Muhammad Usman, Nguyen Van Huynh, Marianna Lezzi, Mariangela Lazoi
The paper introduces Adaptive Conformal Inference, an online calibration method that replaces static quantile estimation in hardware‑aware neural architecture search (NAS). By providing distribution‑free guarantees that at most a fraction δ of candidates are wrongly discarded, the method maintains steerable coverage across arbitrary sequences of reinforcement‑learning proposals. Experiments on three neural‑network families show that it tracks requested coverage levels within ~10⁻³ while pruning 25–50% of evaluations without accuracy loss, outperforming static calibration and a Gaussian‑process baseline.
By Pedro Brandimarte, Nerea Aranjuelo, Marcos Nieto, Oihana Otaegui
arXiv:2610.03165v1 Announce Type: new
Abstract: Several variance-reduced versions of REINFORCE based on importance sampling achieve an improved $O(\epsilon^{-3})$ sample complexity to find an $\epsil...
By Gabor Paczolay, Matteo Papini, Alberto Maria Metelli, Istvan Harmati, Marcello Restelli
arXiv:2610.03641v1 Announce Type: new
Abstract: Masked discrete diffusion models offer a promising alternative to autoregressive generation, but iterative sampling can be costly, and intractable sequ...
By Vladislav Gromadskii, David Li, Samson Gourevitch, Yazid Janati, Eric Moulines, Maxim Panov, Alexander Korotin
arXiv:2610.03642v1 Announce Type: new
Abstract: Success conditioning is a strategy for improving decision-making policies in stochastic environments; it updates a policy by increasing the probability...
By Matthew Brun, Xu Andy Sun
arXiv:2610.02359v1 Announce Type: cross
Abstract: Reinforcement learning from verifiable rewards (RLVR) usually optimizes answer correctness, yet useful language-model behavior also requires high-qua...
By Doseok Jang, Jon Ander Campos, Youran Qi
arXiv:2610.03702v1 Announce Type: new
Abstract: The choice of post-training data for large language models substantially affects downstream performance. Gradient-based data selection is a popular app...
By Lyuxin David Zhang, Eric Wong, Surbhi Goel, Anton Xue
arXiv:2610.02730v1 Announce Type: new
Abstract: This paper proposes a new functional approximation approach to reduce Bellman error in high-dimensional dynamic programming and Reinforcement Learning...
By Haining Yu