The paper introduces Partially Informed Stochastic Optimization (PISO), a method that blends an available gradient component with a running estimate of the missing residual. By centering zeroth‑order observations on this estimate, PISO reduces variance to depend on tracking error, yielding a contraction‑based iteration with a clear momentum interpretation. For smooth nonconvex objectives, the authors prove convergence rates that rely only on the residual’s variation and initial magnitude, achieving state‑of‑the‑art function‑evaluation complexities with minimal probe directions and samples, and demonstrating empirical improvements across five benchmark settings.
By El Mahdi Chayti, Alireza Mirrokni, Martin Jaggi
The paper tackles the problem of designing graph topologies that maximize algebraic connectivity while keeping the number of vertices and edges fixed. It introduces a two‑stage, density‑dependent approach: first constructing a low‑cost structural backbone—using complete multipartite graphs in dense settings and Cayley graphs in low‑to‑intermediate settings—then refining the remaining edges with a reinforcement learning completion stage. Numerical experiments demonstrate that this method achieves high algebraic connectivity with good runtime scalability, outperforming existing techniques and supporting a structural‑family‑first strategy for scalable network design.
By Zeru Zhu, Marco Gamarra, Ji Liu
arXiv:2610. 13901v1 Announce Type: cross Abstract: We study the last iterate of standard asynchronous Q-learning from a single trajectory of a finite Markov decision process.
By Yang Peng
The paper introduces user simulators that provide intrinsic feedback by incorporating latent evaluation states such as internal thoughts, reactions, and scalar ratings. These simulators, trained on real user–assistant interaction logs, outperform prompted baselines in realism and critiquing behaviors, and their turn‑level evaluations better align with human judgments. Additionally, the intrinsic feedback serves as an effective reward signal for multi‑turn reinforcement learning, reducing compute requirements by over 50% compared to setups using external judge LLMs.
By Tarek Naous, Craig Boutilier, Wei Xu, Ofer Meshi
The paper examines how agents in automated driving respond to perceptual uncertainty and how providing explicit uncertainty information affects their behavior. Experiments on a proxy driving task show that unreliable perception leads to defensive driving, while incorporating uncertainty data enables the agent to adapt, complete the task faster, and better manage risks.
By Natalie Grabowsky, Annika M\"utze, Joshua Wendland, Nils Jansen, Matthias Rottmann
MotionRFT introduces a reinforcement fine‑tuning framework for text‑to‑motion generation that unifies heterogeneous motion representations into a shared semantic space using the MotionReward model. The framework includes a fine‑grained, memory‑efficient fine‑tuning method called EasyTune, which optimizes step‑wise denoising steps to reduce computational overhead. Experiments show significant improvements in FID, R‑Precision, and memory usage across several motion models.
By Xiaofeng Tan, Wanjiang Weng, Hongsong Wang, Fang Zhao, Xin Geng, Liang Wang
VaME (Variational Multimodal Embeddings) introduces a framework that treats latent reasoning as a learnable distribution over trajectories, enabling autoregressive exploration in latent space guided by answer reconstruction. By augmenting embedding-token readout with a latent-fused embedding and optimizing over stochastic variational trajectories via reinforcement learning with a Semantic Decoding Reward, VaME achieves superior performance on the MMEB-V2 benchmark and reasoning-intensive tasks like MRMR. The method also delivers at least a 4.25× inference speedup compared to deterministic latent autoregressive baselines.
By Peixi Wu, Mingzhou Jiang, Feipeng Ma, Biao Yang, Yunhao Zhou, Wei Yuan, Bosong Chai, Huizu Lin, Jie Chen, Zhangchi Hu, Fan Yang, Wenwu Ou, Hebei Li, Xiaoyan Sun
The paper introduces a preference-conditioned multi-objective reinforcement learning framework for tactical decision making in heavy-duty vehicles. It explicitly models safety, energy efficiency, and time efficiency as separate objectives and uses Generalized Policy Improvement and Proximal Policy Optimization to learn policies that approximate the Pareto front across these conflicting goals. Experiments in a scalable highway traffic simulation show smooth, interpretable Pareto fronts and meaningful trade-offs, with the policy enabling behavior transitions without retraining.
By Deepthi Pathare, Leo Laine, Morteza Haghir Chehreghani
The paper introduces Graph Direct Preference Optimization (GraphDPO), a generalization of Direct Preference Optimization that uses directed acyclic graphs to capture richer preference structures from multiple rollouts per prompt. GraphDPO encodes dominance relations as edges and optimizes a Plackett–Luce-inspired objective that aggregates supervision over dominated neighborhoods, preserving transitivity and handling ties or incomparable responses via an equivalence-class construction. Experiments on reasoning and program synthesis tasks show that this graph-structured approach outperforms traditional pairwise and listwise alignment objectives while maintaining linear per‑prompt complexity.
By Ning Liu, Chuanneng Sun, Kristina Klinkner, Shervin Malmasi
CB-Orchestrator is an adaptive framework for optimizing large language model (LLM) agent workflows. It separates workflow generation—via evolutionary search—from workflow selection, which is treated as a contextual bandit problem to enable sample‑efficient learning. The approach reduces training‑phase API token usage by up to 40.55% and overall token overhead by at least 29.73%, while outperforming baselines on five benchmarks, including the MATH dataset.
By Jiahang Sun, Zhiwei Shang, Zhipiao Liu, Hongwei Yang, Guoqing Xie, Shuang Qiu, Zhongxiang Dai
arXiv:2610. 13205v1 Announce Type: new Abstract: On-policy distillation (OPD) trains a target policy on its own rollouts using token-level supervision from a stronger teacher, but typically relies on the teacher being substantially more capable than the target.
By Rong Wu, Daocheng Fu, Yu Yang, Xuemeng Yang, Siqi Li, Shu Zou, Yao Zhu, Botian Shi, Bo Zhang, Cong Zhang, Pinlong Cai, Jianbiao Mei
Dream to Remember (D2R) is a continual learning framework that steers a frozen generalist robot policy toward new tasks while preserving previously acquired skills. It uses a pretrained world model to rehearse past tasks through imagined rollouts, eliminating the need for replay buffers or new data. Experiments on CALVIN, LIBERO, and a real Franka robot show that D2R can learn multiple tasks sequentially—often entirely in imagination—without catastrophic forgetting, outperforming traditional fine‑tuning methods.
By Bahey Tharwat, Iman Nematollahi, Marvin Alles, Xingyuan Zhang, Akshay L Chandra, Abhinav Valada
The paper introduces a new survival analysis algorithm that leverages reinforcement learning concepts, specifically a delayed target network inspired by Deep Q-Networks, to enforce temporal consistency across consecutive survival predictions. This framework supports complex architectures and mini‑batch, end‑to‑end training, and demonstrates improved performance on long‑sequence datasets compared to benchmarks. Ablation studies reveal that the delay in target updates significantly affects predictive accuracy and training stability.
By Mariana Vargas Vieyra, Pascal Frossard
The study evaluates multi-harness reinforcement learning (RL) for coding agents by training two Qwen3-8B RL variants—one with rewards normalized within each harness and one across harnesses—using identical replay data. Results show modest gains over the supervised fine-tuning baseline (+0.48 and +0.77 percentage points for within and cross normalization, respectively), but no consistent advantage of cross normalization on unseen harnesses. Performance differences across harnesses are much larger than the training gains, indicating that harness dependence remains a significant challenge.
By Chenqian Le, Jiayi Cheng, Qijia He, Runhao Li, Yinghao Li, Xupeng Chen
The paper investigates how to encourage agents in a multi‑agent game to report unsafe tasks when rewards are shared. It shows that a fully shared policy can be drawn toward silence even when reporting is optimal, and derives audit conditions that guarantee policy‑gradient updates lead to universal reporting. Experiments with PPO on 24 witness graphs demonstrate that grouping witnesses into separate networks reduces unsafe task completion by over 30 percentage points compared to shuffled groups, while a fully shared network only succeeds under a common action draw.
By Avyay M. Casheekar, Hariganesh Tangirala
The paper introduces OWLET, an on-policy weighted learning method for post-training Video Large Language Models. OWLET leverages a richer teacher view that combines global temporal context with question-conditioned local windows to provide visual evidence, and it employs evidence-aware credit redistribution and spatiotemporal state alignment to better utilize this evidence during reinforcement learning. The approach aims to improve video reasoning by aligning token-level supervision with spatiotemporal representations.
By Qitan Lv, Ke Fan, Hong Wang, Wen Wu, Yilin Wang, Bowen Zhou, Feng Wu, Chao Zhang
SOAR (Source-Aware Reinforcement learning) is a new paradigm that transforms monolithic reasoning into source-explicit trajectories for search agents. It disentangles external evidence ($E_{ext}$) and internal knowledge ($E_{int}$) into parallel, observable streams via a Source-Aware Workflow and uses a Source-Consistent Reward to guide policy evolution. Evaluations show that SOAR improves performance on 3B and 7B backbones by 7.3% and 5.4% respectively, while reducing evidence fabrication by 74.8%.
By Shengze Huang, Zihao Tang, Jinluan Yang, Xin Yu, Zhenghao Lin, Xiao Liu, Zengxuan Wen, Yeyun Gong, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Fei Wu, Kun Kuang
The paper introduces PPO‑FP16, a half‑precision variant of proximal policy optimization that incorporates three structure‑aware modules—global scalar protection, policy‑change residual protection, and efficient persistent‑update protection—to maintain numerical stability. These modules preserve the original PPO objective and hyperparameters while enabling reliable training on commodity GPUs. Empirical results on six continuous‑control tasks show PPO‑FP16 achieves the best mean return‑AUC rank, speeds up updates by up to 2×, and reduces optimizer‑state memory by up to 75% and peak GPU memory by up to 45%.
By Linxuan Wu, Liang Chen
AgentTune is a source‑available Python library that offers a unified training lifecycle for tool‑using large language model agents. It introduces a two‑tier EventLog representation to preserve agent trajectories and enable consistent training, evaluation, and auditing. The framework supplies a shared runtime, reinforcement learning for tool‑calling, agentic distillation, retrieval‑augmented generation, decision workflows, evaluation utilities, and a path toward self‑healing agents, all exposed through composable interfaces for flexible combination of strategies and methods.
By R E Zera Marveen Lyngkhoi, Abhivansh Gupta, Vidushee Vats, Ram Mohan Rao Kadiyala, Vinay Kumar Sankarapu, Pratinav Seth
The paper introduces an operator-theoretic approach to offline reinforcement learning for continuous-time stochastic systems. By linking reinforcement learning to the Hamilton‑Jacobi‑Bellman equation, it proposes a dynamic programming recursion that represents the world model through the infinitesimal generator of controlled diffusion processes learned in a reproducing kernel Hilbert space. The authors establish global convergence of the value function under small generator‑approximation error and provide explicit finite‑sample guarantees tied to system properties such as smoothness and stability.
By Nicolas Hoischen, Petar Bevanda, Max Beier, Stefan Sosnowski, Boris Houska, Sandra Hirche