arXiv:2605. 02141v2 Announce Type: replace-cross Abstract: Kullback-Leibler (KL) regularization is widely used in offline decision-making and offers several benefits, motivating recent work on the sample complexity of offline learning with respect to KL-regularized performance metrics.
By Kaixuan Ji, Qiwei Di, Heyang Zhao, Qingyue Zhao, Quanquan Gu
arXiv:2603. 02155v2 Announce Type: replace Abstract: Recent studies have shown that reinforcement learning with KL-regularized objectives can enjoy faster rates of convergence or logarithmic regret, in contrast to the classical $\sqrt{T}$-type regret in the unregularized setting.
By Kaixuan Ji, Qingyue Zhao, Heyang Zhao, Qiwei Di, Quanquan Gu
The paper introduces Dually Regularized AIL, a model‑free algorithm for adversarial imitation learning that jointly applies KL policy regularization and a quadratic reward penalty based on expert and learner occupancies. It proves fast convergence rates, achieving a ×O(1/K+1/N) bound on the regularized imitation gap in finite‑horizon MDPs with general function approximation, and establishes the first algorithm to attain ×O(1/ε) sample complexity in both expert demonstrations and online interactions for this regularized objective.
By Hanbin Zhou, Shangzhe Li, Alexander Braverman, Weitong Zhang
The paper investigates offline KL‑regularized reinforcement learning in finite discounted Markov decision processes with unknown stochastic transitions. It establishes lower bounds on sample complexity for estimators covering the regularized optimum and proposes a pessimistic soft value iteration algorithm that achieves near‑optimal rates under certain temperature and concentrability conditions. The study also highlights the limitations of plug‑in estimators on reference‑policy data and the impact of temperature on sample requirements.
By Andrew Liu
arXiv:2610.08963v1 Announce Type: new
Abstract: Asynchronous reinforcement learning (RL) for large language model (LLM) agents trains one policy on trajectories generated by another: rollouts come fr...
By Yifan Zhang
arXiv:2609.36945v1 Announce Type: new
Abstract: We study the learning dynamics of fine-tuning a policy model on self-generated and reward-weighted data, with particular focus on a generalized version...
By Zhiwei Wang, Yanxi Chen, Yaliang Li, Bolin Ding
arXiv:2603. 10184v2 Announce Type: replace-cross Abstract: Statistical inference with bandit data presents fundamental challenges owing to adaptive sampling, which violates the independence assumptions underlying classical asymptotic theory.
By Budhaditya Halder, Ishan Sengupta, Koustav Chowdhury, Samya Praharaj, Koulik Khamaru
arXiv:2610.18839v1 Announce Type: cross
Abstract: This paper investigates offline Q-learning for finite-horizon Markov decision processes with Kullback-Leibler (KL) regularization. Recent studies hav...
By Yize Hao, Guangxin Jiang, Shuang Qiu
arXiv:2602. 23116v3 Announce Type: replace Abstract: We consider the problem of regularized best-response max-regret minimization in online RLHF under general preferences and bandit feedback.
By Junghyun Lee, Minju Hong, Kwang-Sung Jun, Chulhee Yun, Se-Young Yun
arXiv:2605. 09454v2 Announce Type: replace-cross Abstract: We study the $\textit{single-index bandit}$ problem, where rewards depend on an unknown one-dimensional projection of high-dimensional contexts through an unknown reward function.
By Devdan Dey, Sujoy Bhore, Avishek Ghosh
The paper investigates transfer learning in stochastic multi‑armed bandits when historical data come from an unknown adaptive policy. It establishes asymptotic problem‑dependent lower bounds and introduces the KL‑UCB‑Transfer algorithm, which combines online and historical data to achieve logarithmic regret with an improved constant under certain mean‑gap conditions. The study also relaxes the source‑target discrepancy assumption to a one‑sided condition, demonstrating that logarithmic regret can still be achieved under misspecification and identifying regimes of improvement, degradation, or loss of guarantee.
By Adrien Prevost, Timothee Mathieu, Waris Radji, Odalric-Ambrym Maillard
arXiv:2510. 22819v3 Announce Type: replace Abstract: The convergence analysis of online learning algorithms is central to machine learning theory, where the last-iterate convergence is particularly important, as it captures the learner's actual decisions and describes the evolution of the learning process over time.
By Jingxin Zhan, Yuze Han, Zhihua Zhang