arXiv Machine Learning

Global Convergence of Wasserstein Policy Gradient for Entropy-Regularized Reinforcement Learning

arXiv:2605. 26078v3 Announce Type: replace Abstract: Wasserstein policy gradient (WPG) is a policy optimization method for reinforcement learning (RL) that exploits the optimal-transport geometry of action distributions.

arXiv Machine Learning

Jul 21

Distributional Soft Bellman Operator under the Cram\'er Geometry

arXiv:2607. 17897v1 Announce Type: new Abstract: Distributional soft policy iteration (DSPI) provides an important framework for combining distributional reinforcement learning (DRL) with maximum-entropy control, in which the policy evaluation step is governed by a distributional soft Bellman operator acting on entropy-regularised returns.

By Keru Wang, Yixin Deng, Yao Lyu, Stephen Redmond, Shengbo Eben Li

reinforcement-learning

arXiv Machine Learning

Aug 10

Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Control

arXiv:2608. 07433v1 Announce Type: cross Abstract: Wasserstein policy gradient (WPG) updates state-conditional action laws by transport in the action space.

By Zhaoyu Zhu, Rui Gao, Shuang Li

reinforcement-learning

arXiv Machine Learning

Jul 22

Linear convergence of proximal descent schemes on the Wasserstein space

arXiv:2411. 15067v2 Announce Type: replace-cross Abstract: We investigate proximal descent methods, inspired by the minimizing movement scheme introduced by Jordan, Kinderlehrer and Otto, for optimizing entropy-regularized functionals on the Wasserstein space.

By Razvan-Andrei Lascu, Mateusz B. Majka, David \v{S}i\v{s}ka, {\L}ukasz Szpruch

arXiv AI

Jun 16

Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning

arXiv:2606. 16515v1 Announce Type: cross Abstract: Hamilton-Jacobi-Bellman theory implies that the optimal goal-conditioned action depends on the goal only through the gradient of the goal-reaching distance at the current state, yet standard online GCRL still conditions the actor on the raw goal -- a signal that is geometrically uninformative when the goal is far from the data distribution.

By Swaminathan S K, Damiya Gondha, Theyanesh Eswaramoorthy Rajahkrishnan, Aritra Hazra

reinforcement-learning robotics

arXiv Machine Learning

Jun 16

Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward

arXiv:2606. 16759v1 Announce Type: new Abstract: We study inverse reinforcement learning for discrete-time, infinite-horizon mean-field games (MFGs) under an average-reward criterion.

By \c{S}evket Kaan Alk{\i}r, Naci Sald{\i}, Berkay Anahtarc{\i}, Can Deha Kar{\i}ks{\i}z

reinforcement-learning

arXiv AI

Jun 6

Retry Policy Gradients in Continuous Action Spaces

arXiv:2606. 05888v1 Announce Type: new Abstract: Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses.

By Soichiro Nishimori, Paavo Parmas

arXiv Machine Learning

2d ago

Offline Deep Q* Estimation with Diffusion Models

arXiv:2608. 14401v1 Announce Type: cross Abstract: In offline RL, estimating the optimal action-value function $Q^*$ can be formulated as solving the optimal Bellman equation based solely on offline observations.

By Xiaohong Chen, Yuling Jiao, Lican Kang, Jerry Zhijian Yang, Chen Zhong

diffusion reinforcement-learning

arXiv Machine Learning

Jun 26

Mean-Field PhiBE: Continuous-Time Mean-Field Reinforcement Learning from Discrete-Time Data

arXiv:2606. 26498v1 Announce Type: cross Abstract: This paper addresses model-free continuous-time mean-field control in a setting where the population dynamics evolve continuously according to an unknown McKean-Vlasov stochastic differential equation, while only discrete-time transition data are available.

By Erhan Bayraktar, Martin Hernandez, Qinxin Yan, Yuhua Zhu

diffusion reinforcement-learning benchmarks

arXiv Machine Learning

Jul 17

Bridging the Gap between Newton-Raphson Method and Regularized Policy Iteration

arXiv:2310. 07211v2 Announce Type: replace Abstract: Regularization is a cornerstone of modern reinforcement learning.

By Zeyang Li, Chuxiong Hu, Yunan Wang, Guojian Zhan, Jie Li, Yao Lyu, Shengbo Eben Li

reinforcement-learning

arXiv Machine Learning

Jun 30

Entropy-Regularized Reinforcement Learning for Linear-Quadratic Stackelberg Differential Games in Regime-Switching Diffusion Models

arXiv:2606. 28671v1 Announce Type: new Abstract: Stackelberg differential games (SDGs) provide a powerful framework for hierarchical decision-making in stochastic and continuous-time environments, yet their solution remains computationally challenging due to the complexity of traditional dynamic programming and Hamilton-Jacobi-Bellman-Isaacs (HJBI) methods, especially in high-dimensional systems.

By Congde Hu, Danping Li, Lin Xu, Wenying Xu

diffusion reinforcement-learning

arXiv Machine Learning

Jun 18

Wasserstein Policy Learning for Distributional Outcomes

arXiv:2606. 19117v1 Announce Type: cross Abstract: Offline policy learning has received growing attention in causal inference.

By Yiyan Huang, Cheuk Hang Leung, Qi Wu, Zhiheng Zhang

arXiv AI

Jun 19

Conditional Diffusion Guidance under Hard Constraint: A Stochastic Analysis Approach

arXiv:2602. 05533v3 Announce Type: replace Abstract: We study conditional generation in diffusion models under hard constraints, where generated samples must satisfy prescribed events with probability one.

By Zhengyi Guo, Wenpin Tang, Renyuan Xu

diffusion safety

Related stories