The paper introduces a statistical framework for Inverse Entropy-regularized Reinforcement Learning that resolves the non-uniqueness of reward functions by combining entropy regularization with a least-squares reconstruction of the reward from the soft Bellman residual. It models expert demonstrations as a Markov chain, estimates the expert policy via penalized maximum likelihood, and provides high-probability bounds on the excess Kullback–Leibler divergence between the estimated and true policies. These results yield non-asymptotic minimax optimal convergence rates for the least-squares reward function, highlighting the trade-offs among smoothing, model complexity, and sample size.
By Denis Belomestny, Alexey Naumov, Artemy Rubtsov, Sergey Samsonov
arXiv:2605. 14599v2 Announce Type: replace-cross Abstract: We establish novel structural and statistical results for entropy-regularized min-max inverse reinforcement learning (Min-Max-IRL) in finite-horizon MDPs with Borel state and action spaces.
By Andreas Schlaginhaufen, Maryam Kamgarpour
arXiv:2606. 16759v1 Announce Type: new Abstract: We study inverse reinforcement learning for discrete-time, infinite-horizon mean-field games (MFGs) under an average-reward criterion.
By \c{S}evket Kaan Alk{\i}r, Naci Sald{\i}, Berkay Anahtarc{\i}, Can Deha Kar{\i}ks{\i}z
arXiv:2610.32125v1 Announce Type: new
Abstract: Offline maximum entropy reinforcement learning seeks to estimate the optimal soft action-value function \(Q^*\) from a fixed dataset without further in...
By Hao Cheng, Yuling Jiao, Lican Kang, Jerry Zhijian Yang, Chen Zhong
The paper introduces Diffusion-Augmented Markov Decision Processes (DA‑MDPs), a framework that extends Maximum Entropy Reinforcement Learning to diffusion-based policies. DA‑MDPs treat each reverse‑diffusion step as an RL decision, deriving a tractable reverse‑KL bound that decomposes across denoising transitions and yields diffusion‑augmented soft rewards, value functions, and policy objectives. The authors implement this framework with PPO, REPPO, and a maximum‑entropy WPO variant, showing improved continuous‑control performance, higher success rates on manipulation tasks, and memory‑efficient training with action chunking.
By Sebastian Sanokowski, Kaustubh Patil, Majid Khadiv
arXiv:2501.06926v5 Announce Type: replace
Abstract: Double reinforcement learning (DRL) provides efficient off-policy inference for policy values in nonparametric Markov decision processes (MDPs), bu...
By Lars van der Laan, David Hubbard, Allen Tran, Nathan Kallus, Aur\'{e}lien Bibaut
arXiv:2606. 19117v1 Announce Type: cross Abstract: Offline policy learning has received growing attention in causal inference.
By Yiyan Huang, Cheuk Hang Leung, Qi Wu, Zhiheng Zhang
arXiv:2607. 14373v1 Announce Type: new Abstract: We propose a noise-robust elicit-to-optimize framework that integrates inverse reinforcement learning (IRL) and reinforcement learning (RL) for eliciting agents' risk preferences and optimizing policies under a broad class of risk objectives characterized by distortion riskmetrics.
By Yang Liu, Yuhao Liu, Yunran Wei
The paper introduces a doubly robust framework for distributionally robust offline contextual pricing, addressing challenges of distributional shifts and continuous actions. It proposes a localized DR estimator that is computationally efficient and semiparametrically optimal, and a learning algorithm that exploits demand noise smoothness to achieve finite‑sample regret bounds of τ^{-σ/(2σ+1)} for smoothness orders s=1,2. Experiments across varying distribution shifts demonstrate the effectiveness of the proposed methods.
By Min Xu, Xinyi Yin, Yunfan Zhang, Yuxuan Han, Houcai Shen, Caihua Chen
Hyperbolic Inverse Reinforcement Learning (Hyperbolic IRL) introduces a reward function that depends on the Hyperbolic distance between feature vectors and a learned parameter vector, offering an alternative to traditional linear or neural network approaches. The method matches state‑of‑the‑art performance on standard benchmarks, performs especially well with limited data and in dense environments, and uncovers complex, non‑separable reward structures in human navigation tasks. Hyperbolic IRL thus enhances both predictive accuracy and interpretability of inferred reward functions.
By Iulia Rusu, Anoop Praturu, Sergei Gepshtein, Tatyana Sharpee
arXiv:2607. 14522v1 Announce Type: new Abstract: We formulate reinforcement learning (RL) in continuous time with discrete state spaces and possibly arbitrary action spaces via a stochastic control approach, where the state dynamics are modeled as a controlled continuous-time Markov chain (CTMC).
By Zikun Zhang, Jiayuan Sheng, David D. Yao, Wenpin Tang
arXiv:2607. 08647v1 Announce Type: cross Abstract: As autonomous agents are increasingly deployed across diverse operational contexts, aligning their behavior with human intent demands reward functions that remain robust to such changes rather than overfitting to any single environment.
By Ali Larian, Qian Lin, Chang Zong Wu, Daniel S. Brown