arXiv Machine Learning
Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 11
Statistical analysis of Inverse Entropy-regularized Reinforcement Learning
The paper introduces a statistical framework for Inverse Entropy-regularized Reinforcement Learning that resolves the non-uniqueness of reward functions by combining entropy regularization with a least-squares reconstruction of the reward from the soft Bellman residual. It models expert demonstrations as a Markov chain, estimates the expert policy via penalized maximum likelihood, and provides high-probability bounds on the excess Kullback–Leibler divergence between the estimated and true policies. These results yield non-asymptotic minimax optimal convergence rates for the least-squares reward function, highlighting the trade-offs among smoothing, model complexity, and sample size.
arXiv AI
Aug 7
Fast Rates for Inverse Reinforcement Learning
arXiv:2605. 14599v2 Announce Type: replace-cross Abstract: We establish novel structural and statistical results for entropy-regularized min-max inverse reinforcement learning (Min-Max-IRL) in finite-horizon MDPs with Borel state and action spaces.
arXiv Machine Learning
Jun 16
Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
arXiv:2606. 16759v1 Announce Type: new Abstract: We study inverse reinforcement learning for discrete-time, infinite-horizon mean-field games (MFGs) under an average-reward criterion.
arXiv AI
1d ago
Diffusion-Augmented Markov Decision Processes for Maximum Entropy Reinforcement Learning
The paper introduces Diffusion-Augmented Markov Decision Processes (DA‑MDPs), a framework that extends Maximum Entropy Reinforcement Learning to diffusion-based policies. DA‑MDPs treat each reverse‑diffusion step as an RL decision, deriving a tractable reverse‑KL bound that decomposes across denoising transitions and yields diffusion‑augmented soft rewards, value functions, and policy objectives. The authors implement this framework with PPO, REPPO, and a maximum‑entropy WPO variant, showing improved continuous‑control performance, higher success rates on manipulation tasks, and memory‑efficient training with action chunking.
arXiv Statistics ML
Aug 25
Semiparametric Double Reinforcement Learning with Applications to Long-Term Causal Inference
arXiv:2501.06926v5 Announce Type: replace Abstract: Double reinforcement learning (DRL) provides efficient off-policy inference for policy values in nonparametric Markov decision processes (MDPs), bu...
arXiv Machine Learning
Jun 18
Wasserstein Policy Learning for Distributional Outcomes
arXiv:2606. 19117v1 Announce Type: cross Abstract: Offline policy learning has received growing attention in causal inference.