arXiv AI September 10, 2026 By Olivier Lepel, Anas Barakat

Policy Gradients for Cumulative Prospect Theory in Reinforcement Learning

The paper presents a policy gradient theorem tailored to Cumulative Prospect Theory (CPT) objectives in finite-horizon reinforcement learning, extending the classic policy gradient framework to include distortion-based risk measures. Leveraging this theorem, the authors develop a first‑order policy gradient algorithm that uses a Monte Carlo estimator based on order statistics, providing statistical guarantees and proving asymptotic convergence to first‑order stationary points of the generally nonconvex CPT objective. The work also offers a non‑asymptotic sample complexity bound for reaching an approximate stationary policy and demonstrates the qualitative effects of CPT through simulations, comparing the new first‑order method to existing zeroth‑order approaches.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

reinforcement-learning

arXiv Machine Learning

Jun 16

Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process

arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.

By Jongmin Lee, Ernest K. Ryu, Vaneet Aggarwal

reinforcement-learning

arXiv AI

4d ago

On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning

arXiv:2405.14749v3 Announce Type: replace-cross Abstract: Risk-sensitive reinforcement learning (RL) is crucial for maintaining reliable performance in high-stakes applications. While traditional RL...

By Xian Yu, Minheng Xiao, Lei Ying

reinforcement-learning

arXiv Machine Learning

Sep 22

Reinforcement learning with an expectile-based objective

arXiv:2602. 09300v2 Announce Type: replace Abstract: We consider the policy evaluation and control in a finite horizon reinforcement learning (RL) setting under an expectile-based objective.

By Shrey Rakeshkumar Patel, Sumedh Gupte, Soumen Pachal, Prashanth L. A., Sanjay P. Bhat

reinforcement-learning benchmarks

arXiv Machine Learning

Sep 23

Tight Sample Complexity Bounds for Entropic Best Policy Identification

The paper investigates best‑policy identification in finite‑horizon, risk‑sensitive reinforcement learning using the entropic risk measure. It identifies a gap between known lower bounds ≥ η(e^{|eta|H}) and upper bounds ≤ O(e^{2|eta|H}) for sample complexity, attributing the excess factor to loose concentration bounds for exponential utilities. By employing a forward‑model algorithm with KL‑based exploration bonuses and a novel stopping rule, the authors achieve a sample complexity that matches the lower bound, closing the previously open exponential gap.

By Amer Essakine, Claire Vernade

reinforcement-learning benchmarks

arXiv AI

Jul 14

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

arXiv:2506. 07040v4 Announce Type: replace-cross Abstract: We study model-free methods for distributionally robust infinite-horizon average-reward Markov decision processes (MDPs).

By Yang Xu, Swetha Ganesh, Vaneet Aggarwal

reinforcement-learning

arXiv Machine Learning

Jul 31

Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs

arXiv:2607. 28390v1 Announce Type: new Abstract: Constrained Markov Decision Processes (CMDPs) provide a natural framework for reinforcement learning in safety-critical applications, where agents maximize long-term reward while satisfying long-term constraints.

By Ankur Naskar, Vaneet Aggarwal

agents reinforcement-learning safety

One email a morning, machine-written

Related stories

Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process

On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning

Reinforcement learning with an expectile-based objective

Tight Sample Complexity Bounds for Entropic Best Policy Identification

Efficient Q-Learning and Actor-Critic Methods for Robust Average-Reward Reinforcement Learning

Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs