arXiv AI
Jul 13
Risk-Aware General-Utility Markov Decision Processes
arXiv:2607. 09298v1 Announce Type: cross Abstract: We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives.
Long-Term Sequential Decision Making under Risk
arXiv:2607. 19914v1 Announce Type: new Abstract: We study finite-horizon MDP planning under \emph{root-based} (resolute) risk objectives that apply a rank-dependent functional to the distribution of total returns.
Related stories
arXiv AI
4d ago
Risk-Averse Online POMDP Planning via CVaR of the Immediate Cost with Performance Guarantees
arXiv:2609.35874v1 Announce Type: new Abstract: Online POMDP planners optimize the expected cumulative cost, which can mask dangerous states when the belief places significant mass on high-cost state...
arXiv AI
Jul 1
Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity
arXiv:2602. 03778v2 Announce Type: replace-cross Abstract: Tail-end risk measures such as static conditional value-at-risk (CVaR) are used in safety-critical applications to prevent rare, yet catastrophic events.
arXiv AI
Aug 11
Control-Oriented Scenario Tree Construction through Reinforcement Learning
arXiv:2608. 09335v1 Announce Type: new Abstract: Multistage stochastic model predictive control (MPC) handles uncertainty by optimizing over a scenario tree, a finite branching approximation of future outcomes constructed from sampled forecasts.
arXiv Machine Learning
5d ago
Learning Chance-Constrained MDPs with Bellman Distributional Certificates
The paper introduces a new approach to learning chance-constrained Markov decision processes (CCMDPs) using a Bellman distributional certificate. It provides both model-based and model-free algorithms with theoretical guarantees, including matching upper and lower bounds for tabular discounted CCMDPs with bounded successor support. Numerical experiments on synthetic CCMDPs and an IEEE 14-bus energy storage benchmark demonstrate the safety and effectiveness of the proposed methods.
Hugging Face Trending Papers
Aug 10
Control-Oriented Scenario Tree Construction through Reinforcement Learning
Multistage stochastic model predictive control (MPC) handles uncertainty by optimizing over a scenario tree, a finite branching approximation of future outcomes constructed from sampled forecasts. To build such a tree, conventional methods focus on matching the underlying probability distribution---e.
arXiv Machine Learning
Jun 19
Quantile of Means: A Bonus-Free Ensemble Method for Minimax Optimal Reinforcement Learning
arXiv:2606. 20107v1 Announce Type: new Abstract: Optimal Reinforcement Learning (RL) algorithms typically rely on carefully constructed count-based uncertainty estimates to drive exploration.
arXiv Machine Learning
Jul 1
Policy Optimization Achieves Data-Dependent Regret Bounds in MDPs with Unknown Transitions
arXiv:2606. 31769v1 Announce Type: new Abstract: We study policy optimization for online episodic tabular Markov decision processes with unknown transition kernels, aiming for best-of-both-worlds guarantees together with data-dependent regret bounds.
arXiv Machine Learning
Aug 28
Sequential Additivity in Distributionally Robust Ranking and Selection
The paper studies distributionally robust ranking and selection (DRR&S), where the goal is to identify the best alternative under input uncertainty by considering multiple plausible input distributions. It introduces the concept of sequential additivity, showing that efficient sampling should focus on a small, additive set of critical scenarios rather than a multiplicative number. The authors prove an algorithm‑independent lower bound on sampling, design an additive allocation (AA) procedure that meets this bound and achieves exponentially decreasing error probability, and extend the approach to a general additive allocation (GAA) framework that incorporates traditional R&S sampling rules.
arXiv AI
Jul 7
Graph Sparse Sampling: Breaking the Curse of the Horizon in Continuous MDP Planning
arXiv:2607. 05359v1 Announce Type: new Abstract: Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding.
arXiv AI
Jun 29
RS-Diffuser: Risk-Sensitive Diffusion Planning with Distributional Value Guidance
arXiv:2606. 27766v1 Announce Type: cross Abstract: Offline reinforcement learning enables policy learning from fixed datasets without additional environment interaction, making it appealing for safety-critical applications where online exploration is costly or unsafe.
arXiv Machine Learning
Sep 23
Tight Sample Complexity Bounds for Entropic Best Policy Identification
The paper investigates best‑policy identification in finite‑horizon, risk‑sensitive reinforcement learning using the entropic risk measure. It identifies a gap between known lower bounds ≥ η(e^{|eta|H}) and upper bounds ≤ O(e^{2|eta|H}) for sample complexity, attributing the excess factor to loose concentration bounds for exponential utilities. By employing a forward‑model algorithm with KL‑based exploration bonuses and a novel stopping rule, the authors achieve a sample complexity that matches the lower bound, closing the previously open exponential gap.