The paper introduces a new approach to learning chance-constrained Markov decision processes (CCMDPs) using a Bellman distributional certificate. It provides both model-based and model-free algorithms with theoretical guarantees, including matching upper and lower bounds for tabular discounted CCMDPs with bounded successor support. Numerical experiments on synthetic CCMDPs and an IEEE 14-bus energy storage benchmark demonstrate the safety and effectiveness of the proposed methods.
By Chenbei Lu, Hongyu Yi
The tutorial titled "Deep Learning for Sequential Decision Making under Uncertainty: Foundations, Frameworks, and Frontiers" explores how modern deep learning techniques—such as neural networks, transformers, large language models, and deep reinforcement learning—can be integrated with operations research and management science to address complex, uncertain, and dynamic decision problems. It argues that deep learning should complement, not replace, optimization, offering adaptability and scalable approximation while OR/MS provides rigorous constraint and uncertainty modeling. The tutorial organizes the field around predict‑then‑optimize, decision‑aware learning, constraint‑aware decision generation, and deep reinforcement learning, and highlights applications across supply chains, healthcare, energy, and autonomous systems.
By I. Esra Buyuktahtakin
arXiv:2608. 02343v1 Announce Type: cross Abstract: Many operational problems are constrained sequential decision processes with large, combinatorial action spaces and interdependent feasibility constraints.
By Patrick Helm, Jan-Niklas Doerr, Joren Gijsbrechts, Stefan Minner
arXiv:2607. 14373v1 Announce Type: new Abstract: We propose a noise-robust elicit-to-optimize framework that integrates inverse reinforcement learning (IRL) and reinforcement learning (RL) for eliciting agents' risk preferences and optimizing policies under a broad class of risk objectives characterized by distortion riskmetrics.
By Yang Liu, Yuhao Liu, Yunran Wei
arXiv:2607. 06610v1 Announce Type: cross Abstract: Portfolio optimization under uncertainty is inherently a multi-objective decision problem involving complex interactions among return, risk, market dynamics, and practical investment constraints.
By Sounaq Das, Tanmay Sen, Raghu Nandan Sengupta, Aditya Gupta
arXiv:2608. 03562v1 Announce Type: new Abstract: Reinforcement learning (RL) with general utility extends classic RL by optimizing an arbitrary utility functional of the policy-induced occupancy measure, thereby enabling a broader range of applications.
By Zixuan Liu, Fangzheng Wu, Brian Summa, Zizhan Zheng
The paper presents a convergence framework for deep $V$‑learning over a finite horizon $H$, deriving explicit bounds on policy loss by decomposing the Bellman update error into six residuals. It shows how $L^s$ concentrability controls expected $L^1$ loss, quantifies the impact of shared sampling across horizon levels, and provides optimal and near‑optimal sample allocations for statistical error rates. The work also establishes sharp action‑gap bounds under a margin condition, transfers optimal‑gap results to frozen‑iterate gaps, and offers consistency guarantees for generative‑reset approximate‑ERM procedures with exact action scores.
By Yury Kolomeytsev
arXiv:2606. 10979v1 Announce Type: new Abstract: Many Markov decision processes (MDPs) in operations research have feasible actions that are state dependent and defined implicitly by various operational constraints.
By Yi Chen (Lucy), Rushuai Yang (Lucy), Qiang Chen (Lucy), Dongyan (Lucy), Huo
arXiv:2602. 03778v2 Announce Type: replace-cross Abstract: Tail-end risk measures such as static conditional value-at-risk (CVaR) are used in safety-critical applications to prevent rare, yet catastrophic events.
By Aneri Muni, Vincent Taboga, Esther Derman, Pierre-Luc Bacon, Erick Delage
arXiv:2608. 01151v1 Announce Type: cross Abstract: In this paper, we consider stochastic optimal control problems with infinite-horizon joint chance constraints.
By Francesco Cordiano, Kanghui He, Bart De Schutter
arXiv:2606. 24999v1 Announce Type: new Abstract: High-dimensional partial differential equations (PDEs) with unknown coefficients arise widely in scientific machine learning, including continuous-time reinforcement learning, yet solving them efficiently in a data-driven way remains challenging.
By Yanwei Jia, Du Ouyang, Huy\^en Pham, Xun Yu Zhou
The paper introduces reinforcement learning for Continuous-Time Jump Markov Decision Processes (CTJMDPs) with general discrete state spaces and continuous/discrete actions. It develops entropy‑regularized continuous‑time control and establishes theoretical foundations for q‑learning in this setting, providing model‑free algorithms that outperform naive discretization. Numerical tests on network dynamic pricing demonstrate the method’s ability to learn near‑optimal policies and scale to large networks.
By Huiling Meng, Ningyuan Chen, Xuefeng Gao