The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
By Tuan Dam, Kishan Panaganti, Brahim Driss, Adam Wierman
arXiv:2602. 17315v3 Announce Type: replace-cross Abstract: We introduce Flickering Multi-Armed Bandits (FMAB) to model sequential decision-making in environments with changing action availability, where accessibility of the next action is restricted to a subset dependent on the agent's current choice.
By Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni, Lijun Chen
Graph-Based Stochastic-Power-UCT (GS-Power-UCT) is a Monte‑Carlo graph search algorithm that shares states reached at the same planning depth while keeping separate values for different depths, thereby reducing duplicate simulations in stochastic MDPs. The method guarantees that, for a fixed horizon, the root estimate converges to the finite‑horizon value at an $O(n^{-1/2})$ rate, matching tree‑based Stochastic‑Power‑UCT but with improved sample reuse. Two full‑state variants—GS‑Power‑UCT‑F and GS‑Power‑UCT‑F$^+$—extend the approach to single‑node per physical state and adaptive horizons, respectively, with the latter converging to the optimal infinite‑horizon discounted value when cross‑depth bias vanishes. Experiments on stochastic planning benchmarks demonstrate that GS‑Power‑UCT outperforms both tree‑based and other graph‑based baselines in sample efficiency.
By Tung Tran, Viet Bao Mai, Hoang Ta, Tuan Dam
Finding policies for Markov Decision Processes (MDPs) is a central problem in areas such as Reinforcement Learning and Operations Research. Here, we have to repeatedly choose an action that should be...
The paper introduces a new algorithm for a nonstationary bandit setting where actions influence both immediate rewards and the evolution of an unobserved latent linear state. By approximating the infinite‑memory reward process with a finite‑memory block‑level proxy and applying a UCB‑based block algorithm, the authors achieve a regret bound of “~O(√T)”, improving upon the previous “~O(T^{2/3})” guarantee. This represents the first such “~O(√T)” result for latent linear‑dynamics bandits with bilinear rewards and an open‑loop action‑sequence benchmark.
By Taehyun Hwang, Hyunjun Choi, Heesang Ann, Min-hwan Oh
arXiv:2610. 23565v1 Announce Type: new Abstract: Monte-Carlo Tree Search (MCTS) typically stores only scalar mean estimates, which makes it poorly suited for tail-risk objectives such as conditional value-at-risk (CVaR).
By Quynh Trinh, Trung Nguyen, Nam Nguyen, Tuan Dam
arXiv:2610.15461v1 Announce Type: new
Abstract: MDP planners often rely on nominal transition models that may differ from the true dynamics. Robust MDP planning accounts for this uncertainty through...
By Tomer Massas, Vadim Indelman
arXiv:2609.15539v1 Announce Type: cross
Abstract: Finding policies for Markov Decision Processes (MDPs) is a central problem in areas such as Reinforcement Learning and Operations Research. Here, we...
By Lars Rohwedder, Rico Zenklusen
arXiv:2607. 05359v1 Announce Type: new Abstract: Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding.
By Idan Lev-Yehudi, Vadim Indelman
The paper introduces Graph-Based Stochastic-Power-UCT (GS-Power-UCT), a Monte‑Carlo graph search algorithm that shares states reached at the same planning depth while maintaining separate values for different depths. It achieves an $O(n^{-1/2})$ convergence rate for the root estimate in finite‑horizon stochastic MDPs, matching tree‑based methods but with better sample reuse. Two full‑state variants, GS-Power-UCT‑F and GS-Power-UCT‑F$^+$, further explore sample sharing and bias control, with GS-Power-UCT‑F$^+$ converging to the optimal infinite‑horizon discounted value when the cross‑depth gap vanishes. Experiments on stochastic planning benchmarks demonstrate improved sample efficiency over existing tree‑based and graph‑based baselines.
arXiv:2610.15046v1 Announce Type: new
Abstract: Beliefs in a Partially Observable Markov Decision Process (POMDP) are posterior distributions over the hidden state given the agent's history of action...
By Christopher Solinas, Radovan Halu\v{s}ka, Martin Schmid, Michael Buro, Nathan R. Sturtevant, Michael Bowling
arXiv:2607. 09298v1 Announce Type: cross Abstract: We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives.
By Pedro P. Santos, F\'abio Vital, Alberto Sardinha, Francisco S. Melo