The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
By Tuan Dam, Kishan Panaganti, Brahim Driss, Adam Wierman
arXiv:2607. 19914v1 Announce Type: new Abstract: We study finite-horizon MDP planning under \emph{root-based} (resolute) risk objectives that apply a rank-dependent functional to the distribution of total returns.
By Irmaan (Mohammad), Mirzanejad, Nadjet Bourdache, Abdel-Illah Mouaddib
arXiv:2609.06489v1 Announce Type: cross
Abstract: Monte Carlo Tree Search (MCTS) has demonstrated success in online planning for deterministic environments, yet significant challenges remain in adapt...
By Tuan Dam
arXiv:2608. 09335v1 Announce Type: new Abstract: Multistage stochastic model predictive control (MPC) handles uncertainty by optimizing over a scenario tree, a finite branching approximation of future outcomes constructed from sampled forecasts.
By Fabio Pavirani, Bert Claessens, Pierre Pinson, Chris Develder
Multistage stochastic model predictive control (MPC) handles uncertainty by optimizing over a scenario tree, a finite branching approximation of future outcomes constructed from sampled forecasts. To build such a tree, conventional methods focus on matching the underlying probability distribution---e.
arXiv:2609.40035v1 Announce Type: new
Abstract: The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We...
By Junyu Lu, Shichao Weng, Zhiqiang Wang, Haojie Luo, Jingfan Zhang, Yuhua Zhou, Cheng Du, Yuzhuo Zhang, Xi Li, Jinwei Du, Tiancheng Feng, Chuan Xiao, Shuyuan Zheng
arXiv:2607. 09298v1 Announce Type: cross Abstract: We study general-utility Markov decision processes (GUMDPs) with risk-aware objectives.
By Pedro P. Santos, F\'abio Vital, Alberto Sardinha, Francisco S. Melo
arXiv:2610.27575v1 Announce Type: new
Abstract: Model-based reinforcement learning in partially observable environments requires inferring hidden states, learning latent dynamics, and planning under...
By Kevin Tan, Ziping Xu
The paper introduces a new algorithm for a nonstationary bandit setting where actions influence both immediate rewards and the evolution of an unobserved latent linear state. By approximating the infinite‑memory reward process with a finite‑memory block‑level proxy and applying a UCB‑based block algorithm, the authors achieve a regret bound of “~O(√T)”, improving upon the previous “~O(T^{2/3})” guarantee. This represents the first such “~O(√T)” result for latent linear‑dynamics bandits with bilinear rewards and an open‑loop action‑sequence benchmark.
By Taehyun Hwang, Hyunjun Choi, Heesang Ann, Min-hwan Oh
TreeHCA introduces a method that couples hindsight-guided exploration with hindsight credit assignment for multi-turn large language model agents. By using changes in reference-answer likelihood to revisit intermediate histories and sample alternative continuations, it builds a tree of reasoning and tool interactions, and applies an inverse-hindsight importance correction to propagate terminal outcomes into turn-level advantages. Experiments on seven question-answering benchmarks with three Qwen backbones show that TreeHCA outperforms all baselines in aggregate exact match.
By Debajoy Mukherjee, Jeremy Carleton, Sushil Vemuri, Srinivas Shakkottai, Dileep Kalathil
arXiv:2602. 06014v2 Announce Type: replace-cross Abstract: Thompson sampling (TS) is widely used for stochastic multi-armed bandits, yet its inferential properties under adaptive data collection are subtle.
By Shunxing Yan, Han Zhong
Graph-Based Stochastic-Power-UCT (GS-Power-UCT) is a Monte‑Carlo graph search algorithm that shares states reached at the same planning depth while keeping separate values for different depths, thereby reducing duplicate simulations in stochastic MDPs. The method guarantees that, for a fixed horizon, the root estimate converges to the finite‑horizon value at an $O(n^{-1/2})$ rate, matching tree‑based Stochastic‑Power‑UCT but with improved sample reuse. Two full‑state variants—GS‑Power‑UCT‑F and GS‑Power‑UCT‑F$^+$—extend the approach to single‑node per physical state and adaptive horizons, respectively, with the latter converging to the optimal infinite‑horizon discounted value when cross‑depth bias vanishes. Experiments on stochastic planning benchmarks demonstrate that GS‑Power‑UCT outperforms both tree‑based and other graph‑based baselines in sample efficiency.
By Tung Tran, Viet Bao Mai, Hoang Ta, Tuan Dam