The article argues that Monte Carlo Tree Search (MCTS) and every‑visit Monte Carlo (MC) control are essentially the same method, differing only in terminology and presentation. It shows that MCTS’s four stages—selection, expansion, simulation, and backup—can be reduced to two core operations: sampling trajectories under the current policy and performing every‑visit MC updates. The note aims to make this equivalence explicit and easier to recognize.
arXiv:2606. 15247v1 Announce Type: cross Abstract: The asymptotic behaviour of Monte Carlo Exploring Starts (MCES) is a long-standing open question in reinforcement learning, even in the tabular setting.
arXiv:2608. 15700v1 Announce Type: new Abstract: Background: Distillation of training targets generated thru search/planning has proven useful in reinforcement learning, but search can take exceedingly long.
The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
arXiv:2605.08982v4 Announce Type: replace Abstract: Monte Carlo Tree Search (MCTS) is a widely used approach for policy improvement and action selection in Reinforcement Learning. Due to its sequenti...
arXiv:2609.06489v1 Announce Type: cross Abstract: Monte Carlo Tree Search (MCTS) has demonstrated success in online planning for deterministic environments, yet significant challenges remain in adapt...
arXiv:2607. 15610v1 Announce Type: cross Abstract: Reinforcement learning (RL) has become a key approach for training LLM agents, yet popular methods such as GRPO/RLOO rely on multiple independently sampled complete trajectories for advantage estimation.
arXiv:2609.40035v1 Announce Type: new Abstract: The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We...
arXiv:2606. 10580v1 Announce Type: cross Abstract: The asymptotic behaviour of Monte Carlo optimistic policy iteration (MC-O-PI) is a long-standing open question.
The paper introduces Particle MCTS (PMCTS), a parallelized version of Monte Carlo Tree Search designed for GPU acceleration and batch-parallel neural network evaluations. PMCTS preserves policy improvement guarantees of modern MCTS algorithms while scaling efficiently with parallel compute. Empirical results show that PMCTS consistently outperforms or matches heuristic-based baselines across various MCTS and reinforcement learning domains, including chess, Go, and both discrete and continuous control benchmarks.
DiffuSearch is a hybrid trajectory planner for autonomous driving that unifies objectives across both generation and refinement stages. It first uses a guided diffusion model to produce scene-consistent joint trajectories, then refines them with a Monte Carlo Tree Search that shares the same driving goals—collision avoidance, drivable area compliance, comfort, and progress. Experiments on nuPlan and interPlan benchmarks show that this synergy reduces collisions and improves comfort, especially in complex interactive scenarios.
GEM-MPC is a reinforcement learning method that blends MPPI planning with policy learning to balance exploration and exploitation in high-dimensional continuous control tasks. It trains a policy to clone the planner while also maintaining a KL-regularized policy that explores around the planner’s suggestions, thereby improving the synergy between planning and learning. The approach introduces Gated Prior Distillation, which selectively updates policies from stored planning distributions only when they offer better targets, reducing the influence of stale data without costly reanalysis. Across continuous-control benchmarks, GEM-MPC outperforms existing planning-based baselines while using lower computational budgets.
arXiv:2610.15046v1 Announce Type: new Abstract: Beliefs in a Partially Observable Markov Decision Process (POMDP) are posterior distributions over the hidden state given the agent's history of action...