arXiv Machine Learning
Particle Monte Carlo Tree Search
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Sep 10
PMCTS: Principled Parallelized Inference Time Scaling with Particle Monte Carlo Tree Search
The paper introduces Particle MCTS (PMCTS), a parallelized version of Monte Carlo Tree Search designed for GPU acceleration and batch-parallel neural network evaluations. PMCTS preserves policy improvement guarantees of modern MCTS algorithms while scaling efficiently with parallel compute. Empirical results show that PMCTS consistently outperforms or matches heuristic-based baselines across various MCTS and reinforcement learning domains, including chess, Go, and both discrete and continuous control benchmarks.
arXiv AI
Aug 18
Adaptive Mixing of Policies from Searching and Policies from Learning
arXiv:2608. 15700v1 Announce Type: new Abstract: Background: Distillation of training targets generated thru search/planning has proven useful in reinforcement learning, but search can take exceedingly long.
arXiv AI
Sep 17
Online Robust Reinforcement Learning Through Monte-Carlo Planning
The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
arXiv AI
Sep 10
The Surprising Effectiveness of Approximate Value Iteration in Self-Play
arXiv:2609.09094v1 Announce Type: new Abstract: Combining search with function approximation has driven major advances in game-playing programs, making self-play algorithms more competitive than ever...
arXiv Machine Learning
Aug 31
Is Monte Carlo Tree Search Just Every-Visit Monte Carlo Control?
The article argues that Monte Carlo Tree Search (MCTS) and every‑visit Monte Carlo (MC) control are essentially the same method, differing only in terminology and presentation. It shows that MCTS’s four stages—selection, expansion, simulation, and backup—can be reduced to two core operations: sampling trajectories under the current policy and performing every‑visit MC updates. The note aims to make this equivalence explicit and easier to recognize.
arXiv Machine Learning
Jun 5
Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
arXiv:2606. 05296v1 Announce Type: new Abstract: LLM agents operate in two distinct regimes: open-weight agents amenable to reinforcement learning (RL) and black-box agents whose behaviour must be controlled purely at test time.