arXiv:2607. 08984v1 Announce Type: cross Abstract: AlphaZero has demonstrated that a neural-guided Monte Carlo Tree Search can achieve superhuman performance, but strong play does not necessarily imply perfect play.
The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
arXiv:2608. 15146v1 Announce Type: new Abstract: We revisit Tesauro's TD-Gammon for backgammon money games in the setting of no evaluation-time search.
arXiv:2607. 26946v1 Announce Type: new Abstract: Recent advancements in Computer Go, driven by AlphaZero and MuZero, rely heavily on Monte Carlo Tree Search (MCTS) to correct the errors of the neural network policy.
The paper introduces LUGL (Local Updates, Global Learning), a framework that separates data collection from model fitting, allowing non‑incremental learners such as gradient‑boosted trees (LightGBM) to be used in reinforcement learning for games. LUGL alternates between a local update phase—where agents play self‑play games and store tabular updates—and a global learning phase—where a function approximator is trained on the accumulated table before it is reset. Experiments on both perfect‑information and imperfect‑information games show that LightGBM‑based agents perform competitively or better than neural‑network baselines like DQN and DeepCFR.
arXiv:2606. 23995v1 Announce Type: cross Abstract: Recent work has established that regularized policy gradient methods such as PPO, when used in self-play, can match or exceed specialized game-theoretic algorithms for solving two-player zero-sum imperfect-information games.
The paper introduces Local Updates, Global Learning (LUGL), a framework that separates data collection from model training, allowing non‑incremental learners such as gradient‑boosted trees (LightGBM) to be used in reinforcement learning for games. LUGL alternates between a local phase—where self‑play generates tabular updates—and a global phase—where these updates train a function approximator before resetting the table. Experiments on both perfect‑information and imperfect‑information games show that LightGBM agents perform competitively or better than neural‑network baselines like DQN and DeepCFR.
arXiv:2606. 04860v1 Announce Type: cross Abstract: Finding optimal solution paths for combinatorial puzzles like the Rubik's Cube, sliding tile puzzles, and Lights Out remains a classical challenge in artificial intelligence.
The paper introduces Particle MCTS (PMCTS), a parallelized version of Monte Carlo Tree Search designed for GPU acceleration and batch-parallel neural network evaluations. PMCTS preserves policy improvement guarantees of modern MCTS algorithms while scaling efficiently with parallel compute. Empirical results show that PMCTS consistently outperforms or matches heuristic-based baselines across various MCTS and reinforcement learning domains, including chess, Go, and both discrete and continuous control benchmarks.
arXiv:2305.17327v4 Announce Type: replace Abstract: Imperfect Information Games (IIGs) are used to model games under uncertainty or lack complete information. Counterfactual Regret Minimization (CFR)...
arXiv:2607. 21971v1 Announce Type: new Abstract: Test-time scaling through iterative self-evolution with environment feedback, as demonstrated by AlphaEvolve, shows remarkable performance gains.
SPADE (Self-Play in Adaptive Synthetic Executable Environments) is a reinforcement‑learning framework where a single large language model acts as both an Environment Designer—creating executable, long‑horizon training environments—and a Reasoning Agent—learning to act within those environments. The framework uses a regret signal based on the difference between rewarded performance with and without privileged hints to guide the Designer toward environments that are challenging yet solvable. Experiments show that, when scaled to 30‑billion‑parameter models, SPADE outperforms fixed‑environment baselines by significant margins across math, science, code, and reasoning benchmarks, and improves tool‑use performance on BFCL‑v4 and ACEBench‑Agent. whyItMatters":"By making environment design a learnable component, SPADE enables continuous self‑improvement and demonstrates that adaptive, self‑generated training environments can substantially boost language‑model performance across diverse tasks."