arXiv AI
The Surprising Effectiveness of Approximate Value Iteration in Self-Play
Read the original on arXiv AI →The Flow has not summarised this story yet — read it at arXiv AI.
Related stories
arXiv AI
Jul 13
AlphaZero in Sparsely Rewarded Games: Limits and Auxiliary Supervision
arXiv:2607. 08984v1 Announce Type: cross Abstract: AlphaZero has demonstrated that a neural-guided Monte Carlo Tree Search can achieve superhuman performance, but strong play does not necessarily imply perfect play.
arXiv AI
22h ago
Online Robust Reinforcement Learning Through Monte-Carlo Planning
The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
arXiv Machine Learning
Aug 18
PureTD: Reinforcement Learning for Backgammon Money Games with No Evaluation-time Search
arXiv:2608. 15146v1 Announce Type: new Abstract: We revisit Tesauro's TD-Gammon for backgammon money games in the setting of no evaluation-time search.
arXiv AI
Jul 31
Belief-Guided Decision Making with Uncertainty Gating in the Game of Go
arXiv:2607. 26946v1 Announce Type: new Abstract: Recent advancements in Computer Go, driven by AlphaZero and MuZero, rely heavily on Monte Carlo Tree Search (MCTS) to correct the errors of the neural network policy.
arXiv AI
Sep 4
Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners
The paper introduces LUGL (Local Updates, Global Learning), a framework that separates data collection from model fitting, allowing non‑incremental learners such as gradient‑boosted trees (LightGBM) to be used in reinforcement learning for games. LUGL alternates between a local update phase—where agents play self‑play games and store tabular updates—and a global learning phase—where a function approximator is trained on the accumulated table before it is reset. Experiments on both perfect‑information and imperfect‑information games show that LightGBM‑based agents perform competitively or better than neural‑network baselines like DQN and DeepCFR.
arXiv AI
Jun 24
EMAgnet: Parameter-Space EMA Regularization for Policy Gradient Self-Play in Large Games
arXiv:2606. 23995v1 Announce Type: cross Abstract: Recent work has established that regularized policy gradient methods such as PPO, when used in self-play, can match or exceed specialized game-theoretic algorithms for solving two-player zero-sum imperfect-information games.