The article argues that Monte Carlo Tree Search (MCTS) and every‑visit Monte Carlo (MC) control are essentially the same method, differing only in terminology and presentation. It shows that MCTS’s four stages—selection, expansion, simulation, and backup—can be reduced to two core operations: sampling trajectories under the current policy and performing every‑visit MC updates. The note aims to make this equivalence explicit and easier to recognize.
By Xianyi Wu
arXiv:2608. 02993v1 Announce Type: new Abstract: (Flat) Reinforcement Learning (RL) agents face significant challenges in environments with sparse rewards that require long-horizon reasoning.
By Subrat Prasad Panda, Blaise Genest, Arvind Easwaran
arXiv:2505. 13372v2 Announce Type: replace Abstract: Recent work investigated the use of Reinforcement Learning (RL) for the synthesis of heuristic guidance to improve the performance of temporal planners when a domain is fixed and a set of training problems (not plans) is given.
By Irene Brugnara, Alessandro Valentini, Andrea Micheli
Reinforced Planning with Latent World Models (RP1) is a novel method that learns to evaluate imagined outcomes via a critic and to improve multi‑step plans through an optimizer trained offline on world‑model roll‑outs. It is the first approach to fully learn plan improvement and can be attached to any pretrained latent world model. In experiments on visual navigation, arm reaching, and robotic manipulation, RP1 outperforms hand‑designed search algorithms, achieving near‑perfect success while using far fewer roll‑outs and running up to 67× faster than the strongest alternative.
By Armin Sommer, Jannik Schilling
arXiv:2605. 05481v2 Announce Type: replace Abstract: We revisit a classic "chicken-and-egg" problem in reinforcement learning: to safely improve a policy, the value function must be accurate on the state-visitation distribution of the updated policy.
By Dillon Sandhu, Ronald Parr
arXiv:2406.03678v2 Announce Type: replace
Abstract: On-policy reinforcement learning methods, like Trust Region Policy Optimization (TRPO) and Proximal Policy Optimization (PPO), often demand extensi...
By Yaozhong Gan, Renye Yan, Zhe Wu, Junliang Xing
GEM-MPC is a reinforcement learning method that blends MPPI planning with policy learning to balance exploration and exploitation in high-dimensional continuous control tasks. It trains a policy to clone the planner while also maintaining a KL-regularized policy that explores around the planner’s suggestions, thereby improving the synergy between planning and learning. The approach introduces Gated Prior Distillation, which selectively updates policies from stored planning distributions only when they offer better targets, reducing the influence of stale data without costly reanalysis. Across continuous-control benchmarks, GEM-MPC outperforms existing planning-based baselines while using lower computational budgets.
By Alvaro Serra-Gomez, Thomas Moerland
arXiv:2606. 15197v1 Announce Type: cross Abstract: Optimization modeling is inherently hierarchical, requiring a precise sequence of symbolic commitments.
By Jiajun Li, Yu Ding, Shisi Guan, Ran Hou, Wanyuan Wang
arXiv:2608.30162v1 Announce Type: new
Abstract: We present a reinforcement-learning agent that solves symbolic equations step by step, covering both nonlinear closed equations (radicals, exponentials...
By Kevin P O Keeffe
arXiv:2608. 09805v1 Announce Type: cross Abstract: Exploration has been a focus of reinforcement learning research for a long time.
By Vatsal Venkatkrishna, Nico Daheim, Iryna Gurevych
The paper presents an approach to automated theorem proving by framing the construction of clausal connection tableaux as a policy in a transition system. It introduces a graph neural network that scores proof edits based on structure, trained via imitation learning from existing proofs. Experiments on M2k, MPTP2078-bushy, and TPTP v9.2.1 show that the learned policies solve up to 46% more problems than leanCoP and find proofs in an order of magnitude fewer steps.
By Fredrik R{\o}mming, Mantas Bak\v{s}ys, Martin S. Fixman, Sean B. Holden
Reinforced Planning with Latent World Models introduces RP1, a neural planner that learns to evaluate imagined outcomes via a critic and improve multi‑step plans through an optimizer trained offline on world‑model roll‑outs. Unlike existing planners that are hand‑designed or only inform policies, RP1 fully learns to refine plans and can be attached to any pretrained latent world model. In experiments on visual navigation, arm reaching, and robotic manipulation, RP1 outperforms hand‑designed search algorithms, achieving near‑perfect success while using 1,000× fewer roll‑outs and up to 67× faster inference.