Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding. Tree-based search methods such as Monte Carlo Tree Search (MCTS) remain popular, but their branching structure can require sampling budgets that grow exponentially with lookahead depth in the worst case.
arXiv:2607. 05359v1 Announce Type: new Abstract: Planning under uncertainty in continuous domains is essential for autonomous systems, yet computationally demanding.
The paper introduces Graph-Based Stochastic-Power-UCT (GS-Power-UCT), a Monte‑Carlo graph search algorithm that shares states reached at the same planning depth while maintaining separate values for different depths. It achieves an $O(n^{-1/2})$ convergence rate for the root estimate in finite‑horizon stochastic MDPs, matching tree‑based methods but with better sample reuse. Two full‑state variants, GS-Power-UCT‑F and GS-Power-UCT‑F$^+$, further explore sample sharing and bias control, with GS-Power-UCT‑F$^+$ converging to the optimal infinite‑horizon discounted value when the cross‑depth gap vanishes. Experiments on stochastic planning benchmarks demonstrate improved sample efficiency over existing tree‑based and graph‑based baselines.
Graph-Based Stochastic-Power-UCT (GS-Power-UCT) is a Monte‑Carlo graph search algorithm that shares states reached at the same planning depth while keeping separate values for different depths, thereby reducing duplicate simulations in stochastic MDPs. The method guarantees that, for a fixed horizon, the root estimate converges to the finite‑horizon value at an $O(n^{-1/2})$ rate, matching tree‑based Stochastic‑Power‑UCT but with improved sample reuse. Two full‑state variants—GS‑Power‑UCT‑F and GS‑Power‑UCT‑F$^+$—extend the approach to single‑node per physical state and adaptive horizons, respectively, with the latter converging to the optimal infinite‑horizon discounted value when cross‑depth bias vanishes. Experiments on stochastic planning benchmarks demonstrate that GS‑Power‑UCT outperforms both tree‑based and other graph‑based baselines in sample efficiency.
The paper introduces a robust variant of Monte Carlo Tree Search that addresses ambiguities in transition dynamics and reward distributions, bridging the gap between simulation-based planning and real-world deployment. It incorporates a robust power mean backup operator and exploration bonuses to guarantee finite-sample convergence at every node, achieving an ≠O(n−1/2) convergence rate for root value estimation comparable to standard MCTS. Empirical results demonstrate robust performance in planning tasks even under significant model mismatches.
arXiv:2609.06489v1 Announce Type: cross Abstract: Monte Carlo Tree Search (MCTS) has demonstrated success in online planning for deterministic environments, yet significant challenges remain in adapt...
arXiv:2607. 08894v1 Announce Type: new Abstract: Large Language Model (LLM) agents have shown promise in multi-step planning tasks, but existing approaches like LATS (Language Agent Tree Search) and ReAct rely heavily on LLM inference during planning, leading to high computational costs and stochastic behavior.
GEM-MPC is a reinforcement learning method that blends MPPI planning with policy learning to balance exploration and exploitation in high-dimensional continuous control tasks. It trains a policy to clone the planner while also maintaining a KL-regularized policy that explores around the planner’s suggestions, thereby improving the synergy between planning and learning. The approach introduces Gated Prior Distillation, which selectively updates policies from stored planning distributions only when they offer better targets, reducing the influence of stale data without costly reanalysis. Across continuous-control benchmarks, GEM-MPC outperforms existing planning-based baselines while using lower computational budgets.
The paper introduces Graph-Guided Quasimetric Dense Reward (G2QDR), a framework that learns a state connectivity model to predict pairwise connectivity strengths in asymmetric environments. These strengths are converted into scalar auxiliary dense rewards, offering continuous guidance across hierarchical levels. G2QDR can be integrated into any existing Goal-Conditioned Hierarchical Reinforcement Learning architecture and shows empirical performance improvements in sparse reward settings with modest computational cost.
arXiv:2607. 02915v1 Announce Type: cross Abstract: In many scientific and engineering domains, maximizing discovery within a limited sampling budget demands strategic, observation-guided exploration.
arXiv:2605. 08732v2 Announce Type: replace-cross Abstract: Modern vision-based world models can represent observations as compact yet expressive latent manifolds, but fast goal-oriented planning in these spaces remains challenging.
Large Language Model (LLM) agents have shown promise in multi-step planning tasks, but existing approaches like LATS (Language Agent Tree Search) and ReAct rely heavily on LLM inference during planning, leading to high computational costs and stochastic behavior. We present \textbf{GATS} (Graph-Augmented Tree Search), a planning framework that combines systematic UCB1-based tree search with a layered world model to eliminate LLM calls during inference while achieving superior planning performance.
The paper introduces Sampling-Guided Policy Search (SGPS), a method that combines sampling-based model‑predictive control with first‑order policy gradients to accelerate visual policy learning for locomotion and manipulation tasks. SGPS starts with behavior cloning from sampled actions and then alternates between sampling‑based refinement and short‑horizon policy updates under varied initial states and dynamics. The approach is demonstrated on simulated Unitree Go2 and G1 robots, learning tasks such as obstacle traversal and bimanual carrying, and the distilled policies transfer zero‑shot to a real Go2 robot using onboard depth perception.