The paper investigates reinforcement learning with multi‑step transition look‑ahead, where an agent can foresee the states resulting from any sequence of λ actions before choosing its next move. It proves that exact planning remains NP‑hard for every fixed rational discount factor γ in (0,1), and introduces a randomized polynomial‑time approximation scheme that works for any fixed look‑ahead depth. Extending this to unknown transitions and stochastic rewards, the authors develop an algorithm with cumulative regret matching classical tabular discounted RL up to logarithmic factors, showing that efficient near‑optimal planning and learning are achievable despite the NP‑hardness of exact planning.
By Corentin Pla, Hugo Richard, Marc Abeille, Vianney Perchet
arXiv:2603. 23461v2 Announce Type: replace Abstract: We study reinforcement learning (RL) with linear function approximation in Markov Decision Processes (MDPs) satisfying \emph{linear Bellman completeness} -- a fundamental setting where the Bellman backup of any linear value function remains linear.
By Zakaria Mhammedi, Alexander Rakhlin, Nneka Okolo
arXiv:2610.20486v1 Announce Type: new
Abstract: In reinforcement learning, an agent must explore an unknown environment to learn how to make good decisions. Unlike methods that can freely sample any...
By Tongxi Wang
arXiv:2601.10418v2 Announce Type: replace-cross
Abstract: We study tabular finite-horizon reinforcement learning problems with multiple steps of lookahead information. Before acting, the learner obse...
By Nadav Merlis
arXiv:2607. 03385v1 Announce Type: cross Abstract: Policy learning has received substantial attention with the goal of learning policies from observational data for decision-making.
By Hamsa Bastani, Osbert Bastani, Shihan Chen
arXiv:2606. 00367v1 Announce Type: cross Abstract: Reinforcement learning problems typically define the goal as maximizing the expected value of a scalar reward function.
By Jonathan Cola\c{c}o Carr, Prakash Panangaden, Doina Precup, Benjamin Van Roy
The paper introduces Quasar, a model‑free Q‑learning algorithm that guarantees asymptotic convergence for reachability objectives in Markov Decision Processes that are free of non‑terminal maximal end components (MECs). Unlike prior model‑based methods, Quasar does not estimate transition probabilities, reducing memory usage from O(|S|²|A|) to O(|S||A|). Experiments on the Quantitative Verification Benchmark Set show that Quasar converges to optimal policies with far fewer samples than existing state‑of‑the‑art model‑based approaches.
By Lu-Chin Chang, Suguman Bansal
arXiv:2512. 06244v2 Announce Type: replace Abstract: The exploration-exploitation dilemma in reinforcement learning (RL) is a fundamental challenge to efficient RL algorithms.
By Caleb Ju, Guanghui Lan
arXiv:2604. 12474v3 Announce Type: replace-cross Abstract: In many robotic tasks, agents must traverse a sequence of spatial regions to complete a mission.
By Lidor Erez, Shahaf S. Shperberg, Ayal Taitler
arXiv:2610.28360v1 Announce Type: cross
Abstract: In reinforcement learning, one presumes the action set to be known ahead of time, and, thus, one must know if any state-action pairs are not permissi...
By Sebastian Reboul, Lorenzo Croissant, H\'el\`ene Halconruy
arXiv:2607. 17823v1 Announce Type: new Abstract: Reinforcement Learning is a cornerstone technique for modern large reasoning models.
By Riccardo Poiani, Martino Bernasconi, Andrea Celli
arXiv:2602. 17315v3 Announce Type: replace-cross Abstract: We introduce Flickering Multi-Armed Bandits (FMAB) to model sequential decision-making in environments with changing action availability, where accessibility of the next action is restricted to a subset dependent on the agent's current choice.
By Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni, Lijun Chen