arXiv Machine Learning
Jun 25
Low-Complexity Policy Tessellations in Structured Markov Decision Processes
arXiv:2606. 25593v1 Announce Type: new Abstract: We study optimal-policy geometry in structured Markov decision processes.
Low-Complexity Policy Tessellations in Structured Markov Decision Processes
We study optimal-policy geometry in structured Markov decision processes. While approximate dynamic programming and reinforcement learning typically approximate high-dimensional value functions, we show that optimal policies induce simpler decision tessellations.
Related stories
arXiv AI
Jun 10
Bellman-Taylor Score Decoding for Markov Decision Processes with State-Dependent Feasible Action Sets
arXiv:2606. 10979v1 Announce Type: new Abstract: Many Markov decision processes (MDPs) in operations research have feasible actions that are state dependent and defined implicitly by various operational constraints.
arXiv Machine Learning
Sep 17
A Geometric Theory of Decision Boundaries in Structured Markov Decision Processes
The paper develops a geometric theory of decision boundaries for structured Markov Decision Processes, treating the geometry induced by optimal policies as the key analytical object. It shows that, under structural regularity, this geometry yields the minimal representation needed for policy reconstruction and dictates the statistical and computational complexity of the reconstruction problem. The authors introduce intrinsic notions of boundary and decision complexity, derive information-theoretic measures of decision compression, and provide statistical guarantees for boundary estimation and policy reconstruction from black-box queries, supported by controlled numerical experiments.
arXiv Machine Learning
Jun 9
Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence
arXiv:2601. 18840v4 Announce Type: replace Abstract: Markov decision problems are most commonly solved via dynamic programming.
arXiv Machine Learning
1d ago
Towards Optimal Policy Improvement
The paper introduces a framework for optimal policy improvement in reinforcement learning, defining it as the best single update under given constraints. It shows that restricting improvement to a subset of states is equivalent to solving an induced Markov Decision Process, linking planning with explicit or implicit models to optimal policy improvement. The authors develop a novel operator for greedification under approximate evaluation, demonstrating empirical gains across several RL algorithms and settings.
arXiv Machine Learning
Jun 17
Performance-Driven Environment Abstraction with Multi-Timescale Learning
arXiv:2606. 17377v1 Announce Type: new Abstract: We study performance-driven environment abstraction for decision-making in large Markov decision processes.
arXiv Machine Learning
Jul 28
Online Policy Evaluation for MDPs with Dynamic UBSR Measures
arXiv:2607. 23030v1 Announce Type: new Abstract: Developing efficient function-approximation methods for policy evaluation is a fundamental challenge in risk-aware reinforcement learning.
arXiv Machine Learning
Jul 1
End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions
arXiv:2603. 23461v2 Announce Type: replace Abstract: We study reinforcement learning (RL) with linear function approximation in Markov Decision Processes (MDPs) satisfying \emph{linear Bellman completeness} -- a fundamental setting where the Bellman backup of any linear value function remains linear.
arXiv Machine Learning
Jun 11
Impact of Connectivity on Laplacian Representations in Reinforcement Learning
arXiv:2603. 08558v3 Announce Type: replace Abstract: Learning compact state representations in Markov Decision Processes (MDPs) has proven crucial for addressing the curse of dimensionality in large-scale reinforcement learning (RL) problems.
arXiv AI
Jun 10
Model-Based Reinforcement Learning in Discrete-Action Non-Markovian Reward Decision Processes
arXiv:2512. 14617v2 Announce Type: replace-cross Abstract: Many practical decision-making problems involve tasks whose success depends on the entire system history, rather than on achieving a state with desired properties.
arXiv AI
Sep 21
Reinforcement Learning under External Influence: Guarantees, Algorithms, and Sample Complexity
The paper investigates reinforcement learning in Markov decision processes whose dynamics are perturbed by non‑Markovian external events. It identifies conditions that make the problem tractable by limiting consideration to a finite history of events, and proposes a policy iteration algorithm that learns state‑dependent policies conditioned on this history. The authors provide theoretical guarantees for policy improvement, analyze sample complexity for least‑squares evaluation and improvement, and extend their results to discrete‑time Hawkes processes with Gaussian marks, validating their approach with experiments in control environments.
Hugging Face Trending Papers
Sep 23
Limiting-Kernel Q($λ$): Bridging Short and Long Horizons
In value-based reinforcement learning, improving the accuracy of policy evaluation has been shown to improve downstream policy optimization performance. The widely adopted family of approximations rel...