The paper develops a geometric theory of decision boundaries for structured Markov Decision Processes, treating the geometry induced by optimal policies as the key analytical object. It shows that, under structural regularity, this geometry yields the minimal representation needed for policy reconstruction and dictates the statistical and computational complexity of the reconstruction problem. The authors introduce intrinsic notions of boundary and decision complexity, derive information-theoretic measures of decision compression, and provide statistical guarantees for boundary estimation and policy reconstruction from black-box queries, supported by controlled numerical experiments.
We study optimal-policy geometry in structured Markov decision processes. While approximate dynamic programming and reinforcement learning typically approximate high-dimensional value functions, we show that optimal policies induce simpler decision tessellations.
arXiv:2606. 25593v1 Announce Type: new Abstract: We study optimal-policy geometry in structured Markov decision processes.
arXiv:2604.01024v2 Announce Type: replace Abstract: We study model-based learning of finite-window policies in tabular partially observable Markov decision processes (POMDPs). A common approach to le...
arXiv:2403. 19883v2 Announce Type: replace Abstract: Fully-observable non-deterministic (FOND) planning is at the core of artificial intelligence planning with uncertainty.
arXiv:2602. 12963v2 Announce Type: replace Abstract: An important question in the field of AI is the extent to which successful behaviour requires an internal representation of the world.
arXiv:2607. 03385v1 Announce Type: cross Abstract: Policy learning has received substantial attention with the goal of learning policies from observational data for decision-making.
arXiv:2606. 16729v1 Announce Type: new Abstract: While there is an extensive body of work characterizing the sample complexity of discounted cumulative-reward MDPs, finite sample analyses for average-reward MDPs have been limited, and most existing works rely on restrictive assumptions such as ergodicity or access to a generative model.
arXiv:2603. 03480v2 Announce Type: replace Abstract: We study reinforcement learning with delayed state observation, where the agent observes the current state after some random number of time steps.
arXiv:2607. 17823v1 Announce Type: new Abstract: Reinforcement Learning is a cornerstone technique for modern large reasoning models.
arXiv:2603. 08287v2 Announce Type: replace-cross Abstract: We analyze the Bayesian regret of the Gaussian process posterior sampling reinforcement learning (GP-PSRL) algorithm.
arXiv:2609.06484v1 Announce Type: cross Abstract: Planning with a generative model aims to estimate the value of a state using as few simulator calls as possible. SmoothCruiser achieves problem-indep...
arXiv:2505. 12037v2 Announce Type: replace Abstract: Learning the optimal policy for Markov decision process problems (MDPs) from samples is a fundamental problem in online and data-driven decision-making.