The paper presents a finite‑model synthesis that integrates operational state abstraction with optimal control within a stability‑evidence‑revision (SER) framework, termed "Poincaré meets Bellman." It distinguishes qualitative dynamics for reusable action‑response structures from dynamic programming that governs acquisition, retention, reuse, merging, and forgetting, and introduces a Bellman recursion over the joint law of hidden state and deployed memory. The authors derive explicit retention rules, demonstrate how factor sharing and informative observations improve identification, and verify coding and retention calculations through finite enumerations.
By Xin Li
arXiv:2604.01024v2 Announce Type: replace
Abstract: We study model-based learning of finite-window policies in tabular partially observable Markov decision processes (POMDPs). A common approach to le...
By Philip Jordan, Maryam Kamgarpour
The paper challenges the common assumption that the successor measure in reinforcement learning is approximately low-rank, showing instead that a low-rank structure emerges in a shifted successor measure that ignores initial transitions. It provides finite-sample guarantees for estimating this low-rank approximation, introduces Type II Poincaré inequalities to bound spectral recoverability, and links the necessary shift to the decay of high-order singular values and local mixing properties. Experiments confirm that shifting the successor measure improves goal-conditioned RL performance.
By Bastien Dubail, Stefan Stojanovic, Alexandre Prouti\`ere
arXiv:2609.39837v1 Announce Type: new
Abstract: Policy mirror descent (PMD) enjoys fast convergence in regularized Markov decision processes (MDPs), but existing guarantees often rely on exact or inc...
By Qipei Chen, Wenye Li, Yule Sun, Ke Wei
arXiv:2601. 18930v4 Announce Type: replace-cross Abstract: We are interested in enabling autonomous agents to learn and reason about systems with hidden states, such as locking mechanisms.
By Seiji Shaw, Travis Manderson, Chad Kessens, Nicholas Roy
The paper introduces Nous, a framework that separates learning, calibration, and revision certification for belief‑based agent memory. It shows that learning and certifying useful memory decisions can require quadratically fewer records than source calibration, and provides finite‑sample certificates for policy improvement without needing to recover source reliability. Experiments on MiniGrid environments demonstrate that the new certificate reliably accepts improvements over incumbents, outperforming earlier methods.
By Pranav Singh