arXiv:2608. 03353v1 Announce Type: cross Abstract: For a Markov kernel $T$ with an invariant probability measure $\pi$, we give a self-contained proof of the Markov chain convergence theorem via a criterion called asymptotic equivalence with the target.
arXiv:2506.19186v2 Announce Type: replace-cross Abstract: We study the importance-tempered random-walk Metropolis (RWM) algorithm on $\mathbb{R}^d$, which targets the tempered density $\pi_\beta(x) \...
arXiv:2609. 38880v2 Announce Type: replace-cross Abstract: We study the last iterate of standard importance-weighted temporal-difference (TD) learning from a single trajectory of a finite Markov decision process.
arXiv:2608.25279v1 Announce Type: cross Abstract: The OBABO and BAOAB schemes and the other standard Strang splittings of kinetic (underdamped) Langevin dynamics are widely used Markov chain Monte Ca...
arXiv:2609.38880v1 Announce Type: new Abstract: We study the last iterate of standard tabular temporal-difference (TD) learning from a single trajectory of a finite Markov reward process. For discoun...
arXiv:2605.06866v3 Announce Type: replace Abstract: We study finite-iteration behavior of asynchronous categorical distributional temporal-difference methods, covering scalar categorical TD (CTD) in...
While entropy regularization is widely used to stabilize and accelerate Natural Policy Gradient methods, its ability to yield faster convergence rates for the unregularized objective remains underexplored. Existing analyses often rely on double-loop architectures and invoke a linear entropy penalty.
arXiv:2608. 26515v1 Announce Type: cross Abstract: We study online prediction for a specific finite-alphabet, exogenously driven source with infinite input memory.
arXiv:2602. 13906v2 Announce Type: replace-cross Abstract: Stochastic approximation (SA) is a method for finding the root of an operator perturbed by noise.
arXiv:2608. 19587v1 Announce Type: new Abstract: While entropy regularization is widely used to stabilize and accelerate Natural Policy Gradient methods, its ability to yield faster convergence rates for the unregularized objective remains underexplored.
arXiv:2602.13960v2 Announce Type: replace Abstract: Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency, yet the distribution of the iterates is ty...
We study the last iterate of standard tabular temporal-difference (TD) learning from a single trajectory of a finite Markov reward process. For discount factor $γ$, write $H=(1-γ)^{-1}$, and let $μ_{\...
arXiv:2104. 11547v5 Announce Type: replace-cross Abstract: Statistical models contain variables that are not random: parameters, treatments, environments, design points.