arXiv Machine Learning
Thompson Sampling for Infinite-Horizon Discounted Decision Processes
Read the original on arXiv Machine Learning →The Flow has not summarised this story yet — read it at arXiv Machine Learning.
Related stories
arXiv Machine Learning
Jun 24
Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spaces
arXiv:2603. 08287v2 Announce Type: replace-cross Abstract: We analyze the Bayesian regret of the Gaussian process posterior sampling reinforcement learning (GP-PSRL) algorithm.
arXiv Machine Learning
Jun 11
On Regret Bounds of Thompson Sampling for Bayesian Optimization
arXiv:2603. 09276v2 Announce Type: replace-cross Abstract: We study a widely used Bayesian optimization method, Gaussian process Thompson sampling (GP-TS), under the assumption that the objective function is a sample path from a GP.
arXiv Machine Learning
Jul 21
Dynamic Decision-Making under Model Misspecification: A Stochastic Stability Approach
arXiv:2602. 17086v2 Announce Type: replace-cross Abstract: Dynamic decision-making under model uncertainty is central to many economic environments, yet existing bandit and reinforcement learning algorithms rely on the assumption of correct model specification.
arXiv Machine Learning
Sep 25
Exact Bayes Regret and Asymptotic Optimality in High-Dimensional Gaussian Bandits
The paper analyzes Bayesian linear bandits with isotropic Gaussian parameters, independent Gaussian arms, and Gaussian reward noise when the time horizon scales with the dimension. It derives explicit limits for the normalized posterior uncertainty and parameter overlaps, yielding exact regret curves for several policies—including Thompson sampling, posterior‑mean greedy selection, and scaled‑covariance variants. The results show that posterior‑mean greedy selection achieves the optimal Bayes regret, while Thompson sampling incurs a strictly larger leading regret whose ratio to greedy lies between one and two, approaching two for long horizons.
arXiv Machine Learning
Jul 31
On-Policy and Off-Policy Learning for Large Action Spaces
arXiv:2607. 28408v1 Announce Type: new Abstract: This thesis studies policy learning in interactive systems where an agent observes a context, selects an action from a very large set, and receives partial feedback.
arXiv Machine Learning
Aug 4
Conformal bandits: bringing statistical validity and reward efficiency under weak arm separability
arXiv:2512. 09850v2 Announce Type: replace Abstract: We introduce Conformal Bandits, a novel framework integrating Conformal Prediction (CP) into bandit problems, a classic paradigm for sequential decision-making under uncertainty.