arXiv Machine Learning

When and why randomised exploration works (in linear bandits)

arXiv:2502. 08870v2 Announce Type: replace Abstract: We provide an approach for the analysis of randomised exploration algorithms like Thompson sampling that does not rely on forced optimism or posterior inflation.

arXiv Machine Learning
Jun 30

Randomized Exploration for Linear Bandits via Absolute Perturbations

arXiv:2606. 28616v1 Announce Type: new Abstract: In stochastic linear bandits, the canonical Upper Confidence Bound (UCB) algorithm admits a simple frequentist regret analysis but can be computationally demanding, while Thompson Sampling (TS) is computationally attractive yet typically harder to analyze due to its non-optimistic nature.

By Toshinori Kitamura, Shuai Liu, Csaba Szepesv\'ari
arXiv Machine Learning
Jun 5

Exploration via linearly perturbed loss minimisation

arXiv:2311. 07565v3 Announce Type: replace Abstract: We introduce exploration via linear loss perturbations (EVILL), a randomised exploration method for structured stochastic bandit problems that works by solving for the minimiser of a linearly perturbed regularised negative log-likelihood function.

By David Janz, Shuai Liu, Alex Ayoub, Csaba Szepesv\'ari
arXiv Machine Learning
Jul 7

Prior Diffusiveness and Regret in the Linear-Gaussian Bandit

arXiv:2601. 02022v2 Announce Type: replace Abstract: We prove that Thompson sampling exhibits $\tilde{O}(\sigma d \sqrt{T} + d r \sqrt{\mathrm{Tr}(\Sigma_0)})$ Bayesian regret in the linear-Gaussian bandit with a $\mathcal{N}(\mu_0, \Sigma_0)$ prior distribution on the coefficients, where $d$ is the dimension, $T$ is the time horizon, $r$ is the maximum $\ell_2$ norm of the actions, and $\sigma^2$ is the noise variance.

By Yifan Zhu, John C. Duchi, Benjamin Van Roy
arXiv Machine Learning
Sep 10

Improved Dimension Dependence for Bandit Convex Optimization with Gradient Variations

The paper presents an improved analysis of non‑consecutive gradient variation in Bandit Convex Optimization (BCO) with two‑point feedback, leading to better dimension dependence for both convex and strongly convex functions compared to prior work. It also derives new problem‑dependent guarantees such as gradient‑variance and small‑loss regret bounds, extends the technique to one‑point bandit linear optimization over hyper‑rectangular domains, and establishes the first gradient‑variation dynamic and universal regret bounds for two‑point BCO.

By Hang Yu, Yu-Hu Yan, Peng Zhao