← Back to all news
arXiv AI By Weixin Wang, Haoyang Zheng, Guang Lin, Wei Deng, Pan Xu

Rethinking Langevin Thompson Sampling from A Stochastic Approximation Perspective

Read the original on arXiv AI →

The paper introduces TS-SA, a variant of Langevin-based Thompson Sampling that integrates stochastic approximation into the bandit framework. TS-SA performs a single Langevin step per new reward and updates a persistent Markov chain for each arm, avoiding repeated sampling of past data. This approach yields a near‑optimal worst‑case regret bound of ≥O(√{KT}) under log‑concavity and smoothness, and empirical results show strong performance against standard baselines.

Machine-generated by The Flow from the publisher's headline and feed description — not written or checked by a human. The full article lives at arXiv AI.

Related stories

arXiv Machine Learning
Oct 15

High-Probability Regret Bounds for Feel-Good Thompson Sampling

The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).

By Xuheng Li, Qingyue Zhao, Quanquan Gu
llms
More like this →
arXiv Machine Learning
20h ago

The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits

The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.

By Sharan Sahu, Martin T. Wells
reinforcement-learning
More like this →