The Flow has not summarised this story yet — read it at arXiv Machine Learning.
arXiv:2502. 08870v2 Announce Type: replace Abstract: We provide an approach for the analysis of randomised exploration algorithms like Thompson sampling that does not rely on forced optimism or posterior inflation.
The paper investigates preference-based bandits where a learner selects pairs of arms and receives binary preference feedback modeled by Bradley–Terry. It introduces the locally sensitive eluder dimension, a new complexity measure for logistic preference feedback, and proposes the GINOP algorithm that uses log-loss confidence sets to balance optimism and exploration. The authors prove a first-order regret bound showing that learning with preference feedback can be as statistically efficient as learning from direct rewards, and they validate their theory with empirical experiments.
arXiv:2610.32910v1 Announce Type: new Abstract: Regularised reinforcement learning (RL) algorithms rely on estimators, like Monte Carlo tree search or one-step look-ahead, to provide policy, value, o...
The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).
arXiv:2607. 29375v1 Announce Type: cross Abstract: Organizations increasingly rely on sequential experimentation to improve decision-making.
The paper introduces the tilted sampling problem, where an online learner aims to sample from a Gibbs distribution defined by an unknown reward function. It defines sampling regret as the temperature‑normalized cumulative KL divergence between the learner’s distribution and the target Gibbs distribution, linking it to entropy‑regularized reward maximization. The authors develop sampling analogues of UCB and Thompson sampling, proving that sampling regret can achieve logarithmic growth in time, unlike standard regret which scales as ×O(√T).