The paper investigates stochastic contextual bandits in an agnostic setting, aiming to compete with the best policy in a given class without assuming realizability or specific loss/reward models. It introduces an algorithm that updates the policy each round by minimizing a pessimistic objective— a clipped inverse‑propensity estimate of the policy value plus a variance penalty— and proves the first fast regret rates relative to the best‑in‑class policy. By exploiting entropy assumptions on the policy class and a H"olderian error‑bound condition, the authors achieve fast best‑in‑class regret rates, including polylogarithmic rates in the parametric case, using a sequential self‑normalized maximal inequality for bounded martingale empirical processes to derive uniform variance‑adaptive confidence bounds and ensure pessimism under adaptive data collection.
By Samuel Girard, Aurelien Bibaut, Arthur Gretton, Nathan Kallus, Houssam Zenati
The paper presents empirical concentration inequalities for stationary chains with infinite memory, extending previous results that were limited to Markov chains. It establishes a bounded‑differences inequality under a summable Wasserstein contraction condition and shows how to estimate the necessary dependence coefficients from a single trajectory, using transition counts for finite state spaces or kernel density derivatives for continuous spaces. The authors provide general error bounds that reflect the memory tail and demonstrate that, after applying a fixed safety factor, the resulting empirical bounds are asymptotically as tight as the theoretical ones, while remaining fully adaptive without requiring knowledge of the underlying constants.
By Pierre Alquier, Paul Doukhan
We prove that $ρ\text{-}\mathrm{NPTS}_{\mathrm{SG}}$, an anchor-free nonparametric Thompson Sampling algorithm for risk-averse bandits, achieves regret matching the instance-dependent lower bound to leading order in $\log n$, establishing it as asymptotically optimal for any continuous risk functional $ρ$ (CVaR, mean-variance, Sharpe ratio, distortion risk measures, and more) on the class of distributions with bounded density and sub-Gaussian tails, including Gaussian arms. Both this result and its bounded-support counterpart require only continuity of $ρ$: strictly weaker than the dominance condition of prior parametric Thompson Sampling results, and strictly weaker than the Lipschitz condition of UCB-type algorithms, yielding the first instance-optimal guarantees for non-Lipschitz functionals such as the Sharpe ratio without parametric reward assumptions.
arXiv:2605. 09454v2 Announce Type: replace-cross Abstract: We study the $\textit{single-index bandit}$ problem, where rewards depend on an unknown one-dimensional projection of high-dimensional contexts through an unknown reward function.
By Devdan Dey, Sujoy Bhore, Avishek Ghosh
arXiv:2606. 09191v1 Announce Type: new Abstract: We prove that $\rho\text{-}\mathrm{NPTS}_{\mathrm{SG}}$, an anchor-free nonparametric Thompson Sampling algorithm for risk-averse bandits, achieves regret matching the instance-dependent lower bound to leading order in $\log n$, establishing it as asymptotically optimal for any continuous risk functional $\rho$ (CVaR, mean-variance, Sharpe ratio, distortion risk measures, and more) on the class of distributions with bounded density and sub-Gaussian tails, including Gaussian arms.
By Joel Q. L. Chang
arXiv:2606. 27448v1 Announce Type: new Abstract: This paper studies the problem of regret minimization in Markovian bandits with \emph{non-observable states} and possibly \emph{constrained} decision epochs.
By Thomas Hira, Victor Boone, Urtzi Ayesta, Ina Maria Verloop
The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).
By Xuheng Li, Qingyue Zhao, Quanquan Gu
The paper investigates best‑policy identification in finite‑horizon, risk‑sensitive reinforcement learning using the entropic risk measure. It identifies a gap between known lower bounds ≥ η(e^{|eta|H}) and upper bounds ≤ O(e^{2|eta|H}) for sample complexity, attributing the excess factor to loose concentration bounds for exponential utilities. By employing a forward‑model algorithm with KL‑based exploration bonuses and a novel stopping rule, the authors achieve a sample complexity that matches the lower bound, closing the previously open exponential gap.
By Amer Essakine, Claire Vernade
arXiv:2602. 09300v2 Announce Type: replace Abstract: We consider the policy evaluation and control in a finite horizon reinforcement learning (RL) setting under an expectile-based objective.
By Shrey Rakeshkumar Patel, Sumedh Gupte, Soumen Pachal, Prashanth L. A., Sanjay P. Bhat
arXiv:2606. 09668v1 Announce Type: new Abstract: Contextual queueing bandits provide a framework for learning to schedule heterogeneous jobs under unknown context-dependent service rates.
By Seoungbin Bae, Dabeen Lee
arXiv:2607. 23679v1 Announce Type: new Abstract: Recent years have witnessed increasing interests in tackling heteroscedastic noise in bandits and reinforcement learning.
By Heyang Zhao, Tianyuan Jin, Weixin Wang, Vincent Y. F. Tan, Pan Xu, Quanquan Gu
The paper extends the idea that contexts are cheap for linear bandits from i.i.d. settings to Markovian context processes. By assuming uniform geometric ergodicity, the authors construct a stationary surrogate action set and use a delayed‑update scheme to mitigate bias from nonstationary conditional context distributions. They provide a phased algorithm for unknown stationary distributions and achieve high‑probability regret bounds comparable to standard linear bandit oracles in fast‑mixing regimes, with empirical validation showing gains over LinUCB.
By Kaan Buyukkalayci, Osama Hanna, Christina Fragouli