The paper introduces a new approach to safety in contextual bandits with continuous actions by enforcing high‑probability constraints on the realized cost rather than on its expectation. It proposes the High‑Probability Constrained UCB algorithm, which balances optimistic reward exploration with pessimistic safety estimation. The authors provide theoretical regret guarantees for linear models and extend the analysis to general function classes, demonstrating experimentally that realized‑cost constraints significantly reduce safety violations compared to expected‑cost baselines.
The paper introduces a new approach to safety in contextual bandits with continuous actions, focusing on high‑probability constraints on the realized cost rather than expected cost. It presents the High‑Probability Constrained UCB algorithm, which balances reward exploration with conservative safety estimation, and provides theoretical regret guarantees for linear models and extensions to general function classes. Experiments demonstrate that this realized‑cost safety framework significantly reduces safety violations compared to expected‑cost constrained methods.
By Spyros Dragazis, Aldo Pacchiano
arXiv:2307. 03587v4 Announce Type: replace Abstract: In non-stationary linear contextual bandits, existing efficient algorithms typically rely on the Weighted Regularized Least-Squares (WRLS) estimator.
By Nicklas Werge, Yi-Shan Wu, Abdullah Akg\"ul, Melih Kandemir
arXiv:2512. 00517v3 Announce Type: replace-cross Abstract: Sequential optimization of black-box functions from noisy evaluations has been widely studied, with Gaussian Process bandit algorithms such as GP-UCB guaranteeing no-regret in stationary settings.
By Eliabelle Mauduit, Elo\"ise Berthier, Andrea Simonetto
The paper introduces a hierarchical Bayesian framework for multi-task contextual bandits, capturing both heterogeneity and correlations among bandit instances. It proposes an empirical Bayesian method to estimate the covariance matrix of the prior, leading to two new algorithms—ebmTS and ebmUCB—that incorporate this prior into decision making. The authors provide frequentist regret bounds and show through synthetic and real-world experiments that these algorithms outperform existing methods, especially in complex settings.
By Xia Jiang, Rong J. B. Zhu
The paper analyzes Bayesian linear bandits with isotropic Gaussian parameters, independent Gaussian arms, and Gaussian reward noise when the time horizon scales with the dimension. It derives explicit limits for the normalized posterior uncertainty and parameter overlaps, yielding exact regret curves for several policies—including Thompson sampling, posterior‑mean greedy selection, and scaled‑covariance variants. The results show that posterior‑mean greedy selection achieves the optimal Bayes regret, while Thompson sampling incurs a strictly larger leading regret whose ratio to greedy lies between one and two, approaching two for long horizons.
By Prakhar Singhvi (Abstract Math Institute), Yi Zou (Abstract Math Institute), Abhishek Bhattacharjee (Abstract Math Institute)
arXiv:2502. 01226v4 Announce Type: replace Abstract: Gaussian process (GP) bandits provide a powerful framework for performing blackbox optimization of unknown functions.
By Jack Sandberg, Morteza Haghir Chehreghani
arXiv:2004. 06321v2 Announce Type: replace Abstract: We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end.
By Yanjun Han, Zhengqing Zhou, Zihao Hu, Jose Blanchet, Peter W. Glynn, Yinyu Ye, Zhengyuan Zhou
The paper presents an improved analysis of non‑consecutive gradient variation in Bandit Convex Optimization (BCO) with two‑point feedback, leading to better dimension dependence for both convex and strongly convex functions compared to prior work. It also derives new problem‑dependent guarantees such as gradient‑variance and small‑loss regret bounds, extends the technique to one‑point bandit linear optimization over hyper‑rectangular domains, and establishes the first gradient‑variation dynamic and universal regret bounds for two‑point BCO.
By Hang Yu, Yu-Hu Yan, Peng Zhao
arXiv:2607. 28408v1 Announce Type: new Abstract: This thesis studies policy learning in interactive systems where an agent observes a context, selects an action from a very large set, and receives partial feedback.
By Imad Aouali
arXiv:2610.08677v1 Announce Type: new
Abstract: Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly...
By Nicol\`o Felicioni, Michael Benigni, Maurizio Ferrari Dacrema, Paolo Cremonesi
Off-policy evaluation (OPE) for contextual bandit policies becomes challenging when action-level importance weighting incurs excessive variance. Doubly robust (DR) estimation remains unbiased under co...