arXiv Machine Learning

High-dimensional Linear Bandits with Knapsacks

The paper studies high‑dimensional linear contextual bandits with knapsack constraints (CBwK), aiming to exploit sparsity for tighter regret bounds. It introduces an online hard‑thresholding estimator integrated into a primal‑dual framework, achieving sub‑linear regret that grows only logarithmically with the feature dimension. Under either a diverse‑covariate or margin condition, the regret improves to τ‑dependent rates, and when both hold simultaneously, a dual resolving scheme yields an even tighter bound. The approach also recovers optimal rates for high‑dimensional contextual bandits without knapsacks, and experiments demonstrate its practical effectiveness.

arXiv Machine Learning
Aug 18

Sequential Batch Learning in Finite-Action Linear Contextual Bandits

arXiv:2004. 06321v2 Announce Type: replace Abstract: We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end.

By Yanjun Han, Zhengqing Zhou, Zihao Hu, Jose Blanchet, Peter W. Glynn, Yinyu Ye, Zhengyuan Zhou
arXiv Machine Learning
Oct 12

Gaussian-Width Bounds for Offline-to-Online Linear Bandits

The paper investigates how offline data can improve online learning in stochastic linear bandits, focusing on the case where only a subset of feature directions is poorly covered. It shows that previous regret bounds of ≥O(√{dkT}) stem from loose reward prediction error control and derives tighter bounds using concentration of Lipschitz functions of Gaussian vectors. The authors propose a phased elimination algorithm achieving regret ≥O(k√T) and establish matching lower bounds for both partial and uniform coverage, while also addressing pure exploration with offline data and providing matching bounds for simple regret.

By Kushagra Chandak, Volodymyr Tkachuk, Xiaoqi Tan, Csaba Szepesv\'ari