The paper explores the use of set membership estimation (SME) for uncertainty quantification in linear contextual bandits, contrasting it with the more common confidence‑interval approach. An SME‑based optimism‑in‑the‑face‑of‑uncertainty (OFU) algorithm is proposed, and a sublinear regret bound is proven under suitable assumptions. Simulations show that SME‑OFU performs favorably compared to existing methods.
arXiv:2606. 20022v1 Announce Type: cross Abstract: This paper considers stochastic linear contextual bandits (SLCB) with bounded reward noise.
arXiv:2610.28136v1 Announce Type: cross Abstract: Gaussian process upper confidence bound (GP-UCB) has been widely studied, though its known regret upper bound is suboptimal. To obtain near-optimal r...
arXiv:2604. 08149v2 Announce Type: replace Abstract: We consider a linear contextual bandit model where contexts and rewards are governed by a finite hidden Markov chain.
arXiv:2604. 05859v2 Announce Type: replace Abstract: We study Contextual Multi-Armed Bandits (CMABs) for non-episodic decision-making problems where the context includes both textual and numerical information (e.
arXiv:2004. 06321v2 Announce Type: replace Abstract: We study the sequential batch learning problem in linear contextual bandits with finite action sets, where the decision maker is constrained to split incoming individuals into (at most) a fixed number of batches and can only observe outcomes for the individuals within a batch at the batch's end.
arXiv:2508. 02812v3 Announce Type: replace Abstract: Causal graphical models can encode large amounts structural knowledge, both from the background knowledge of domain experts and the structural knowledge discovered from randomized experiments or observational data.
arXiv:2409. 18909v2 Announce Type: replace Abstract: Motivated by real-world applications that necessitate responsible experimentation, we introduce the problem of best arm identification (BAI) with minimal regret.
The paper investigates stochastic contextual bandits in an agnostic setting, aiming to compete with the best policy in a given class without assuming realizability or specific loss/reward models. It introduces an algorithm that updates the policy each round by minimizing a pessimistic objective— a clipped inverse‑propensity estimate of the policy value plus a variance penalty— and proves the first fast regret rates relative to the best‑in‑class policy. By exploiting entropy assumptions on the policy class and a H"olderian error‑bound condition, the authors achieve fast best‑in‑class regret rates, including polylogarithmic rates in the parametric case, using a sequential self‑normalized maximal inequality for bounded martingale empirical processes to derive uniform variance‑adaptive confidence bounds and ensure pessimism under adaptive data collection.
The paper introduces a new approach to safety in contextual bandits with continuous actions by enforcing high‑probability constraints on the realized cost rather than on its expectation. It proposes the High‑Probability Constrained UCB algorithm, which balances optimistic reward exploration with pessimistic safety estimation. The authors provide theoretical regret guarantees for linear models and extend the analysis to general function classes, demonstrating experimentally that realized‑cost constraints significantly reduce safety violations compared to expected‑cost baselines.
The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.
arXiv:2307. 03587v4 Announce Type: replace Abstract: In non-stationary linear contextual bandits, existing efficient algorithms typically rely on the Weighted Regularized Least-Squares (WRLS) estimator.
arXiv:2502. 13467v2 Announce Type: replace Abstract: The $K$-Max combinatorial multi-armed bandit problem arises in applications such as recommendation and distributed decision making, where the reward is determined by the maximum outcome among $K$ selected arms.