arXiv AI

On the Complexity of Preference-Based Bandits

The paper investigates preference-based bandits where a learner selects pairs of arms and receives binary preference feedback modeled by Bradley–Terry. It introduces the locally sensitive eluder dimension, a new complexity measure for logistic preference feedback, and proposes the GINOP algorithm that uses log-loss confidence sets to balance optimism and exploration. The authors prove a first-order regret bound showing that learning with preference feedback can be as statistically efficient as learning from direct rewards, and they validate their theory with empirical experiments.

arXiv AI
Oct 28

Conservative Dueling Bandits: Preference Learning under Baseline Constraints

The paper introduces conservative contextual dueling bandits, which enforce that each displayed item’s cumulative score remains above a specified fraction of a trusted baseline’s score. It presents algorithms—MaxInP-BC for linear utilities and BTIGW-BC for general utilities—that guarantee safety with high probability while achieving sublinear regret. Experiments show these methods satisfy the safety constraint more reliably than unconstrained counterparts, with only a moderate increase in regret.

By Rohan Deb, Mohammad Ghavamzadeh, Arindam Banerjee
arXiv Machine Learning
Jun 30

Randomized Exploration for Linear Bandits via Absolute Perturbations

arXiv:2606. 28616v1 Announce Type: new Abstract: In stochastic linear bandits, the canonical Upper Confidence Bound (UCB) algorithm admits a simple frequentist regret analysis but can be computationally demanding, while Thompson Sampling (TS) is computationally attractive yet typically harder to analyze due to its non-optimistic nature.

By Toshinori Kitamura, Shuai Liu, Csaba Szepesv\'ari
arXiv Machine Learning
Oct 22

The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits

The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.

By Sharan Sahu, Martin T. Wells
arXiv Machine Learning
Nov 5

Risk-Sensitive Non-Parametric and Kernel Bandits

The paper introduces Non-Parametric CPT Bandits (NPCB), an optimistic algorithm for bandit problems where arm rewards are evaluated using Cumulative Prospect Theory instead of expected value. It provides regret bounds that match classical non‑parametric bandits, showing that incorporating risk sensitivity does not increase order‑wise regret. Extensions to unknown noise distributions and kernel‑based CPT bandits are also presented, with regret guarantees that align with risk‑neutral kernel rates up to logarithmic factors.

By Devdan Dey, Sujoy Bhore, Avishek Ghosh
arXiv Machine Learning
Oct 15

High-Probability Regret Bounds for Feel-Good Thompson Sampling

The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).

By Xuheng Li, Qingyue Zhao, Quanquan Gu
arXiv AI
Oct 15

When More Objectives Make Identification Easier: The Role of Preferences and Feedback

The paper investigates best‑arm identification (BAI) in multi‑objective bandits, showing that additional objectives can actually reduce the number of samples needed. For Gaussian bandits, a characteristic‑time lower bound is derived that applies uniformly across preference orders, and explicit comparisons under Pareto, linear, and lexicographic preferences reveal that a larger optimal set can be identified with fewer samples, that the benefit of vector feedback depends on the preference, and that multi‑objective BAI can outperform the easiest single‑objective case. The authors also introduce Order‑Aware Track‑and‑Stop (OATS), an algorithm that achieves the lower bound while guaranteeing correctness at any desired confidence level.

By Bo Xue, Ji Cheng, Yuanyu Wan, Shuang Qiu