arXiv Machine Learning

Pointwise or Pairwise: When Do Pairwise Losses Help Reward Learning, Provably?

arXiv AI
Oct 1

On the Complexity of Preference-Based Bandits

The paper investigates preference-based bandits where a learner selects pairs of arms and receives binary preference feedback modeled by Bradley–Terry. It introduces the locally sensitive eluder dimension, a new complexity measure for logistic preference feedback, and proposes the GINOP algorithm that uses log-loss confidence sets to balance optimism and exploration. The authors prove a first-order regret bound showing that learning with preference feedback can be as statistically efficient as learning from direct rewards, and they validate their theory with empirical experiments.

By Ahmed Ben Yahmed (CREST, ENSAE Paris, FAIRPLAY), Marc Abeille (FAIRPLAY), Cl\'ement Calauz\`enes (FAIRPLAY)
arXiv Machine Learning
Sep 10

Improved Dimension Dependence for Bandit Convex Optimization with Gradient Variations

The paper presents an improved analysis of non‑consecutive gradient variation in Bandit Convex Optimization (BCO) with two‑point feedback, leading to better dimension dependence for both convex and strongly convex functions compared to prior work. It also derives new problem‑dependent guarantees such as gradient‑variance and small‑loss regret bounds, extends the technique to one‑point bandit linear optimization over hyper‑rectangular domains, and establishes the first gradient‑variation dynamic and universal regret bounds for two‑point BCO.

By Hang Yu, Yu-Hu Yan, Peng Zhao
arXiv Machine Learning
Oct 22

The Price of Current-Effect Inference in Nonstationary Multi-Armed Bandits

The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.

By Sharan Sahu, Martin T. Wells
arXiv Machine Learning
Sep 16

Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Bandits

Meta-LinEXP3 is an online-within-online algorithm designed for adversarial linear contextual bandits with random action sets. It builds a task-level prior from completed tasks to guide an inner LinEXP3 learner, achieving an σO(√n) per‑task regret when context distributions are known and an σO(n^{2/3}) regret with a past‑only regularized moment estimator when they are unknown. The paper also links prior accuracy to transfer regret, showing that better priors yield sublinear, transfer‑dependent regret across tasks, and demonstrates the method on structured hyperspectral tensor sampling.

By Hao Li, Jie Xu, Zheng Xie
arXiv Machine Learning
Oct 23

Eluder-Dimension-Free Regret for Stage-Wise Constrained Contextual Bandits via Offline Regression

The paper introduces CS-UCCB, an algorithm for stage‑wise constrained contextual bandits that observes only the reward and cost of the chosen action. By combining offline regression, cost confidence sets, and upper counterfactual confidence bounds, it achieves feasibility at every round and a regret of ≥O(√T) without relying on an eluder dimension, provided the reward and cost classes have polylogarithmic covering entropy and low offline estimation error. The authors also extend their results to infinite action spaces under a reward‑design condition and compare their bounds to existing eluder‑dimension‑based guarantees.

By Haichen Hu, David Simchi-Levi, Navid Azizan