arXiv:2609.13564v1 Announce Type: new
Abstract: We study KL-regularized contextual bandits under both reward and preference feedback. We show that greedy sampling can achieve logarithmic regret witho...
By Zichen Wang, Haoyang Hong, Huazheng Wang
arXiv:2605. 09454v2 Announce Type: replace-cross Abstract: We study the $\textit{single-index bandit}$ problem, where rewards depend on an unknown one-dimensional projection of high-dimensional contexts through an unknown reward function.
By Devdan Dey, Sujoy Bhore, Avishek Ghosh
The paper introduces conservative contextual dueling bandits, which enforce that each displayed item’s cumulative score remains above a specified fraction of a trusted baseline’s score. It presents algorithms—MaxInP-BC for linear utilities and BTIGW-BC for general utilities—that guarantee safety with high probability while achieving sublinear regret. Experiments show these methods satisfy the safety constraint more reliably than unconstrained counterparts, with only a moderate increase in regret.
By Rohan Deb, Mohammad Ghavamzadeh, Arindam Banerjee
Contextual dueling bandits learn from pairwise comparisons, the same feedback used to fine-tune language models from human preferences. Standard algorithms explore aggressively and can show users two...
arXiv:2606. 28616v1 Announce Type: new Abstract: In stochastic linear bandits, the canonical Upper Confidence Bound (UCB) algorithm admits a simple frequentist regret analysis but can be computationally demanding, while Thompson Sampling (TS) is computationally attractive yet typically harder to analyze due to its non-optimistic nature.
By Toshinori Kitamura, Shuai Liu, Csaba Szepesv\'ari
The paper studies the trade‑off between maximizing rewards and accurately estimating current treatment effects in nonstationary multi‑armed bandits. For $K$‑armed bandits with H"older‑smooth mean rewards, the authors derive minimax bounds on expected dynamic regret under a high‑probability simultaneous coverage requirement for all current effects. They propose the TRACE algorithm, which achieves a regret–precision trade‑off that separates reward‑learning regret from the unavoidable cost of refreshing current‑effect estimates, and extend the method with an empirical‑Bernstein e‑process to adapt exploration while preserving worst‑case guarantees.
By Sharan Sahu, Martin T. Wells
arXiv:2602. 09456v2 Announce Type: replace Abstract: We propose an algorithmic framework, Offline Estimation to Decisions (OE2D), that efficiently reduces contextual bandit learning with general reward function approximation to offline regression.
By Hao Qin, Chicheng Zhang
arXiv:2607. 08979v1 Announce Type: new Abstract: We study the active learning problem of fixed-confidence top-$k$ identification from noisy pairwise comparisons.
By Motti Goldberger, Nils Rudi
The paper introduces Non-Parametric CPT Bandits (NPCB), an optimistic algorithm for bandit problems where arm rewards are evaluated using Cumulative Prospect Theory instead of expected value. It provides regret bounds that match classical non‑parametric bandits, showing that incorporating risk sensitivity does not increase order‑wise regret. Extensions to unknown noise distributions and kernel‑based CPT bandits are also presented, with regret guarantees that align with risk‑neutral kernel rates up to logarithmic factors.
By Devdan Dey, Sujoy Bhore, Avishek Ghosh
The paper presents the first high‑probability regret bounds for both cumulative and current‑step variants of Feel‑Good Thompson Sampling (FGTS) in contextual bandits with general realizable function approximation. The bounds rely on decoupling coefficients and prior mass near the true reward function, and in the linear setting they achieve ≠d√T log(2T)+√T log(1/δ) regret with probability at least 1‑δ. This matches the dimension and horizon dependence of upper‑confidence‑bound algorithms up to logarithmic factors, while the high‑probability cost adds only linear dependence on log(1/δ).
By Xuheng Li, Qingyue Zhao, Quanquan Gu
The paper investigates best‑arm identification (BAI) in multi‑objective bandits, showing that additional objectives can actually reduce the number of samples needed. For Gaussian bandits, a characteristic‑time lower bound is derived that applies uniformly across preference orders, and explicit comparisons under Pareto, linear, and lexicographic preferences reveal that a larger optimal set can be identified with fewer samples, that the benefit of vector feedback depends on the preference, and that multi‑objective BAI can outperform the easiest single‑objective case. The authors also introduce Order‑Aware Track‑and‑Stop (OATS), an algorithm that achieves the lower bound while guaranteeing correctness at any desired confidence level.
By Bo Xue, Ji Cheng, Yuanyu Wan, Shuang Qiu
arXiv:2609.37209v1 Announce Type: new
Abstract: Pairwise losses are increasingly used for reward learning even when pointwise rewards are observed, with mixed empirical results. When and why do pairw...
By Junghyun Lee, Minsoo Ha, Sanghwa Kim, Yeongjong Kim, Eunjee Lee, Seiyun Shin, Kwang-Sung Jun