The paper introduces a new variant of the multi‑armed bandit problem that incorporates dueling feedback—pairwise comparisons of model responses—and heterogeneous sampling costs to identify the best large language model (LLM) from a set with varying query costs. Assuming a Condorcet winner, the authors propose a Track‑and‑Stop style algorithm that guarantees asymptotically optimal cost as the error probability approaches zero. Extensive experiments on synthetic and real‑world data show that this cost‑aware approach consistently outperforms both classical cost‑unaware algorithms and other cost‑aware extensions.
By Sarvesh Gharat, Nikhil Karamchandani, Jayakrishnan Nair
The paper introduces SELECT, an algorithmic framework for satisficing regret minimization in bandit problems, achieving constant expected satisficing regret when a satisficing arm exists. A variant, SELECT‑LITE, further ensures a light‑tailed satisficing regret distribution while maintaining constant expected regret in the realizable case and sub‑linear standard regret otherwise. Experiments on synthetic data and a real‑world dynamic pricing scenario demonstrate the practical effectiveness of both algorithms.
By Qing Feng, Tianyi Ma, Ruihao Zhu
The paper presents a new scaling law for reward optimization in AI alignment, showing that performance scales as Θ(√min{log(M), K}), where M is the number of preference comparisons used to train a proxy reward model and K is the KL‑divergence budget relative to a reference policy. The authors derive this law using an information‑theoretic model, prove its tightness, and validate it with extensive experiments involving a 70B gold reward model and smaller proxy models (0.6B–4B). The empirical results demonstrate a strong fit (R² 97–99 %) across different model sizes, noise levels, and optimization methods, suggesting that reward optimization behaves like a simple selection task over IID Gaussian variables with noisy feedback.
By Ali Aouad, Aymane El Gadarri, Vivek F. Farias
arXiv:2607. 14706v1 Announce Type: new Abstract: We design and analyze \underline{M}echanism-\underline{E}nforced \underline{S}equential \underline{HA}lving (MESHA), an algorithm for Best Arm Identification (BAI) in strategic linear bandits.
By Xin Li, Zixin Zhong
arXiv:2605. 09448v2 Announce Type: replace Abstract: We study the operational problem of automated bidding in repeated first-price auctions under budget and return-on-spend (RoS) constraints.
By Zihao Hu, Yuxiao Wen, Yuan Yao, Jiheng Zhang, Zhengyuan Zhou
arXiv:2608. 06545v1 Announce Type: new Abstract: Distributionally robust Markov decision processes provide a principled framework for sequential decision making under model uncertainty.
By Yuepeng Yang, Yuxin Chen, Yuejie Chi
arXiv:2606. 14690v1 Announce Type: new Abstract: We study a \emph{max-risk} objective for active learning in a multi-group mean estimation $d$-armed bandits: a learner adaptively allocates a budget of $T$ samples across $d$ groups to minimize the worst-case uncertainty index $\max_{k\in[d]}\sigma_k^2/n_k$, where $\sigma_k$ is the standard deviation of the distribution of arm $d$, and $n_k$ is the number of times arm $d$ is sampled.
By Abdellah Aznag, Rachel Cummings, Adam N. Elmachtoub
arXiv:2606. 30923v1 Announce Type: cross Abstract: Imitation Learning is a natural framework for learning in sequential decision-making systems and has emerged as the dominant paradigm through which we understand language model training.
By Ved Sriraman, Peihan Liu, Daniel Hsu, Adam Block
We design and analyze \underline{M}echanism-\underline{E}nforced \underline{S}equential \underline{HA}lving (MESHA), an algorithm for Best Arm Identification (BAI) in strategic linear bandits. In this setting, each arm may strategically misreport its feature vector to maximize the probability of being identified as the best arm, when rewards are generated from the arms' true but unobservable features.
arXiv:2606. 01799v1 Announce Type: new Abstract: We study $N$-armed stochastic dueling bandits under the Condorcet-winner assumption, where three widely adopted objectives are considered: best-arm identification (BAI), weak regret, and strong regret.
By Pu Wang, Yao-Xiang Ding
arXiv:2608. 04324v1 Announce Type: cross Abstract: This paper studies generalized low-rank matrix bandits with multiple prioritized objectives.
By Bo Xue, Ji Cheng, Haodong Jing, Hongzong Li, Shuang Qiu
arXiv:2606. 01081v1 Announce Type: new Abstract: Decision-focused learning (DFL) trains predictive models by optimizing downstream decision quality rather than standalone prediction accuracy.
By Wyame Benslimane, Tinghan Ye, Pascal Van Hentenryck, Paul Grigas